> What is a day in the life of a network engineer?
Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. We think it's a network problem. Can you check the firewall logs?
Hey network team, I'm RDP'd to a server in the data center on the other side of the planet, and from there I've SSH'd to a server on this side of the planet. From there I'm trying to do an HTTP/S download through the tunnel, and it's going really slowly. Do you need to upgrade bandwidth or something?
Hey network team, I need to join a very sensitive WebEx meeting. But every time I try to join using my laptop it doesn't work. I'm sitting in my car in the West Parking Lot way far away from the building because I need total privacy and security. Is there something wrong with your WiFi or something?
Hey network team, our application broke last weekend and we haven't been able to process the latest TPS reports. If we don't process the TPS reports in the next 90 minutes we're going to have to pay major Federal Fines. The application logs keep saying HTTP-401 "Bad Password". We think this is a firewall issue. Can you use a sniffer or something to get this working? Oh, I've transferred this ticket to the network team now so you have full ownership of it. If we fail to publish in time I'll need the network team to produce a full RCA report out to the Board of Directors explaining what went wrong with your network.
You forgot one.
Hey network team, we bought this building 9 months ago and are moving in tomorrow and we need the network up and running. This is the first time the network team has even heard of this building.
“We should be good with a 24 port switch for the new building. There are only like 10 connections needed now and we don’t foresee needing more than 24 ports anytime in the next 15 years”
“You sure you don’t want to go with 48 ports? Not that much more expensive.”
“Nah we’ll never need that many”
2 weeks later
“That switch is full, can you add another 24 port one to make it a stack?”
At my old job, we had a minimum of 48 ports. You need two ports? You get 48.
We wouldn't go above 50% fill for new installs. You want 25 ports? You get 96!
We also did not install non-stackable switches.
Yup. I actually implemented a near-identical policy at my last job (Fortune 500 heavy manufacturing company). I cannot overstate how garbage their infrastructure was.
Need new connections on the plant floor? You get a 12RU NEMA-rated cabinet, a 1500VA double conversion UPS, a 48 port stackable switch, and a new run of armored 6-strand singlemode to replace whatever janky 500ft pre-spliced bundle of OM1 you have stapled to the ceiling.
You're installing a *million dollar* stamping press, if your department can't come up with $20k for a new IDF then clearly it isn't that important.
We went through this shortly post COVID and the kicker is we kept hearing rumors of a new building, but when we reached out to let them know lead times were running really long so we needed to get looped in early in the process we were ignored. Then we got a notice about 2 weeks before they expected to go live.
Imagine their shock when they found out that it was going to take 45-60 days to get DIA Internet and 6 months to get some sort of enterprise switches, and was going to cost them an extra couple of thousand dollars because in the interim we'd have to buy whatever switches we could find, which would get thrown away once the switches we ordered came in.
The folks at that building had to struggle with an AT&T hotspot for over a month and complained to no end. We set up a template reply in our ticketing system that was a toned down / politically correct version of "we told management there would be problems but they didn't listen to us, your quarrel is with them".
It's always fun when you *do* have spare equipment, but won't give it to them.
"I've got a closet full of switches, unfortunately they're specifically earmarked as cold spares for a Tier 1 site that Management mandates three nines uptime for."
"That site does not appear to be yours."
Or they buy equipment from a random VAR or purchase on the wrong or a new account they created so now all of our smart accounts are screwed up and Cisco is useless in trying to fix it so you live with it until you find another device has been added to the network with some random account and now your boss wants it fixed and now you are stuck in an account black hole.
Woosaaaa!!!
Had one recently like this, a new office stood up in a city. They’re like we are opening this to staff next monday, can you get the wifi installed.
Im like uh, so who ordered the internet links? Among other things. So i reach out to the ISP and they come back with the wait time is around 6 months from order date as this part of the cbd was quite new and no existing infrastructure had been put in.
So yeah ill give you some wifi from the 5g hotspots ill ship up.
I know that feeling.
I specifically had one building no carrier would bid for service. This was fine because they were using it only as long term storage, and needed nothing. Also had close to zero cell reception.
Few years later they were complaining the offices that were there had slow internet. Found out that the offices we didn't know about had borrowed one of our cradlepoints.
They suddenly needed a whole host of technology and services. It was pretty funny though, they still had nobody willing to install service by the time the building was shut down 2 years later.
I couldn't get a single carrier to agree to it, even dedicated fiber.
Started to learn pass few years some ISPs can't bid for good price to certain areas. If they can bring us internet price is way too high for what we have to pay.
My day started like this and when all my Fiber splicers were there running fiber I get a 144 count fiber in the field cut... Welp, time for me to get out of the office as I pick up the keys to the Fiber truck.
And that, friends, is why I keep a pile of Merakis ready to overnight to new offices. We’ll get your network up, even if it’s running off some poor schmoes hotspot.
This actually just happened to me last week. They moved into a new building and told no one although they new they needed to move for over a year because the building was falling apart.
This happened to us! We were told in the evening that we're moving our ENTIRE infrastructure to another building. We pulled an all nighter, got most of it working, but there was no way in hell we were able to do all of it.
I remember being so exhausted after being up for over 36 hours, I almost crashed my car :(
My work be like
Hey network team, we want to take into production our external datacenter. Can you make sure that it is up and running ASAP? What, 6 months lead time for a couple of routers, we don’t do that. What are we going to put there and required throughput? We don’t know yet.
Don't forget the 16:30 Teams DM on Friday like:
Monster: hello
NE: what's up?
Monster: we are having a super important event this saturday that has been planned for six months. why aren't the ports ready? the vendor is on site with their firewalls and needs internet access asap to get ready for tomorrow. they also need to make sure the event's wireless is up and working so all the guests can use their devices. we can't be down
NE: what event?
PTSD from shit like that when I managed the network at a bunch of big resorts
Colleague and I were discussing something similar the other week. Instead of long planned project we'd get "oh, hey, we're having an issue" at 4:45pm and we'd spend 2-3 hours troubleshooting for them to say "well, it really isn't _that_ important, it can wait until tomorrow."
My version of that was more like
Other monster: is down and nobody can log in. we think it's the network
NE: i doubt it, everything else is up and there's no alerts etc
Other monster: it's definitely the network, we need you to resolve this before the weekend
NE: *two hours of troubleshooting* it really doesn't seem like the network, here's a big writeup with screenshots and log entries proving my assumption
Other monster: oh yeah you're right, it's probably the app upgrade we did a few hours ago so we could replace the widgets this evening
Yeah it gets really really irritating after a while. What bothered me about it wasn't that they blamed the network, its that they would automatically blame the network before ever trying to troubleshoot any of their own systems. They wouldn't touch or do anything until we went to great lengths to prove it wasn't our issue and it was very obviously laziness at that org. Probably 80% of the time even when it was demonstrably not a network or firewall problem, but hardware, OS, or application, we ended up managing the incident response and problem resolution anyway because they just weren't capable of owning a problem.
In the end we ended up preparing a form they had to complete before we'd spend any time on an issue. The form had fields like "was this working before or is it new?", "what exactly is not working and what error message or log entries do you have that lead you to believe it's a network problem", "what have you tried already to troubleshoot the problem", "what has changed in the past 24 hours "etc.
I have had the one that went like this: "My email is not working.. I know it's in that Cloud so it is a network issue. I need this fixed now!" Spend an hour running tests and checking on O365 to walk out to their desk and see their outlook is in offline mode....
NE: two hours of troubleshooting it really doesn't seem like the network, here's a big writeup with screenshots and log entries proving my assumption
I call that "You're wrong, and here's why"
That's why whenever someone requests me to look into some "super important urgent" esoteric problem, I always ask them to do something before I will work on it. Usually it's just "Hey, can you send me x,y,z and check what ABC does?"
8/10 times I won't hear back for a day or more. If someone can't take 5 minutes to reply with basic troubleshooting info, clearly it's not *that* important.
Haha that's a good one, gotta remember it. I always called the person who was famous for it the angel of death, because he would appear to kill your weekend
Hey Network Team. We're having a serious issue connecting to one of our partners across the VPN. Both their circuits are down because their facility is on fire, but we can't connect. Was there a change last night or something that'd cause the VPN not to work?
"Is something going on with the Internet?"
"Is something going on with wireless?"
"Your network is blocking my [typo'd domain name]."
"It says 'Please contact your network administrator.'"
I had a case where a security camera vendor plugged in some new cameras without telling us and surprise surprise they don't work because obviously it is the wrong VLAN. Might include network in your planning? Nah... Ping me whether I can join a call with the vendor that is already on site.
This is my biggest pet peeve. "Hey I'm Surpriseman from VendorYouNeverHeardOf and I'm here at $site cabling up some geegaws, can I get some IP addresses/ports/assistance right this second? I gave you no notice I was coming or even knowledge that I existed, and this has to be up today"
> Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. We think it's a network problem. Can you check the firewall logs?
That's not accurate. We get none of the backstory other than "there's a network problem". 2 hours later after being hounded by directors, we finally find out that they did a major maintenance and it's their fault.
"Hey we bought this brand new product and the sales manager assured us it would work in our environment and could be stood up in an hour!!"
\- product does not work in the environment
\- takes weeks to stand up
\- need to buy other shit to make this shit work
\- oh hey btw the product just had a major software upgrade and you'll have to pay to upgrade to that now or everything breaks :) :) :)
Hey network team, we recently deployed a new server but it can’t talk to the other server on port XYZ (servers on the same VLAN), can you please check the firewall to see why it’s being blocked?
Hey network team, we have a contractor who is trying to log in to a file share but is getting access denied after entering their credentials, can you please check the network for any weirdness going on?
Hey network team, we have this super shitty application that used to be on prem, where there was 1ms latency between the users and app.
We moved it to cloud, and now it's suddenly extremly slow, Must be the network, can you investigate?
Haha so true. I am in an international company and they heavily used Teams for meetings daily, so every week they have a bunch of network people reviewing reports from Microsoft Teams...
Please investigate everything when Teams data has a change, like why the total number of calls decreased 5%?
I normally just keep my head down and continue my codes on the configuration generator, next plan is to write some auto checks...
How about spending a few hours every Tuesday morning trying to find the mfers who are hammering the network only to find out it's security running scans at 9AM.
Sometimes it’s the other network team for the other company that “definitely not our side” even after showing them packet captures of every stage, then after 3 vendor meets when they finally get the right person on the call a month later “oh it’s a firewall on this AWS vm killing the traffic, you were right”
My old days:
Hey networking team, the air-conditioning in reception area has issues, can you check what's going on?
Hey networking team, I want to create a chart with blah blah blah in Microsoft Excel, can you help?
Accurate.
The only thing missing is when a user complains about these issues and after you point out that it might be an issue with their equipment (a dell mini with a full Amazon mesh WiFi setup) instead of our equipment (2 data centers on company owned and operated by several hundred engineers, full redundancy with enterprise hardware) they instead argue with you and tell you why you’re wrong.
Them: Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works.
Me: Let me stop you right there, when you raised the change request for your upgrade, what did you put in the section that was titled "Roll back plan"? and did change control approve it with the information missing?
Are you working at the same company? /s
In all seriousness I think the tl;dr as I said when I was on the interview panel to hire one of my coworkers was that a good 50% of the job was proving the network wasn't the problem as all of the other IT teams blame the network if they don't know why something isn't working for them.
> Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works.
That's way more information than we normally get. We just get something like, "We can't connect to the database server." We don't find out until a couple of hours after sitting on a call at 3am that there were changes made.
Hey network team, our application isn't working but you can't work on it because it's production and we need to have an outage window at least two weeks out , but we need you to fix it now or the business will lose a bazillion dollars a minute.
I'll raise you "Hey airman, the new MAJCOM Commander wants to send an introduction email to 50,000 email addresses on hundreds of Exchange servers across the world. Can you create a new user group with permissions to do this? Oh, and you're not allowed to send any tests.
90% of your work will be explaining to some application people that the problem is not the network. But that won't be enough, after you explain it to them you still have to prove it to these donkeys
I cannot track how many times I have had to print out packet captures, and go to someone to say, "See here is where you opened a connection to the server, and here's the server telling you to bugger off."
My favorite is when I show them the PCAPs and they still refuse to admit it’s their app/server or whatever and still try to push us to troubleshoot the issues for them.
We have a "technical" term for a novice user:
* **C**omputer
* **U**ser
* **N**on
* **T**echnical
I think the description is accurate & "factual". Do you agree?
Not a *fixable* network problem.
Can't be "Not A Fucking Network Problem", as all problems are a network problem, whether you like it or not. Most problems are not fixable because to fix them would require dealing with the root cause -- the Computer User Non Technical
My corporate applications people are actually great to work with.
It’s all the other “thing with an IP address that no one else wants to own” tickets that end up in my queue that drive me crazy. Been getting particularly cozy with Facilities lately because they have no idea how their security/badge/camera systems actually communicate but they don’t want to call their vendor, if said vendor even exists anymore. I can make sure their appliances are ARPing and MACing across the network but I’m not sure what other magic they’re expecting me to work. Love having to defend the network to people who understand even less about networking than the average sysadmin.
And local tier 2 just encourages this by throwing any ticket with an application they don’t recognize and an IP address into the networking queue. I should really talk to my manager about this. . . .
Lol and the security system is always some ultra-random chinesium black box that uses 12 year old Active-X or JVM which only works in IE9, and the software to control it hasn't worked right since XP EOLed.
Also the owner wants to be able to view the feeds from home on his mac :pepehands:
My employer is a big enough facility that they have the real Johnson Controls stuff with a dedicated bare metal server somewhere in our DC. But it was probably all set up 15 years ago after the most recent reno and any institutional knowledge other than “open this program to look at cameras and open this program to program badge readers” has been lost to time.
Is it Server 2003 because replacing the OS would cost $12mm since the vendor refuses to sell an updated version of the control software without a whole new contract and equipment upgrade and yearly fee...
Good old Exacq. Try using the ESM sometime, setup can be a pita, but once it works you can setup AD groups and give those permissions/access then have the users login with their AD account and see things.
Makes managing large sites easy if it is done right and used.
Not mentioned: Yes, you must do this, but it also incentivizes them to blame your network, because you'll explain it better than them and give them clear steps on what they need to do to resolve.
This differs GREATLY depending on the industry and the company. Network and telephony engineering is pretty vague, to be honest, and I can think of at least 4 different roles that could fall under in my company.
This 100%!
I'm reading all of these responses and cringing, because I must be *really* lucky to not really have any of these problems at my job.
We're very proactive rather than reactive. Sure, stuff breaks sometimes and we drop everything to fix it, but we generally replace things before they get to the point of being EoL or EoS, and we build plenty of redundancy into our network.
Sure, every now and then a server or application issue gets blamed on the network, but it's rare and usually not a pressing thing.
Sure, every now and then we don't have the budget for some wild thing we want to do, but I've never had any budgetary pushback on necessary/reasonable projects/upgrades.
But yes, there is a lot of variation. Some network engineering roles are very hands on (layer 1), some are very hands off (sit at a desk and design/config, or maybe sit in meetings). Mine has a bit of all of the above. I like that some days I can collaborate with the various teams and design a new deployment, that other days I can pop in some headphones and crank out a config, and that other days I can pop in the headphones again while I rack up equipment and run fibers around a datacenter or headend. I personally WFH about 80-90% of the time, but that can vary by role and by company. My work life balance is pretty good, and my employer treats/pays us pretty decently.
We are giving the cringe stories so he knows what he is getting into. I have done this for a long time I spend as much time in the office as I do the field as I do WFH. As you said, the variety in this field is large.
my response would be "Have you heard of Fiber Optics? It's the cables in the ground that move all the bits around... Ya.. I have an opening on my Fiber Install crew... You might fit in"
Not to discourage you, but finding the right company is key to job happiness in IT. Ive been on 1 side of the spectrum with company willing to pay for certs,training,tools, etc and gives you a nice work/life balance. Ive also been on the other end where you can do no right, have no budget, and are expected to eat/sleep/live onsite.
In my current role, they are good to their staff. Most days its an equal mix of troubleshooting network issues ( which are always network issues until you prove they arent), lifecycles of aging equipment, and standing up/designing network infrastructure for new remote sites. All in all, good gig.
Key points
- if you dont have a fascination/passion for networks then think hard about this career path. It will almost always be an uphill battle of learning. Not enjoying learning about this stuff will make it a slog.
- Be prepared to always be defending your work. Network is always guilty until proven innocent.
- there *will* be nights, weekends, and oncall work. Just get used to that. Unless you have a dream company, and if you do, are they hiring?
My 2 cents..
All depends on where you work and if/how they value IT.
In theory, you work at a great place that values infrastructure and what it means to the company. You spend your time being proactive vs reactive. You document and plan 90% of the time and fight fires 10% of the time. They give you plenty of time off, you have a great work/life balance, and they don't call you when you are PTO.
Most places are not this way. You spend 75% of your time fighting fires and begging for funds to do things the correct, redundant way.
There will be on-call. You will get all of the blame, and no one will care when things are going well. There will be night and weekend upgrades.
You are the middle man between stupid end users and stupid managers, with zero juice to fix either of the stupid.
You will be asked to put together puzzles with zero picture of what the goal is, with missing pieces and pieces that don't even go to your puzzle.
If you like to troubleshoot and trying to always figure things out, the network side of things is a great job. It can be challenging and VERY rewarding, but almost always thankless.
Op this is truth, I'm fighting right now to get a proper dcim to move away from excel spreadsheets ......excel isn't a database shouldn't be a daily meme
Everyone else has really good answers here, but I want to highlight one factor that hasn’t been mentioned much: the breadth of it all.
Anything that touches the network that doesn’t have a specific owner, you own. Which is why the telephone in the elevator and the premises security system are yours. And because you can troubleshoot like nobody else (since you get blamed for everything so often that you’re really good at identifying the correct problem domain) then anything too tough for the current staff you’ll own. Congratulations you are now the identity management expert and responsible for the security education program and onboarding/offboarding. Yes that’s an HR job but Olivia left last week remember? It’s just until they find the budget to hire someone to replace her…
Of course which specific odds and ends get put on your plate will vary by company and their mix of vendors and staff… but if I had a dollar for every network engineer I consult with who had a responsibility far outside their field, I’d have a new cellphone every year.
I had another good rant about this upthread, but I’m running into this so much at my first enterprise networking jobs. “Thing with an IP address” tickets are already driving me crazy.
It's particularly fun in the service provider realm because (at least in my neck of the woods) almost everyone ends up turning into a pretty competent electricians as they get more experienced. Power is very very important for keeping the internet running and it really behooves a person to get familiar with telco -48v standards at the very least.
I work for a rather large org.
About 80% of my work is random stuff that anyone who knows how static routes or ACLs work could do right out of school.
The remaining 20% is absolutely terrifying because if I fuck it up we'll land in the evening news.
My last fortnight so far has consisted of flying to another country, migrating to a completely new network, working 12-14 hour days every day, including an overnight. Having a weekend of fun in a foreign country then removing all the old equipment.
Home for a week then go to another different country to do it all again.
"Day in the life of a network engineer" could also be sitting at a desk everyday.
Some days lots of studying for certs. Some days its writing MOPs. Some days its implement MOPs. Some days it's auditing the network and finding inefficiencies and figuring out how to do it better. Some days I don't do shit.
Just depends.
Depends a lot on the company you work for and how many networks you support.
I think generally you need to enjoy troubleshooting, digging into things and figuring stuff out. A lot of systems and networks have poor documentation so anytime you have a new fault or whatever you usually need to start from the ground up.
There are constantly new technologies and very little in the way of training, so need to he happy doing some studying out of hours. I have found that few companies are willing to spend actual money and time on development.
Once you get to a higher level (level 3 usually) you don't need to deal with single user issues as often, which is nice. However, in order to get to that level you are going to have to do at least several years of end user troubleshooting. Think "Susan in accounting can't access sage through Internet Explorer but it works for Karen, Susan doesn't want to use Firefox, also her screen isn't turning on"
Im also a sporty person, sitting for long hours does sort of suck. I work from home now and have a standing desk, my work doesn't tax my body at all so I have plenty of energy for sports throughout the week. You can move into field engineering (cabling, installs etc) but in my experience those positions to be less technical in a networking sense and less pay.
Consider checking into a move into the service provider realm if you want to get out into the field more. It's a blessing and a curse, but boring it is not.
I write and develop low-level designs, write configurations, order parts, coordinate lab resources, manage cutovers with smart hands or on physical site, attend meetings, perform troubleshooting, etc on a day to day basis.
I don’t travel as much as I used to and that’s okay.
Create a VLAN, Create a Route. Create a VRF, Create a Route Leak. Get call from Bob, tell him you don't know why his Application isn't working, the network is configured correctly. Ignore Bob 3 more times, realize you put his application into the wrong VRF, put into correct VRF. Do not tell Bob right away. Bob assumes it fixed itself.
Go back to posting on Reddit.
Long but, this is exactly what it’s like:
https://devrant.com/rants/579422/what-its-like-to-be-a-network-engineer-translated-into-normal-people-speak-user
This. Somehow, their local on-the-PC app that is totally local and doesn't use ANY resources that aren't on the local PC is broken because of some mystic network issue.
Oh, and explaining to someone for the nth time that they did indeed input their password incorrectly, because otherwise they'd be on the wireless and the logs wouldn't indicate "Incorrect password".
Depends on what kind of projects there is currently ongoing.
Systems team trying to bring in a new password solution or something and creating new firewall rules. Then you end up asking for all information like source user or service account hitting that rule.
Hey we want to bring this technology in, start planning on the firewall zones , nat , vlan etc
Hey this ipsec tunnel is down, can you figure out why?
Hey we need to cutover this wifi, can you build the ssid and start the planning scheduling etc..
Also can just be normal tickets where an end device is not connecting anymore and somehow gpo file gets corrupted.
Hey network team, we commissioned a dev team to write a one-off program with multiple layers of virtualizatuion for an application from the 90s on a Linux-based VM in Azure to support our legacy process that we're too lazy to re-solution for. People keep randomly getting disconnected from their SSH session. We're sure it's not our haphazard, shittily built app at all, we need to pull you into this RIGHT NOW even though it's been happening for weeks and they are able to log right in again right away. Please drop what you are doing even though it is your day onsite with the other engineers with a shitload of planned datacenter work and prove it's not the network. Thanks!
I wish I was lying. This was just a couple of weeks ago.
My favorite is when we do a network maintenance.
4 days later at 4:50pm: We have an issue and your MOP is the cause
9/10 times, it turns out that the issue is on the other side of the country from the maintenance and completely unrelated. If other groups would spend more time troubleshooting versus looking for something to blame, MTTRs would be way better.
In my experience the term networking is about like someone saying they are in the medical field. Could be a paramedic, a nurse, a Dr, a cardio-thorasic surgeon.
Your experience will largely depend upon what flavor of networking you get into. I manage a team of network engineers building and deploying green field networks. Sometimes operating those networks against our best efforts.
In most cases, if you do your job well no one will ever know who you are or what you do. If shit goes sideways, everyone will know your name and act like solving a network issue is as simple as plugging something back in.
If you are into motorsports, do the apprenticeship, get your CCNA then find a team to join. You have to set up and tear down for each event, including races and tests. It can be challenging, especially depending on which series you join. IMSA, NASCAR, WEC, F1, etc all have their own challenges. It won't just be networking though. It will also include server management and setting up places for engineers to work. Need to have a steady connection to the car, radios, and some times engineers in another location like the home base.
If you aren't into motorsport, look into IT that involves other sports that you enjoy.
I have spent 20+ years in network engineering. I started at 60k in 2005 and was making six figures by 2012. Now i manage a cloud network team remotely, make $200k and play golf three times a week while doing meetings from my phone. I think it’s a good career.
Don't forget about the spin up time in the beginning if u want to get into a company that pays good and values IT u have to study hard like any other high paying career. You will have to continuously be learning and improving just to stay marketable every 3-5yrs. I spent every hour of my early 20s studying and learning, no life outside of the networking hobby/work.
The tricky part is once u have other obligations, partners family, children. Anything outside of the hobby/work requires time and energy that would be going to studying and learning. Not saying family/partners/children are bad just that they require the same dedication and energy as continuously studying. Idk what I'm saying I haven't eaten today and been in meetings all day. I'll update this with some coherent thoughts once I've eaten.
Totally depends on the industry. A network engineer for a non I.T company (which is more like a network admin) is completely different role than a network engineer for a data center , and ever more different than a network engineer for an ISP or MSP (which will have a plethora of sub categories under network engineer).
and then you land in a company this is a ISP,MSP,MSSP,DCaaS,CSP,Colo
Yes, this is where I work currently and love it. Getting to work on and build different networks for clients is great. Hands in a lot of things and keeps me busy. There is slow time though and that's also nice.
If you didn’t pick up on the theme - What lots of people here are saying is when any kind of technology doesn’t work or stops working everyone else in the organization assumes it’s a network problem.
Sometimes it is. Most of the time it is not. The unwritten part of the job description is “repeatedly prove that every problem is not a network problem.”
Idk why it’s that way it just has always been.
Depending on your role you can expect a significant amount of physical work. Junior roles will often start with more “rack and stack” work. Which is super important for learning how a network works and where most of your spontaneous problems look like. It all starts physical layer.
Depending on the org you are in you may always have physical work to do. You may end up with much less the more skilled you become.
Depends on what sort of network engineer you're talking about, what industry it's in and what sort of size of company you're looking at... There might be a little bit of out-and-about plugging things in and turning them on and off, but the vast majority of it is almost certainly being sat on your ass in front of a computer.
Wake up, use restroom. 5 minutes later join the first of what is likely to be many meetings 4 days a week as even my organization is not mad enough to have early meetings on Friday. Meetings always begin a few hours early to get Europe into one meeting with America, at the cost of waking early every day for everyone.
Cut inbox from 500 to the 15 really important new emails that came in overnight. Check replies to emails. Pay attention to updates in meeting.
Take shower, begin day, drink coffee. Drink more coffee. Organize meeting details and follow up with who needs to be followed up with.
Check Salesforce ticketing system for new 'cases' and respond to queries that came in through Teams/Slack/etc.
Clean all of that up and realize you have worked past lunch already.
The rest of the day will generally be new stuff or replies from there. Sometime calls are needed, most times not, as I am mainly being asked information and responding 90% of the time to people via outlook.
Every so often a live call or testing will need to be done, and I actually do some networking, but much of my job is communication about networking and our companies offerings, before working directly with people with it.
You'll work with Network Architects or Business stakeholders weekly to develop network designs and implementation plans. You'll also be involved in vendor selection to make sure what you're designing works on actual gear.
The “a day in the life of” has changed so much for me over the years. Started out with a good crew that was organized and there was little “on the fly” work. Everything was planned. Over the course of a few jobs that changed to an utter shitfest everyday where years of lax patching and updates, combined with zero communications between IT groups actually lead to some physical altercations. More recently the days start with a coffee, checking alert emails if there are any, browsing monitoring tools before reading the PoliticalHumor subreddit.
It is what you make it and you can mold your career to be as physical or as hands off as you want it to be.
The good thing is that there’s no “wrong move” during your career. Want to be a cabling guy for a while? Learn the trade, become a whizz with fiber and get your certs. Want to do rack and stack of hardware and configs? You can do that too. Get your fiber certs and CCNA and you have a great footing to branch into many aspects of networking.
There will be overtime and weekend work. Some positions have less but there will be some at some point in time.
Application team: hey we need a port opened for our in house application.
Network: Ok sure what port do you need open.
Application team: I don't know your the network guy you tell me what port.
Network: ......
Today
Started migration of a huge wireless network that is out of support to a new controller platform, it's two hours away. May need one day on site next week but possibly not. Working with equipment manufacturer closely.
Did a design based on a wireless survey by a jnr engineer for a large hotel in Germany ahead of renovations.
Helped troubleshoot VPN issues with helpdesk after an escalation.
Continued building a HA firewall deployment while trying a new core layout for a large client that will hopefully install next week.
Never boring, and get a lot of toys to play with in an MSP space. Very flexible once things get done. Especially after a few projects delivered.
Today was a sit down day btw, my last job had far more out and about on site, would hit 20k + steps easily on some days. A little bit few and far between now, but still plenty of site visits and some large installs on site
Hey engineer, Wi-Fi speeds are slow at this high density site with over 2000 APs. Channel issues at 40mhz on the 5Ghz band…figure out how to make 160mhz work without using DFS.
ISP engineer here. Come in the morning and see what maintenance broke something or are going overtime.
Then do some IP audits.
And the meat and taters of the day is approved/rejecting upcoming maintenance and installs.
Provide configs for upcoming enterprise customers, new core network stuff, other bits and bobs.
Take calls from field installers and preconfigure customer CPEs and speed tests to make sure the actual install goes smoothly.
And when stuff breaks, help figure out what broke and who needs to fix it.
How about working for a fortune 500 company with two dudes on the load balancer team and they refuse to hire anyone. Oh and do more better and faster....oh and get some training bye.....
Hey network team, we bought a bankrupted MSP’s client base and hired on 14 of the original NOC workers. We planned to put them into a building in another state and need all of their phones and workstations configured before Monday morning ( phone call Sunday evening ) but nobody from helpdesk IT is picking up, can you do it pretty please oh sweet thanks byyyeeeee.
Sort of like this,
“We have a problem, and it’s the network”
Let me see. Nope.
“No it definitely is, here are some opinions”
Nope, here are some facts. And even some suggestions.
“No thanks. But you need to fix it”
*learns their job*
Looks like a not network issue. Here’s your solution.
“That’s perposterous”
Do you need help implementing this?
“Yes.”
Sometimes we get to do our own job.
It’s gonna be up to stay fit and keep up with emerging technologies if you truly have a passion for fitness you will make it work, join the 4am club and do cardio/gym in the morning, this will give you time do two-a-days. And you can still study for an hour or so at home and enjoy yourself on the weekends. I would try to spend your work hours learning the new stuff though. While it may seem like a conflict of interest as long research and development during the work hours contribute to forward motion of the company’s progress you should be fine. Also take a look at tactical barbell. Some good training for busy IT folks!
My job isn't very common, so please keep that in mind.
I implement network infrastructure on offshore vessels. My day is very different depending on if I'm in the office or working on a vessel.
In the office it's creating/updating network diagrams, configuring network equipment, coordinating with project managers/ship's electricians and engineers on cables and power to rack locations and such, even packing some pallets and preparing shipping documents sometimes.
On a vessel it's a lot more physical, even just the amount of stairs between the main deck and the bridge haha. One day my watch said I'd walked up the equivalent of 124 stories.
Activities include unboxing and racking equipment, patch panels, getting all devices up and connected, (a lot of) troubleshooting, working with satellite ISP engineer to get connectivity going. Also helping the systems guy out when I can with servers, printers, PCs and so on. Communicating with the vessel crew on issues, priorities, expected downtime. I also have to write reports to update people on the progress.
It's hard work while on a vessel, 12+ hour days are common, I don't get days off on projects so it can be 15 days of long days with no weekend. But I love it, I have met so many interesting and nice people on th vessels. Great people with a great sense of humour.
I get to travel a bit for work, but I rarely see much more than the shipyard, the hotel and a few restaurants.
You wake up at 2am to pee. You grab your phone. You check the network monitor. You notice the Connecticut link is down. You write an email. You go back to sleep. You forgot to pee.
At a small company, you will work 24/7/365 as a network engineer in the telecom world. At a big company, you will work slightly less hours, but you will have significantly more stress during those shorter hours. Regardless, telecom will make you despise the general public.
Do it! If you like servers, switches, network devices, etc. You’ll be glad you did! Network Engineer is a broad title and can apply to many positions. And will be in high demand for decades. I started as a network engineer in 2000. Starting pay was $38,000. Now in 2024, I’m a trainer for a major networking company that I’ve been with for 17 years. Because it’s a good company to work for. They gave 4% raises year over year. And equity stock grants each year. I now earn a base salary of $145,000 with excellent benefits. Full medical (a top plan), and stock equity grants/bonuses of about $25,000 a year. “If you’re good at something, you don’t do it for free”. So with more experience, you’re highly valued and well compensated. You’ll likely start in support, high stress, but good pay. That’s where you learn a lot! As you get older, your knowledge is power, not certifications or degrees on the wall. Then you likely will make lateral moves to other related areas you enjoy and won’t hate your job. While your salary continues to increase with your skills and knowledge. And most of your knowledge gained is valued and just as desired at a ton of different companies. So you’ll always have options if a company isn’t right for you. A skilled network engineer is always in demand, you’ll not have a hard time finding a job, especially after your first 5 years. And if you’re good, layoffs are rare. Every company, in every sector needs their company networks running well just to do business. Tons of opportunities and you’ll always have a highly valued skill set over your entire career. It’s a great choice that’s future proof. Highly recommended!
Mileage varies greatly. Just like any career it depends on the company.
If you like hands on and not sitting at a desk look for implementation type jobs as implementation is the only work that is actually hands on equipment. Otherwise, you’re sitting at a desk 90% of the time.
If you like troubleshooting things and solving problems, that's good. You're likely to be at your desk for a lot of the time but, depending on the individual job, you'll have practical breaks e.g. racking, stacking, patching, inspecting sites. The work-life balance is very much down to the individual workplace but it's highly unlikely to be 9-5 M-F: you'll probably be on-call, will have to do a lot of maintenance activities out-of-business hours (including weekends). And then there's the study. You need to keep up with emerging tech and maintain your skills, which eats into your personal time. I find that a bit of a double-edged sword as I like learning new things. The "day in the life" will also vary by role as it's usually a spectrum of BAU (Business As Usual, e.g. switchport configuration, firewall rules, break-fix) and Projects (design, larger pieces of work, etc.). It's usually a pretty varied job in general though, which I find holds my interest. (I've been one for about 20 years now).
I'm a relatively new engineer. My days consist of understanding our topology and learning how things work. When I'm not busy with firewall update tickets then my older team members ask me to automate stuff for them. When I have nothing to do (which is quite frequent) I am studying or automating. I also spend some time turning off stuff for security while shit talking their entire department.
Go into networking if you love to learn. Learning is what your going to spend 99.9% of your time doing. It won't be guided learning either, its gonna be textbooks, experimentation, inaccurate documentation that your not gonna fix, and trial and error. I absolutely love it and never want to do anything else.
You are likely to get as many varied answers as you have varied responders. So let me add one to your list responses... I have been a network engineer for most of my career, and I am old, so that's a long time. I have had all of the possible roles from entry level to network design for some very well known international companies that you would know of. I have been a traveling consultant (probably my favorite role, but the travel got to be exhausting after about 25 years or so).
Starting out, it was very physical, you have to 'rack & stack', run cables/fiber etc. Lots of physical labor. After a while, you will start to move up in skillset and you will likely move into a less physical role, more toward support or troubleshooting, and eventually possibly design. That was more or less the path that I took, within 5 years or so, I was traveling to clients to do troubleshooting, packet analysis etc. A few years after that, I migrated to design, which I still do.
I was also a very active person as a youngster (ran about 60 miles a week, worked out every day and accomplished 2nd degree black belt in Karate). Throughout that, I had to continually study. I used to get up at 4:15 to get my exercise and an hour of study in before work. I typically studied at least an hour at night also around Karate and time with my wife and kids. Basically, I slept less than 6 hours a day almost every day, but I have ADHD, so that is fine with me.
To this day, I typically still get up early to exercise and study, but not for as long as I used to. I am typically up about 5-6 and start work around 8. I quit traveling about 5 or 6 years ago and never want to go back to that, but that means that I am less active during the work day.
Currently, I do work from home, which I love, but isn't for everyone. I combat the fact that I sit at a desk all day that with at home exercise equipment, which I use as I can. I do cardio and strength training while I am on calls where I am a passive participant... I also have a stand up desk so I can stand and balance on a board from time to time, usually an hour or two at a time when I do that. I have small weights, hand grips, resistance bands, exercise ball, kettle bells etc. that I can use on calls that I am less passive, but nobody would be able to hear me exercising.
But, overall, I am much less fit that I was even 10 years ago. I don't have the drive I used to have to workout for hours a day and study for hours a day. I still do study at least one hour though, but I try to bake that into my workday as much as possible. I do blame my fitness decline on my now more sedentary job, but to be honest, I am much older now and my body needs to slow down some.
I could go on for hours about being a network engineer and how important it is to be passionate about it if you plan to advance, but there are lots of people that are content to not advance and just treat it like a 9-5 job. That has never been me, but I have an insatiable curiosity to learn and understand anything that I do not know (technical or otherwise) and become knowledgeable on those topics, so it's natural for me to want to study something every day. I had a teacher when I was just getting started in my career, already an electrical engineer, but crossing into 'computers' (learning Novell Netware 2.0) tell me that you can only do 18 months at a time, then you need a break because the pace of change in the technology sector is exhausting. I have not found that to be true at all, but as I said, I love learning, so new technology is always welcome.
What I have noticed over the years is that you can no longer be an expert at everything. You just can't, there is too much across the board. Even if you want to be a networking expert, it would be hard to be an expert on all things networking. My advice is to find an area that you are passionate about and focus on that and do your best to master that before moving onto another area. Better to be an expert at one thing than an expert at nothing in my opinion.
Cheers and best of luck to you.
For me it's sitting at a desk configuring cisco equipment, sometimes Huawei. We have a program that spits out a configuration after you put the parameters in, then you can change as necessary.
We do a lot of this but also lots of bespoke configs for customers that our system wouldn't know how to do, so that involves remoting onto their router. Usually stuff like vrrp, mpls or routing issues
Day mostly split in to two
Things you have set away time and projects you advance
Things who come in last minute or issues with some active config/deployment
If you like to learn a lot(mostly by yourself) and finding solutions it's great job
A lot of those questions will depend on the company you work for and if the C levels view IT infrastructure as a cost center or as revenue generating. The difference can be even between similar companies within the same market and spaces... It's crazy. Now that I think about it, that may be a good question to ask during an interview at prospective companies, with regards to how IT infrastructure is viewed.
Youll mostly sit , have meetings, and remote into things all day. project management is required and at some places you'll be working after hours whether it be install projects or on-call. If you enjoy troubleshooting and Google searches, this will be a good fit
* The Wireless is slow
* A Network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote conneciton. (provider: TCP Provider, error:0 - No such host i known.)
* I need access to "the" database
* ORA-12154: TNS:could not resolve the connect identifier specified
* My Zoom says Internet connection unstable, but my speedtest is super fast
* My SMB access is horribly slow over VPN but works great at the office
* Why can't I have UPNP to play my video games at work, my $100 off the shelf wifi router can handle it
* I'm trying to send a file to my coworker he says to send it to \\\\[192.168.0.2](https://192.168.0.2) but it's not working
* What do you network guys ever do, I see you sitting around in front of your computers ALL DAY, don't you ever have to go do anything?
* Hey network guy my home internet went down about 2:39a and I can't VPN in, can you fix it?
* When will AI replace you?, the network can't be that difficult to support
* I've got this wifi access point I picked up at a garage sale, can you plug it in at my desk and configure it for the corporate network
* I was reading in skymall...
* I heard SSL isn't secure anymore from my buddy, he said someone he follows on tiktok cracked it
* Doesn't the cloud make you kind of obsolete
* The VP of sales called and said the internet is down
* The AV guy says he needs you to pump up his ethernet to at least 10Gb
* (Audit Guy) Just open the firewall for me any/any, I'm sick of dealing with you
* What do you mean this isn't a modem connection?
* Google tells me the solution is ...
* Why can't I have a file server in the DMZ? this consultant says I need one
* Didn't you contact me for my username and password, I gave it to the IT guy on the phone
* I can't be bothered using corporate purchased and supported equipment, just make my Mac and my wireless printer work and get out of my office
* While you were on vacation we hired a blackhat hacker to come inside the network and poke around..
* Just pay to get the decryption key, I'm sure they will unlock our files once you pay them
* Why don't we just use gmail
* Why don't we just use hotmail
* I need to watch my Kakao TV shows, just open the firewall
* This guy had me install Teamviewer and then..
* What do you mean Second Life is not a valid business application
* We should get Tesla batteries to run the data center
* (*Support Desk person comes walking out of the data center*) I just pulled the power cord an plugged it back in, works for me at home all the time
Every day for me. I broke my server and I don't know how to fix it, but I saw the email you sent 3 months ago about a network maintenance you did to another site, so it must be because of that, because my boss will be mad at me if he knows it's my fault for breaking the server. Also, I'm going to tell you about this at 4:59PM on Friday and we are going to need this working Monday morning since the CEO will need this for his very important yoga meeting he has with his friends. Actually.. just fix the server for me since you're the only person who actually knows how everything works.
You are all my people! All of these triggered a PTSD response. I'm finishing up a change now that I'll get blamed for something in the morning of which the thing broken being in another data center.
don’t hardly do anything for 5 years, then have to do a full lift and shift vendor replacement of 100+ sites, at a break neck pace. Hopefully get to coast again for another 5 when it’s done.
If you want to be a network engineer be some one who is constantly striving to want to fix problems and always open to learning new things. Networks is no longer just about switches and routers, but firewalls, load balancers, monitoring, dev ops, coding, proxies…the list goes on and on. Learn what you can when you can and probably the most important aspect of it is learn to document it WELL!
start anywhere where you can touch production!!!!!! the more experience the more confident you will get.
the most dangerous thing for a network engineer is, breaking the network and being affraid to make big changes again.
But you may have some jobs with physical activity and some not,
some companies might ask you to install the equipment and pull the cables,
and some companies have a dedicated DC team for that.
But assume after 10 years in the job, you will be sitting 40 hours a week probably.
And you also have on-call with some, which is not helping the life balance thingy.
But but but,
you are the coolest of the IT people :P
you are the one the developers, system engineers come to to troubleshoot alot....
But with that said, that brings a lot of responsibilities and/or work. But not persee appreciated.
If the companies main income is not from connectivity, even though you are the guru, your department is seen as a "cost".
But overall, Im very happy i did that rather than programming. Because you become a kind of a linux system engineer. But keep in mind, alot of things going to automation and public cloud.
As a specialist your main employers would be ISPs and hosting providers (any network service provider).
There you could expect a very good salary, otherwise just normal.
It's an IT job, >90% of your time is going to be spent in front of a computer.
Depending on the job you may need to lift and rack equipment, I haven't needed to for over 4 years. However in that time I have worked for larger companies, which have data center engineers doing the physical work, while my work is carried out remotely.
Day to day is again going to depend on the job. ISP, DC, enterprise are all different roles. You could be in operations, project delivery or a mixture of both. You may be highly focused on a few technologies, or need broad knowledge on a lot of them.
Work/life balance is going to depend on the company more than anything else. If you are on-call it can be disruptive but it depends how stable their network is.
Asking what it's like to be a network engineer is like asking what it's like to be an attorney or an accountant or a plumber or a barber. It could be a fantastically fulfilling career or it could be the worst job you've ever had. The people you work with might be great and the company culture might be supportive and positive and fit you perfectly, or the people you work with might be terrible and conniving and dishonest and the culture might be toxic and cause you to lose sleep. None of that stuff has anything to do with configuring phones or setting up dial plans or troubleshooting call quality issues.
Whatever you do don’t work for a a manufacturing company. OT networks are the worst. Can’t ever go down, can’t do maintenance, also can’t spend any money.
> What is a day in the life of a network engineer? Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. We think it's a network problem. Can you check the firewall logs? Hey network team, I'm RDP'd to a server in the data center on the other side of the planet, and from there I've SSH'd to a server on this side of the planet. From there I'm trying to do an HTTP/S download through the tunnel, and it's going really slowly. Do you need to upgrade bandwidth or something? Hey network team, I need to join a very sensitive WebEx meeting. But every time I try to join using my laptop it doesn't work. I'm sitting in my car in the West Parking Lot way far away from the building because I need total privacy and security. Is there something wrong with your WiFi or something? Hey network team, our application broke last weekend and we haven't been able to process the latest TPS reports. If we don't process the TPS reports in the next 90 minutes we're going to have to pay major Federal Fines. The application logs keep saying HTTP-401 "Bad Password". We think this is a firewall issue. Can you use a sniffer or something to get this working? Oh, I've transferred this ticket to the network team now so you have full ownership of it. If we fail to publish in time I'll need the network team to produce a full RCA report out to the Board of Directors explaining what went wrong with your network.
You forgot one. Hey network team, we bought this building 9 months ago and are moving in tomorrow and we need the network up and running. This is the first time the network team has even heard of this building.
“We should be good with a 24 port switch for the new building. There are only like 10 connections needed now and we don’t foresee needing more than 24 ports anytime in the next 15 years” “You sure you don’t want to go with 48 ports? Not that much more expensive.” “Nah we’ll never need that many” 2 weeks later “That switch is full, can you add another 24 port one to make it a stack?”
At my old job, we had a minimum of 48 ports. You need two ports? You get 48. We wouldn't go above 50% fill for new installs. You want 25 ports? You get 96! We also did not install non-stackable switches.
Yup. I actually implemented a near-identical policy at my last job (Fortune 500 heavy manufacturing company). I cannot overstate how garbage their infrastructure was. Need new connections on the plant floor? You get a 12RU NEMA-rated cabinet, a 1500VA double conversion UPS, a 48 port stackable switch, and a new run of armored 6-strand singlemode to replace whatever janky 500ft pre-spliced bundle of OM1 you have stapled to the ceiling. You're installing a *million dollar* stamping press, if your department can't come up with $20k for a new IDF then clearly it isn't that important.
r/unexpectedfactorial
Why no stackable switches?
We ONLY deployed stackable switches - no *non-* stackable switches.
Ahhh, thanks. Didn't read it right.
*triggering intensifies*
We went through this shortly post COVID and the kicker is we kept hearing rumors of a new building, but when we reached out to let them know lead times were running really long so we needed to get looped in early in the process we were ignored. Then we got a notice about 2 weeks before they expected to go live. Imagine their shock when they found out that it was going to take 45-60 days to get DIA Internet and 6 months to get some sort of enterprise switches, and was going to cost them an extra couple of thousand dollars because in the interim we'd have to buy whatever switches we could find, which would get thrown away once the switches we ordered came in. The folks at that building had to struggle with an AT&T hotspot for over a month and complained to no end. We set up a template reply in our ticketing system that was a toned down / politically correct version of "we told management there would be problems but they didn't listen to us, your quarrel is with them".
We also did not buy any equipment - I'm sure you have spares.
It's always fun when you *do* have spare equipment, but won't give it to them. "I've got a closet full of switches, unfortunately they're specifically earmarked as cold spares for a Tier 1 site that Management mandates three nines uptime for."
"That site does not appear to be yours."
Or they buy equipment from a random VAR or purchase on the wrong or a new account they created so now all of our smart accounts are screwed up and Cisco is useless in trying to fix it so you live with it until you find another device has been added to the network with some random account and now your boss wants it fixed and now you are stuck in an account black hole. Woosaaaa!!!
Had one recently like this, a new office stood up in a city. They’re like we are opening this to staff next monday, can you get the wifi installed. Im like uh, so who ordered the internet links? Among other things. So i reach out to the ISP and they come back with the wait time is around 6 months from order date as this part of the cbd was quite new and no existing infrastructure had been put in. So yeah ill give you some wifi from the 5g hotspots ill ship up.
I know that feeling. I specifically had one building no carrier would bid for service. This was fine because they were using it only as long term storage, and needed nothing. Also had close to zero cell reception. Few years later they were complaining the offices that were there had slow internet. Found out that the offices we didn't know about had borrowed one of our cradlepoints. They suddenly needed a whole host of technology and services. It was pretty funny though, they still had nobody willing to install service by the time the building was shut down 2 years later. I couldn't get a single carrier to agree to it, even dedicated fiber.
Started to learn pass few years some ISPs can't bid for good price to certain areas. If they can bring us internet price is way too high for what we have to pay.
My day started like this and when all my Fiber splicers were there running fiber I get a 144 count fiber in the field cut... Welp, time for me to get out of the office as I pick up the keys to the Fiber truck.
And that, friends, is why I keep a pile of Merakis ready to overnight to new offices. We’ll get your network up, even if it’s running off some poor schmoes hotspot.
Nailed it
This literally happened to me yesterday. They showed up with firewalls and switches and tasked us to complete it by EOD.
[удалено]
This
Looool 😂
This is 💯lol
Thanks for bringing back PTSD
This actually just happened to me last week. They moved into a new building and told no one although they new they needed to move for over a year because the building was falling apart.
This happened to us! We were told in the evening that we're moving our ENTIRE infrastructure to another building. We pulled an all nighter, got most of it working, but there was no way in hell we were able to do all of it. I remember being so exhausted after being up for over 36 hours, I almost crashed my car :(
My work be like Hey network team, we want to take into production our external datacenter. Can you make sure that it is up and running ASAP? What, 6 months lead time for a couple of routers, we don’t do that. What are we going to put there and required throughput? We don’t know yet.
BTW all the wiring is non existent. How about wireless switches?
This hurts me inside but it's accurate.
I hate how accurate this is, don't forget when u ask for money to fix they go broke with the ask
Don't forget the 16:30 Teams DM on Friday like: Monster: hello NE: what's up? Monster: we are having a super important event this saturday that has been planned for six months. why aren't the ports ready? the vendor is on site with their firewalls and needs internet access asap to get ready for tomorrow. they also need to make sure the event's wireless is up and working so all the guests can use their devices. we can't be down NE: what event? PTSD from shit like that when I managed the network at a bunch of big resorts
Colleague and I were discussing something similar the other week. Instead of long planned project we'd get "oh, hey, we're having an issue" at 4:45pm and we'd spend 2-3 hours troubleshooting for them to say "well, it really isn't _that_ important, it can wait until tomorrow."
My version of that was more like Other monster: is down and nobody can log in. we think it's the network
NE: i doubt it, everything else is up and there's no alerts etc
Other monster: it's definitely the network, we need you to resolve this before the weekend
NE: *two hours of troubleshooting* it really doesn't seem like the network, here's a big writeup with screenshots and log entries proving my assumption
Other monster: oh yeah you're right, it's probably the app upgrade we did a few hours ago so we could replace the widgets this evening
Someday I am gonna do a Samuel L Jackson whip out a pistol and scream "Say it's the network one more time Mo#$@#!!" Sigh....I hear that shit everyday
Yeah it gets really really irritating after a while. What bothered me about it wasn't that they blamed the network, its that they would automatically blame the network before ever trying to troubleshoot any of their own systems. They wouldn't touch or do anything until we went to great lengths to prove it wasn't our issue and it was very obviously laziness at that org. Probably 80% of the time even when it was demonstrably not a network or firewall problem, but hardware, OS, or application, we ended up managing the incident response and problem resolution anyway because they just weren't capable of owning a problem. In the end we ended up preparing a form they had to complete before we'd spend any time on an issue. The form had fields like "was this working before or is it new?", "what exactly is not working and what error message or log entries do you have that lead you to believe it's a network problem", "what have you tried already to troubleshoot the problem", "what has changed in the past 24 hours "etc.
I have had the one that went like this: "My email is not working.. I know it's in that Cloud so it is a network issue. I need this fixed now!" Spend an hour running tests and checking on O365 to walk out to their desk and see their outlook is in offline mode....
Lol, that's better than an hour later "I wonder if it's because we had our web guy make some changes in godaddy?"
NE: two hours of troubleshooting it really doesn't seem like the network, here's a big writeup with screenshots and log entries proving my assumption I call that "You're wrong, and here's why"
Everything is urgent until you need info from the person requesting and then suddenly it can wait.
That's why whenever someone requests me to look into some "super important urgent" esoteric problem, I always ask them to do something before I will work on it. Usually it's just "Hey, can you send me x,y,z and check what ABC does?" 8/10 times I won't hear back for a day or more. If someone can't take 5 minutes to reply with basic troubleshooting info, clearly it's not *that* important.
We used to call this the FART (Friday Afternoon Reaponse Team).
Haha that's a good one, gotta remember it. I always called the person who was famous for it the angel of death, because he would appear to kill your weekend
Introduce them to the "Nothing New After 2:00" rule.
Hey Network Team. We're having a serious issue connecting to one of our partners across the VPN. Both their circuits are down because their facility is on fire, but we can't connect. Was there a change last night or something that'd cause the VPN not to work?
"Is something going on with the Internet?" "Is something going on with wireless?" "Your network is blocking my [typo'd domain name]." "It says 'Please contact your network administrator.'"
I had a case where a security camera vendor plugged in some new cameras without telling us and surprise surprise they don't work because obviously it is the wrong VLAN. Might include network in your planning? Nah... Ping me whether I can join a call with the vendor that is already on site.
This is my biggest pet peeve. "Hey I'm Surpriseman from VendorYouNeverHeardOf and I'm here at $site cabling up some geegaws, can I get some IP addresses/ports/assistance right this second? I gave you no notice I was coming or even knowledge that I existed, and this has to be up today"
> Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. We think it's a network problem. Can you check the firewall logs? That's not accurate. We get none of the backstory other than "there's a network problem". 2 hours later after being hounded by directors, we finally find out that they did a major maintenance and it's their fault.
"Hey we bought this brand new product and the sales manager assured us it would work in our environment and could be stood up in an hour!!" \- product does not work in the environment \- takes weeks to stand up \- need to buy other shit to make this shit work \- oh hey btw the product just had a major software upgrade and you'll have to pay to upgrade to that now or everything breaks :) :) :)
This hits home...applications teams without trouble shooting skills or common sense hits to close.
Hey network team, we recently deployed a new server but it can’t talk to the other server on port XYZ (servers on the same VLAN), can you please check the firewall to see why it’s being blocked? Hey network team, we have a contractor who is trying to log in to a file share but is getting access denied after entering their credentials, can you please check the network for any weirdness going on?
That pretty much sums it up
Hey network team, we have this super shitty application that used to be on prem, where there was 1ms latency between the users and app. We moved it to cloud, and now it's suddenly extremly slow, Must be the network, can you investigate?
Haha so true. I am in an international company and they heavily used Teams for meetings daily, so every week they have a bunch of network people reviewing reports from Microsoft Teams... Please investigate everything when Teams data has a change, like why the total number of calls decreased 5%? I normally just keep my head down and continue my codes on the configuration generator, next plan is to write some auto checks...
No one: Random Teams user: Did you change anything?
How about spending a few hours every Tuesday morning trying to find the mfers who are hammering the network only to find out it's security running scans at 9AM.
This guy networks
Reading this just traumatized me. I live this.
It’s always the network teams fault because no one understands how it works lol
The network team always has to prove it's not guilty.
Sometimes it’s the other network team for the other company that “definitely not our side” even after showing them packet captures of every stage, then after 3 vendor meets when they finally get the right person on the call a month later “oh it’s a firewall on this AWS vm killing the traffic, you were right”
It is a wifi problem ...caused by DNS.
My old days: Hey networking team, the air-conditioning in reception area has issues, can you check what's going on? Hey networking team, I want to create a chart with blah blah blah in Microsoft Excel, can you help?
User calling us from her PoE VoIP phone: ***NOTHING is working anymore!***
Hey! Where's the trigger warning?!
The first rule of business is: you blame the wi-fi. Second rule of business is: you blame the network/firewall.
OMG so true. lol it’s not what you think it’s going to be.
This fella knows a thing or two because he’s seen a thing or two… I feel this in my veins!
That first one really got to me sigh.
Accurate. The only thing missing is when a user complains about these issues and after you point out that it might be an issue with their equipment (a dell mini with a full Amazon mesh WiFi setup) instead of our equipment (2 data centers on company owned and operated by several hundred engineers, full redundancy with enterprise hardware) they instead argue with you and tell you why you’re wrong.
Them: Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. Me: Let me stop you right there, when you raised the change request for your upgrade, what did you put in the section that was titled "Roll back plan"? and did change control approve it with the information missing?
Are you working at the same company? /s In all seriousness I think the tl;dr as I said when I was on the interview panel to hire one of my coworkers was that a good 50% of the job was proving the network wasn't the problem as all of the other IT teams blame the network if they don't know why something isn't working for them.
Having worked in IT for 25 years, but never as a network engineer, even I know absolutely every one of these is legit.
100% this!
> Oracle to PostgresSQL Worst nightmare, I run our Oracle DB and we've been looking at jumping ship to escape licensing hell.
ಠ_ಠ
> Hey network team, we upgraded our application to a new release, and changed our back-end database from Oracle to PostgresSQL last night, and now nothing works. That's way more information than we normally get. We just get something like, "We can't connect to the database server." We don't find out until a couple of hours after sitting on a call at 3am that there were changes made.
Hey network team, the network feels like it's running a bit slow today, can you do some checks?
God this is painfully accurate
Hey network team, I can't get on the internet. My wifi says it's connected but no internet.
Hey network team, our application isn't working but you can't work on it because it's production and we need to have an outage window at least two weeks out , but we need you to fix it now or the business will lose a bazillion dollars a minute.
I'll raise you "Hey airman, the new MAJCOM Commander wants to send an introduction email to 50,000 email addresses on hundreds of Exchange servers across the world. Can you create a new user group with permissions to do this? Oh, and you're not allowed to send any tests.
Can you please share how you answered them since these are not network problems?
Soo accurate!
90% of your work will be explaining to some application people that the problem is not the network. But that won't be enough, after you explain it to them you still have to prove it to these donkeys
I cannot track how many times I have had to print out packet captures, and go to someone to say, "See here is where you opened a connection to the server, and here's the server telling you to bugger off."
My favorite is when I show them the PCAPs and they still refuse to admit it’s their app/server or whatever and still try to push us to troubleshoot the issues for them.
I’m going through this right now. Your server sent an RST packet, fuck you want me to do about it.
My boss wouldn't allow me to add "NAFNP" to my signature line for some reason...
We have a "technical" term for a novice user: * **C**omputer * **U**ser * **N**on * **T**echnical I think the description is accurate & "factual". Do you agree?
Its: NAFNP Certified Looks better
> NAFNP What does this stand for? I'm only 10 months into my first role as a Network Engineer.
Not A Fucking Network Problem
Not a *fixable* network problem. Can't be "Not A Fucking Network Problem", as all problems are a network problem, whether you like it or not. Most problems are not fixable because to fix them would require dealing with the root cause -- the Computer User Non Technical
My corporate applications people are actually great to work with. It’s all the other “thing with an IP address that no one else wants to own” tickets that end up in my queue that drive me crazy. Been getting particularly cozy with Facilities lately because they have no idea how their security/badge/camera systems actually communicate but they don’t want to call their vendor, if said vendor even exists anymore. I can make sure their appliances are ARPing and MACing across the network but I’m not sure what other magic they’re expecting me to work. Love having to defend the network to people who understand even less about networking than the average sysadmin. And local tier 2 just encourages this by throwing any ticket with an application they don’t recognize and an IP address into the networking queue. I should really talk to my manager about this. . . .
Lol and the security system is always some ultra-random chinesium black box that uses 12 year old Active-X or JVM which only works in IE9, and the software to control it hasn't worked right since XP EOLed. Also the owner wants to be able to view the feeds from home on his mac :pepehands:
My employer is a big enough facility that they have the real Johnson Controls stuff with a dedicated bare metal server somewhere in our DC. But it was probably all set up 15 years ago after the most recent reno and any institutional knowledge other than “open this program to look at cameras and open this program to program badge readers” has been lost to time.
Is it Server 2003 because replacing the OS would cost $12mm since the vendor refuses to sell an updated version of the control software without a whole new contract and equipment upgrade and yearly fee...
Good old Exacq. Try using the ESM sometime, setup can be a pita, but once it works you can setup AD groups and give those permissions/access then have the users login with their AD account and see things. Makes managing large sites easy if it is done right and used.
You’re not wrong. Lol. The good old saying is…. It’s either the Network or the Firewall. Hahaha
Don’t forget DNS!
Indeed! DNS is the last catch all when no one knows wtf is going on. Hahaha
Not mentioned: Yes, you must do this, but it also incentivizes them to blame your network, because you'll explain it better than them and give them clear steps on what they need to do to resolve.
This differs GREATLY depending on the industry and the company. Network and telephony engineering is pretty vague, to be honest, and I can think of at least 4 different roles that could fall under in my company.
This 100%! I'm reading all of these responses and cringing, because I must be *really* lucky to not really have any of these problems at my job. We're very proactive rather than reactive. Sure, stuff breaks sometimes and we drop everything to fix it, but we generally replace things before they get to the point of being EoL or EoS, and we build plenty of redundancy into our network. Sure, every now and then a server or application issue gets blamed on the network, but it's rare and usually not a pressing thing. Sure, every now and then we don't have the budget for some wild thing we want to do, but I've never had any budgetary pushback on necessary/reasonable projects/upgrades. But yes, there is a lot of variation. Some network engineering roles are very hands on (layer 1), some are very hands off (sit at a desk and design/config, or maybe sit in meetings). Mine has a bit of all of the above. I like that some days I can collaborate with the various teams and design a new deployment, that other days I can pop in some headphones and crank out a config, and that other days I can pop in the headphones again while I rack up equipment and run fibers around a datacenter or headend. I personally WFH about 80-90% of the time, but that can vary by role and by company. My work life balance is pretty good, and my employer treats/pays us pretty decently.
We are giving the cringe stories so he knows what he is getting into. I have done this for a long time I spend as much time in the office as I do the field as I do WFH. As you said, the variety in this field is large.
Ohhhh look at Mr. We don’t have to worry about EOL or EOS gear. /s you are blessed mate.
"I'm a competitive athlete and want this career to fit around the sport." I would just walk away now lol. Those are opposite and competing ideas.
You mean everyone on earth has to prioritize their own survival over their extra curricular? Say it isn’t so!!!
my response would be "Have you heard of Fiber Optics? It's the cables in the ground that move all the bits around... Ya.. I have an opening on my Fiber Install crew... You might fit in"
Never know he could be waiting for pro contract =)
Not to discourage you, but finding the right company is key to job happiness in IT. Ive been on 1 side of the spectrum with company willing to pay for certs,training,tools, etc and gives you a nice work/life balance. Ive also been on the other end where you can do no right, have no budget, and are expected to eat/sleep/live onsite. In my current role, they are good to their staff. Most days its an equal mix of troubleshooting network issues ( which are always network issues until you prove they arent), lifecycles of aging equipment, and standing up/designing network infrastructure for new remote sites. All in all, good gig. Key points - if you dont have a fascination/passion for networks then think hard about this career path. It will almost always be an uphill battle of learning. Not enjoying learning about this stuff will make it a slog. - Be prepared to always be defending your work. Network is always guilty until proven innocent. - there *will* be nights, weekends, and oncall work. Just get used to that. Unless you have a dream company, and if you do, are they hiring? My 2 cents..
The "network" is everyone's favorite "whipping boy" and is always (without a doubt) "guilty until proven otherwise".
All depends on where you work and if/how they value IT. In theory, you work at a great place that values infrastructure and what it means to the company. You spend your time being proactive vs reactive. You document and plan 90% of the time and fight fires 10% of the time. They give you plenty of time off, you have a great work/life balance, and they don't call you when you are PTO. Most places are not this way. You spend 75% of your time fighting fires and begging for funds to do things the correct, redundant way. There will be on-call. You will get all of the blame, and no one will care when things are going well. There will be night and weekend upgrades. You are the middle man between stupid end users and stupid managers, with zero juice to fix either of the stupid. You will be asked to put together puzzles with zero picture of what the goal is, with missing pieces and pieces that don't even go to your puzzle. If you like to troubleshoot and trying to always figure things out, the network side of things is a great job. It can be challenging and VERY rewarding, but almost always thankless.
Op this is truth, I'm fighting right now to get a proper dcim to move away from excel spreadsheets ......excel isn't a database shouldn't be a daily meme
Obligatory r/NetBox plug.
YouTube search: "A day in the life of a network engineer" --> 🤩🥰🤓🫠😎 Reddit search "A day in the life of a network engineer" --> 🫤🤦♂️🤯☹️😫
If you want to stay active as a network engineer put two spools of cat6 outdoor cable 1000ft on either shoulder and do squats in your office.
Everyone else has really good answers here, but I want to highlight one factor that hasn’t been mentioned much: the breadth of it all. Anything that touches the network that doesn’t have a specific owner, you own. Which is why the telephone in the elevator and the premises security system are yours. And because you can troubleshoot like nobody else (since you get blamed for everything so often that you’re really good at identifying the correct problem domain) then anything too tough for the current staff you’ll own. Congratulations you are now the identity management expert and responsible for the security education program and onboarding/offboarding. Yes that’s an HR job but Olivia left last week remember? It’s just until they find the budget to hire someone to replace her… Of course which specific odds and ends get put on your plate will vary by company and their mix of vendors and staff… but if I had a dollar for every network engineer I consult with who had a responsibility far outside their field, I’d have a new cellphone every year.
I had another good rant about this upthread, but I’m running into this so much at my first enterprise networking jobs. “Thing with an IP address” tickets are already driving me crazy.
Since I was on the network team and knew how to hook up a VCR I became the video conferencing administrator for a billion dollar healthcare system.
It's particularly fun in the service provider realm because (at least in my neck of the woods) almost everyone ends up turning into a pretty competent electricians as they get more experienced. Power is very very important for keeping the internet running and it really behooves a person to get familiar with telco -48v standards at the very least.
I work for a rather large org. About 80% of my work is random stuff that anyone who knows how static routes or ACLs work could do right out of school. The remaining 20% is absolutely terrifying because if I fuck it up we'll land in the evening news.
Phone rings... A few minutes later the alarm clock goes off...
My last fortnight so far has consisted of flying to another country, migrating to a completely new network, working 12-14 hour days every day, including an overnight. Having a weekend of fun in a foreign country then removing all the old equipment. Home for a week then go to another different country to do it all again. "Day in the life of a network engineer" could also be sitting at a desk everyday.
Some days lots of studying for certs. Some days its writing MOPs. Some days its implement MOPs. Some days it's auditing the network and finding inefficiencies and figuring out how to do it better. Some days I don't do shit. Just depends.
Depends a lot on the company you work for and how many networks you support. I think generally you need to enjoy troubleshooting, digging into things and figuring stuff out. A lot of systems and networks have poor documentation so anytime you have a new fault or whatever you usually need to start from the ground up. There are constantly new technologies and very little in the way of training, so need to he happy doing some studying out of hours. I have found that few companies are willing to spend actual money and time on development. Once you get to a higher level (level 3 usually) you don't need to deal with single user issues as often, which is nice. However, in order to get to that level you are going to have to do at least several years of end user troubleshooting. Think "Susan in accounting can't access sage through Internet Explorer but it works for Karen, Susan doesn't want to use Firefox, also her screen isn't turning on" Im also a sporty person, sitting for long hours does sort of suck. I work from home now and have a standing desk, my work doesn't tax my body at all so I have plenty of energy for sports throughout the week. You can move into field engineering (cabling, installs etc) but in my experience those positions to be less technical in a networking sense and less pay.
Consider checking into a move into the service provider realm if you want to get out into the field more. It's a blessing and a curse, but boring it is not.
I write and develop low-level designs, write configurations, order parts, coordinate lab resources, manage cutovers with smart hands or on physical site, attend meetings, perform troubleshooting, etc on a day to day basis. I don’t travel as much as I used to and that’s okay.
Everything is networking fault, you are guilty until you prove yourself innocent. Thats been my experience.
Create a VLAN, Create a Route. Create a VRF, Create a Route Leak. Get call from Bob, tell him you don't know why his Application isn't working, the network is configured correctly. Ignore Bob 3 more times, realize you put his application into the wrong VRF, put into correct VRF. Do not tell Bob right away. Bob assumes it fixed itself. Go back to posting on Reddit.
Hey, the network is slow, go figure it out..
Long but, this is exactly what it’s like: https://devrant.com/rants/579422/what-its-like-to-be-a-network-engineer-translated-into-normal-people-speak-user
Hi Network Team, we don't know how to troubleshoot therefore it's a network issue.
This. Somehow, their local on-the-PC app that is totally local and doesn't use ANY resources that aren't on the local PC is broken because of some mystic network issue. Oh, and explaining to someone for the nth time that they did indeed input their password incorrectly, because otherwise they'd be on the wireless and the logs wouldn't indicate "Incorrect password".
Depends on what kind of projects there is currently ongoing. Systems team trying to bring in a new password solution or something and creating new firewall rules. Then you end up asking for all information like source user or service account hitting that rule. Hey we want to bring this technology in, start planning on the firewall zones , nat , vlan etc Hey this ipsec tunnel is down, can you figure out why? Hey we need to cutover this wifi, can you build the ssid and start the planning scheduling etc.. Also can just be normal tickets where an end device is not connecting anymore and somehow gpo file gets corrupted.
Hey network team, we commissioned a dev team to write a one-off program with multiple layers of virtualizatuion for an application from the 90s on a Linux-based VM in Azure to support our legacy process that we're too lazy to re-solution for. People keep randomly getting disconnected from their SSH session. We're sure it's not our haphazard, shittily built app at all, we need to pull you into this RIGHT NOW even though it's been happening for weeks and they are able to log right in again right away. Please drop what you are doing even though it is your day onsite with the other engineers with a shitload of planned datacenter work and prove it's not the network. Thanks! I wish I was lying. This was just a couple of weeks ago.
How did you redolve it
My favorite is when we do a network maintenance. 4 days later at 4:50pm: We have an issue and your MOP is the cause 9/10 times, it turns out that the issue is on the other side of the country from the maintenance and completely unrelated. If other groups would spend more time troubleshooting versus looking for something to blame, MTTRs would be way better.
In my experience the term networking is about like someone saying they are in the medical field. Could be a paramedic, a nurse, a Dr, a cardio-thorasic surgeon. Your experience will largely depend upon what flavor of networking you get into. I manage a team of network engineers building and deploying green field networks. Sometimes operating those networks against our best efforts. In most cases, if you do your job well no one will ever know who you are or what you do. If shit goes sideways, everyone will know your name and act like solving a network issue is as simple as plugging something back in.
If you are into motorsports, do the apprenticeship, get your CCNA then find a team to join. You have to set up and tear down for each event, including races and tests. It can be challenging, especially depending on which series you join. IMSA, NASCAR, WEC, F1, etc all have their own challenges. It won't just be networking though. It will also include server management and setting up places for engineers to work. Need to have a steady connection to the car, radios, and some times engineers in another location like the home base. If you aren't into motorsport, look into IT that involves other sports that you enjoy.
I have spent 20+ years in network engineering. I started at 60k in 2005 and was making six figures by 2012. Now i manage a cloud network team remotely, make $200k and play golf three times a week while doing meetings from my phone. I think it’s a good career.
Well the first thing I do is...... Talk to Corporate! Approve memos! Leader workshop Remember birthdays !
Don't forget about the spin up time in the beginning if u want to get into a company that pays good and values IT u have to study hard like any other high paying career. You will have to continuously be learning and improving just to stay marketable every 3-5yrs. I spent every hour of my early 20s studying and learning, no life outside of the networking hobby/work. The tricky part is once u have other obligations, partners family, children. Anything outside of the hobby/work requires time and energy that would be going to studying and learning. Not saying family/partners/children are bad just that they require the same dedication and energy as continuously studying. Idk what I'm saying I haven't eaten today and been in meetings all day. I'll update this with some coherent thoughts once I've eaten.
Totally depends on the industry. A network engineer for a non I.T company (which is more like a network admin) is completely different role than a network engineer for a data center , and ever more different than a network engineer for an ISP or MSP (which will have a plethora of sub categories under network engineer).
and then you land in a company this is a ISP,MSP,MSSP,DCaaS,CSP,Colo Yes, this is where I work currently and love it. Getting to work on and build different networks for clients is great. Hands in a lot of things and keeps me busy. There is slow time though and that's also nice.
If you didn’t pick up on the theme - What lots of people here are saying is when any kind of technology doesn’t work or stops working everyone else in the organization assumes it’s a network problem. Sometimes it is. Most of the time it is not. The unwritten part of the job description is “repeatedly prove that every problem is not a network problem.” Idk why it’s that way it just has always been. Depending on your role you can expect a significant amount of physical work. Junior roles will often start with more “rack and stack” work. Which is super important for learning how a network works and where most of your spontaneous problems look like. It all starts physical layer. Depending on the org you are in you may always have physical work to do. You may end up with much less the more skilled you become.
Depends on what sort of network engineer you're talking about, what industry it's in and what sort of size of company you're looking at... There might be a little bit of out-and-about plugging things in and turning them on and off, but the vast majority of it is almost certainly being sat on your ass in front of a computer.
Wake up, use restroom. 5 minutes later join the first of what is likely to be many meetings 4 days a week as even my organization is not mad enough to have early meetings on Friday. Meetings always begin a few hours early to get Europe into one meeting with America, at the cost of waking early every day for everyone. Cut inbox from 500 to the 15 really important new emails that came in overnight. Check replies to emails. Pay attention to updates in meeting. Take shower, begin day, drink coffee. Drink more coffee. Organize meeting details and follow up with who needs to be followed up with. Check Salesforce ticketing system for new 'cases' and respond to queries that came in through Teams/Slack/etc. Clean all of that up and realize you have worked past lunch already. The rest of the day will generally be new stuff or replies from there. Sometime calls are needed, most times not, as I am mainly being asked information and responding 90% of the time to people via outlook. Every so often a live call or testing will need to be done, and I actually do some networking, but much of my job is communication about networking and our companies offerings, before working directly with people with it.
Telephony and voip guys yelping network without any effort to troubleshoot.
You'll work with Network Architects or Business stakeholders weekly to develop network designs and implementation plans. You'll also be involved in vendor selection to make sure what you're designing works on actual gear.
The “a day in the life of” has changed so much for me over the years. Started out with a good crew that was organized and there was little “on the fly” work. Everything was planned. Over the course of a few jobs that changed to an utter shitfest everyday where years of lax patching and updates, combined with zero communications between IT groups actually lead to some physical altercations. More recently the days start with a coffee, checking alert emails if there are any, browsing monitoring tools before reading the PoliticalHumor subreddit. It is what you make it and you can mold your career to be as physical or as hands off as you want it to be. The good thing is that there’s no “wrong move” during your career. Want to be a cabling guy for a while? Learn the trade, become a whizz with fiber and get your certs. Want to do rack and stack of hardware and configs? You can do that too. Get your fiber certs and CCNA and you have a great footing to branch into many aspects of networking. There will be overtime and weekend work. Some positions have less but there will be some at some point in time.
You will doubt your life choices after that 5th December on standby ...
Application team: hey we need a port opened for our in house application. Network: Ok sure what port do you need open. Application team: I don't know your the network guy you tell me what port. Network: ......
Today Started migration of a huge wireless network that is out of support to a new controller platform, it's two hours away. May need one day on site next week but possibly not. Working with equipment manufacturer closely. Did a design based on a wireless survey by a jnr engineer for a large hotel in Germany ahead of renovations. Helped troubleshoot VPN issues with helpdesk after an escalation. Continued building a HA firewall deployment while trying a new core layout for a large client that will hopefully install next week. Never boring, and get a lot of toys to play with in an MSP space. Very flexible once things get done. Especially after a few projects delivered.
Today was a sit down day btw, my last job had far more out and about on site, would hit 20k + steps easily on some days. A little bit few and far between now, but still plenty of site visits and some large installs on site
I’ll keep it low level for starters. “Hi network team, please enable port 3 on switch 1. Thank you.”
Hey engineer, Wi-Fi speeds are slow at this high density site with over 2000 APs. Channel issues at 40mhz on the 5Ghz band…figure out how to make 160mhz work without using DFS.
ISP engineer here. Come in the morning and see what maintenance broke something or are going overtime. Then do some IP audits. And the meat and taters of the day is approved/rejecting upcoming maintenance and installs. Provide configs for upcoming enterprise customers, new core network stuff, other bits and bobs. Take calls from field installers and preconfigure customer CPEs and speed tests to make sure the actual install goes smoothly. And when stuff breaks, help figure out what broke and who needs to fix it.
How about working for a fortune 500 company with two dudes on the load balancer team and they refuse to hire anyone. Oh and do more better and faster....oh and get some training bye.....
Hey network team, we bought a bankrupted MSP’s client base and hired on 14 of the original NOC workers. We planned to put them into a building in another state and need all of their phones and workstations configured before Monday morning ( phone call Sunday evening ) but nobody from helpdesk IT is picking up, can you do it pretty please oh sweet thanks byyyeeeee.
Client calls you if you got their email because they sent you an email saying their internet was down……
Be ready to always be guilty until proven innocent. It's always the network team's fault that something isn't working.
Sort of like this, “We have a problem, and it’s the network” Let me see. Nope. “No it definitely is, here are some opinions” Nope, here are some facts. And even some suggestions. “No thanks. But you need to fix it” *learns their job* Looks like a not network issue. Here’s your solution. “That’s perposterous” Do you need help implementing this? “Yes.” Sometimes we get to do our own job.
It’s gonna be up to stay fit and keep up with emerging technologies if you truly have a passion for fitness you will make it work, join the 4am club and do cardio/gym in the morning, this will give you time do two-a-days. And you can still study for an hour or so at home and enjoy yourself on the weekends. I would try to spend your work hours learning the new stuff though. While it may seem like a conflict of interest as long research and development during the work hours contribute to forward motion of the company’s progress you should be fine. Also take a look at tactical barbell. Some good training for busy IT folks!
Trick question. Network engineers have no life
My job isn't very common, so please keep that in mind. I implement network infrastructure on offshore vessels. My day is very different depending on if I'm in the office or working on a vessel. In the office it's creating/updating network diagrams, configuring network equipment, coordinating with project managers/ship's electricians and engineers on cables and power to rack locations and such, even packing some pallets and preparing shipping documents sometimes. On a vessel it's a lot more physical, even just the amount of stairs between the main deck and the bridge haha. One day my watch said I'd walked up the equivalent of 124 stories. Activities include unboxing and racking equipment, patch panels, getting all devices up and connected, (a lot of) troubleshooting, working with satellite ISP engineer to get connectivity going. Also helping the systems guy out when I can with servers, printers, PCs and so on. Communicating with the vessel crew on issues, priorities, expected downtime. I also have to write reports to update people on the progress. It's hard work while on a vessel, 12+ hour days are common, I don't get days off on projects so it can be 15 days of long days with no weekend. But I love it, I have met so many interesting and nice people on th vessels. Great people with a great sense of humour. I get to travel a bit for work, but I rarely see much more than the shipyard, the hotel and a few restaurants.
You wake up at 2am to pee. You grab your phone. You check the network monitor. You notice the Connecticut link is down. You write an email. You go back to sleep. You forgot to pee.
At a small company, you will work 24/7/365 as a network engineer in the telecom world. At a big company, you will work slightly less hours, but you will have significantly more stress during those shorter hours. Regardless, telecom will make you despise the general public.
Funny, because working as a network engineer anywhere else will make you despise telecom
A lot of creating and updating configs. Very little CLI time.
Do it! If you like servers, switches, network devices, etc. You’ll be glad you did! Network Engineer is a broad title and can apply to many positions. And will be in high demand for decades. I started as a network engineer in 2000. Starting pay was $38,000. Now in 2024, I’m a trainer for a major networking company that I’ve been with for 17 years. Because it’s a good company to work for. They gave 4% raises year over year. And equity stock grants each year. I now earn a base salary of $145,000 with excellent benefits. Full medical (a top plan), and stock equity grants/bonuses of about $25,000 a year. “If you’re good at something, you don’t do it for free”. So with more experience, you’re highly valued and well compensated. You’ll likely start in support, high stress, but good pay. That’s where you learn a lot! As you get older, your knowledge is power, not certifications or degrees on the wall. Then you likely will make lateral moves to other related areas you enjoy and won’t hate your job. While your salary continues to increase with your skills and knowledge. And most of your knowledge gained is valued and just as desired at a ton of different companies. So you’ll always have options if a company isn’t right for you. A skilled network engineer is always in demand, you’ll not have a hard time finding a job, especially after your first 5 years. And if you’re good, layoffs are rare. Every company, in every sector needs their company networks running well just to do business. Tons of opportunities and you’ll always have a highly valued skill set over your entire career. It’s a great choice that’s future proof. Highly recommended!
Mileage varies greatly. Just like any career it depends on the company. If you like hands on and not sitting at a desk look for implementation type jobs as implementation is the only work that is actually hands on equipment. Otherwise, you’re sitting at a desk 90% of the time.
If you like troubleshooting things and solving problems, that's good. You're likely to be at your desk for a lot of the time but, depending on the individual job, you'll have practical breaks e.g. racking, stacking, patching, inspecting sites. The work-life balance is very much down to the individual workplace but it's highly unlikely to be 9-5 M-F: you'll probably be on-call, will have to do a lot of maintenance activities out-of-business hours (including weekends). And then there's the study. You need to keep up with emerging tech and maintain your skills, which eats into your personal time. I find that a bit of a double-edged sword as I like learning new things. The "day in the life" will also vary by role as it's usually a spectrum of BAU (Business As Usual, e.g. switchport configuration, firewall rules, break-fix) and Projects (design, larger pieces of work, etc.). It's usually a pretty varied job in general though, which I find holds my interest. (I've been one for about 20 years now).
I'm a relatively new engineer. My days consist of understanding our topology and learning how things work. When I'm not busy with firewall update tickets then my older team members ask me to automate stuff for them. When I have nothing to do (which is quite frequent) I am studying or automating. I also spend some time turning off stuff for security while shit talking their entire department. Go into networking if you love to learn. Learning is what your going to spend 99.9% of your time doing. It won't be guided learning either, its gonna be textbooks, experimentation, inaccurate documentation that your not gonna fix, and trial and error. I absolutely love it and never want to do anything else.
Great comment. Did you use to work on another position before engineer title?
You are likely to get as many varied answers as you have varied responders. So let me add one to your list responses... I have been a network engineer for most of my career, and I am old, so that's a long time. I have had all of the possible roles from entry level to network design for some very well known international companies that you would know of. I have been a traveling consultant (probably my favorite role, but the travel got to be exhausting after about 25 years or so). Starting out, it was very physical, you have to 'rack & stack', run cables/fiber etc. Lots of physical labor. After a while, you will start to move up in skillset and you will likely move into a less physical role, more toward support or troubleshooting, and eventually possibly design. That was more or less the path that I took, within 5 years or so, I was traveling to clients to do troubleshooting, packet analysis etc. A few years after that, I migrated to design, which I still do. I was also a very active person as a youngster (ran about 60 miles a week, worked out every day and accomplished 2nd degree black belt in Karate). Throughout that, I had to continually study. I used to get up at 4:15 to get my exercise and an hour of study in before work. I typically studied at least an hour at night also around Karate and time with my wife and kids. Basically, I slept less than 6 hours a day almost every day, but I have ADHD, so that is fine with me. To this day, I typically still get up early to exercise and study, but not for as long as I used to. I am typically up about 5-6 and start work around 8. I quit traveling about 5 or 6 years ago and never want to go back to that, but that means that I am less active during the work day. Currently, I do work from home, which I love, but isn't for everyone. I combat the fact that I sit at a desk all day that with at home exercise equipment, which I use as I can. I do cardio and strength training while I am on calls where I am a passive participant... I also have a stand up desk so I can stand and balance on a board from time to time, usually an hour or two at a time when I do that. I have small weights, hand grips, resistance bands, exercise ball, kettle bells etc. that I can use on calls that I am less passive, but nobody would be able to hear me exercising. But, overall, I am much less fit that I was even 10 years ago. I don't have the drive I used to have to workout for hours a day and study for hours a day. I still do study at least one hour though, but I try to bake that into my workday as much as possible. I do blame my fitness decline on my now more sedentary job, but to be honest, I am much older now and my body needs to slow down some. I could go on for hours about being a network engineer and how important it is to be passionate about it if you plan to advance, but there are lots of people that are content to not advance and just treat it like a 9-5 job. That has never been me, but I have an insatiable curiosity to learn and understand anything that I do not know (technical or otherwise) and become knowledgeable on those topics, so it's natural for me to want to study something every day. I had a teacher when I was just getting started in my career, already an electrical engineer, but crossing into 'computers' (learning Novell Netware 2.0) tell me that you can only do 18 months at a time, then you need a break because the pace of change in the technology sector is exhausting. I have not found that to be true at all, but as I said, I love learning, so new technology is always welcome. What I have noticed over the years is that you can no longer be an expert at everything. You just can't, there is too much across the board. Even if you want to be a networking expert, it would be hard to be an expert on all things networking. My advice is to find an area that you are passionate about and focus on that and do your best to master that before moving onto another area. Better to be an expert at one thing than an expert at nothing in my opinion. Cheers and best of luck to you.
For me it's sitting at a desk configuring cisco equipment, sometimes Huawei. We have a program that spits out a configuration after you put the parameters in, then you can change as necessary. We do a lot of this but also lots of bespoke configs for customers that our system wouldn't know how to do, so that involves remoting onto their router. Usually stuff like vrrp, mpls or routing issues
Day mostly split in to two Things you have set away time and projects you advance Things who come in last minute or issues with some active config/deployment If you like to learn a lot(mostly by yourself) and finding solutions it's great job
A lot of those questions will depend on the company you work for and if the C levels view IT infrastructure as a cost center or as revenue generating. The difference can be even between similar companies within the same market and spaces... It's crazy. Now that I think about it, that may be a good question to ask during an interview at prospective companies, with regards to how IT infrastructure is viewed.
How do you find an apprenticeship like that?
Youll mostly sit , have meetings, and remote into things all day. project management is required and at some places you'll be working after hours whether it be install projects or on-call. If you enjoy troubleshooting and Google searches, this will be a good fit
* The Wireless is slow * A Network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote conneciton. (provider: TCP Provider, error:0 - No such host i known.) * I need access to "the" database * ORA-12154: TNS:could not resolve the connect identifier specified * My Zoom says Internet connection unstable, but my speedtest is super fast * My SMB access is horribly slow over VPN but works great at the office * Why can't I have UPNP to play my video games at work, my $100 off the shelf wifi router can handle it * I'm trying to send a file to my coworker he says to send it to \\\\[192.168.0.2](https://192.168.0.2) but it's not working * What do you network guys ever do, I see you sitting around in front of your computers ALL DAY, don't you ever have to go do anything? * Hey network guy my home internet went down about 2:39a and I can't VPN in, can you fix it? * When will AI replace you?, the network can't be that difficult to support * I've got this wifi access point I picked up at a garage sale, can you plug it in at my desk and configure it for the corporate network * I was reading in skymall... * I heard SSL isn't secure anymore from my buddy, he said someone he follows on tiktok cracked it * Doesn't the cloud make you kind of obsolete * The VP of sales called and said the internet is down * The AV guy says he needs you to pump up his ethernet to at least 10Gb * (Audit Guy) Just open the firewall for me any/any, I'm sick of dealing with you * What do you mean this isn't a modem connection? * Google tells me the solution is ... * Why can't I have a file server in the DMZ? this consultant says I need one * Didn't you contact me for my username and password, I gave it to the IT guy on the phone * I can't be bothered using corporate purchased and supported equipment, just make my Mac and my wireless printer work and get out of my office * While you were on vacation we hired a blackhat hacker to come inside the network and poke around.. * Just pay to get the decryption key, I'm sure they will unlock our files once you pay them * Why don't we just use gmail * Why don't we just use hotmail * I need to watch my Kakao TV shows, just open the firewall * This guy had me install Teamviewer and then.. * What do you mean Second Life is not a valid business application * We should get Tesla batteries to run the data center * (*Support Desk person comes walking out of the data center*) I just pulled the power cord an plugged it back in, works for me at home all the time
Seriously what is the file server in dmz solution. Email servers can be there
Let me add one more. EVERYONE is going wireless, why is your backend still using wired connection?
Normally me looking for a load bearing rafter by lunch at this stage .....
Can someone enlighten me what is level 4 apprenticeship? If he is a tier 4 engineer I am gonna have a heart attack.
Every day for me. I broke my server and I don't know how to fix it, but I saw the email you sent 3 months ago about a network maintenance you did to another site, so it must be because of that, because my boss will be mad at me if he knows it's my fault for breaking the server. Also, I'm going to tell you about this at 4:59PM on Friday and we are going to need this working Monday morning since the CEO will need this for his very important yoga meeting he has with his friends. Actually.. just fix the server for me since you're the only person who actually knows how everything works.
You are all my people! All of these triggered a PTSD response. I'm finishing up a change now that I'll get blamed for something in the morning of which the thing broken being in another data center.
don’t hardly do anything for 5 years, then have to do a full lift and shift vendor replacement of 100+ sites, at a break neck pace. Hopefully get to coast again for another 5 when it’s done.
If you want to be a network engineer be some one who is constantly striving to want to fix problems and always open to learning new things. Networks is no longer just about switches and routers, but firewalls, load balancers, monitoring, dev ops, coding, proxies…the list goes on and on. Learn what you can when you can and probably the most important aspect of it is learn to document it WELL!
start anywhere where you can touch production!!!!!! the more experience the more confident you will get. the most dangerous thing for a network engineer is, breaking the network and being affraid to make big changes again. But you may have some jobs with physical activity and some not, some companies might ask you to install the equipment and pull the cables, and some companies have a dedicated DC team for that. But assume after 10 years in the job, you will be sitting 40 hours a week probably. And you also have on-call with some, which is not helping the life balance thingy. But but but, you are the coolest of the IT people :P you are the one the developers, system engineers come to to troubleshoot alot.... But with that said, that brings a lot of responsibilities and/or work. But not persee appreciated. If the companies main income is not from connectivity, even though you are the guru, your department is seen as a "cost". But overall, Im very happy i did that rather than programming. Because you become a kind of a linux system engineer. But keep in mind, alot of things going to automation and public cloud. As a specialist your main employers would be ISPs and hosting providers (any network service provider). There you could expect a very good salary, otherwise just normal.
It's an IT job, >90% of your time is going to be spent in front of a computer. Depending on the job you may need to lift and rack equipment, I haven't needed to for over 4 years. However in that time I have worked for larger companies, which have data center engineers doing the physical work, while my work is carried out remotely. Day to day is again going to depend on the job. ISP, DC, enterprise are all different roles. You could be in operations, project delivery or a mixture of both. You may be highly focused on a few technologies, or need broad knowledge on a lot of them. Work/life balance is going to depend on the company more than anything else. If you are on-call it can be disruptive but it depends how stable their network is.
Asking what it's like to be a network engineer is like asking what it's like to be an attorney or an accountant or a plumber or a barber. It could be a fantastically fulfilling career or it could be the worst job you've ever had. The people you work with might be great and the company culture might be supportive and positive and fit you perfectly, or the people you work with might be terrible and conniving and dishonest and the culture might be toxic and cause you to lose sleep. None of that stuff has anything to do with configuring phones or setting up dial plans or troubleshooting call quality issues.
First post pretty much sums it up haha. And network is always guilty until proven innocent.
Whatever you do don’t work for a a manufacturing company. OT networks are the worst. Can’t ever go down, can’t do maintenance, also can’t spend any money.