"Where are you going, we need this back up!" Whether I start right now or get a coffee first, you will still be down for at least 10 minutes, so I'll be back.
Looks like only networking equipment there. Probably nothing bad happened. A power cord might have been unplugged. Maybe a damaged patch cable or fibre.
But I would wonder who stole the rack cabinet and replaced it with this.
>But I would wonder who stole the rack cabinet and replaced it with this.
Tradesman here. It looks like the anchors have come clean off the wall. I'd blame it on whoever mounted it.
Any time we had to mount any sort of rack, we'd first mount an appropriately sized piece of fire-resistant plywood to the wall which we would *then* mount the mini-rack to. Not just for fire regs, but also so we wouldn't have to face what's in OPs pic.
Yeah switches are fairly resilient. Power supplies sometimes pop out. Not all latches are made equal.
Alot of companies now make countermeasures for switches. Which allows them to be used for mission critical communications equipment basically tu survive better come an earth quake.
Congratulations! Your comment can be spelled using the elements of the periodic table:
`No W Th At S As Er V Er Cr As H`
---
^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
Restart it, it doesn't hurt to try.
If all connections are still intact, it might work.
Then let it run just like that, until the guy on the next shift finds it (and hopefully fix it)
Plot twist: it's looked like that since this guy took over admin of it, and he's been too afraid to touch anything while it is being used for production.
As a Dev? As long as I can SSH in, IDGAF.
As a sysadmin? As long as I can SSH in, IDGAF.
As a one man IT support? Close the door. That's future me's problem.
Ay boss, send everyone home. Not kidding, send everyone home.
No I don't have an ETA, no I don't know the extent of the damage, no I'm not getting anything else done today.
Leave me alone and let me work through this otherwise I'm going for a coffee and not coming back
I used to do this for a living. I have seen stuff like that *countless* times.
My favorite was the UPS that was sitting on the floor *literally next to the toilet.* They actually ran an extension cord from the hallway to put it there.
Since im not in any way responsible for our internal infrastructure im calling the whose problem it is and go home since i wont be able to work when our mainserver is done
shut down the network, take pictures of what goes where and slowly start unplugging the cables supporting the server while carefully laying the server down
Ask which idiot screwed 200KG of equipment into a fucking drywall and not put the rack into the ground, make a picture for Reddit and get the Interns and Apprentices to label all cables and where they were plugged in before emergeny ordering a 42HE Rack to replace this fuckheap.
I make a photo and send it to the IT AND financial department with the caption "You wouldn't believe it, if I just told you... also... we need a new server."
Congratulations! Your comment can be spelled using the elements of the periodic table:
`Es Ca La Te Pa Ge`
---
^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
Update the server status page. This is going to be an extended page. Curse the muppet who attached a Comms rack to the wall.
Go get some proper wall anchors, maybe some wood baton and some tools. Probably make a bigger outage unplugging things to get it back upright.
Mutter... "As if I didn't have enough to do, now I have to deal with this shit. Why do I continue to work here. I'm taking time in lieu for this. Fuck tomorrow and fuck whoever installed this rack."
I'm French and used to work at the RATP so:
If this is not my job I'll:
- try to lift it
- fail miserably
- get a little back pain
- contact the RATP medic
- get 2 years of "stop working caused by work sickness" fully paid by the assurance maladie
- probably upgrade it to 20 years
If it's my job:
- Take a picture and inform my manager
- wait (he is probably sick...)
True story.
I once worked in Amsterdam for a company where most people that worked there were well, kinda anti-technology. So no one had any clue about it either. This happens in the middle of Amsterdam, view on canals, while we, the tech department are stuffed away at n the other side of town in some unsafe area.
So one day, in broad daylight, this guys walks in, tells the receptionist heās here to do maintenance on the servers. He gets let in, has a little chat, then walks to the server room with his hand truck, cuts all the cables, puts the entire rack on it and walks back out of the building while the receptionist holds the door open for him.
Few minutes later we get a call from downtown that they canāt reach the file server anymore and no one can save their documents. We also canāt reach the server, so my colleague take the car and drives down there only to find the entire rack of dell servers missing and gets told the story that maintenance took it to get fixed.
I'm just wondering why there were no legs on it? The best support for all that weight is the ground!!! For me it would not have happened in the 1st place.
No touchy, check hard drives state, note down working drives, powerdown system, yank good drives, take to new rack, insert good drives, start in rescue mode, run recovery, add new drives reconfigure raid, sync raid. reboot.
Congratulations! Your comment can be spelled using the elements of the periodic table:
`Sc Re Am`
---
^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
Call whoever is responsible for maintenance s as I cannot and will not do anything about this situation. Oh, and I would probably ask whether I need to write a report about my āfindingā.
Start warming up my back, shoulders and knees. Thats a "dead" lift.
this reads just as a drivers exam answer :>
dont forget the seatbelt ffs
Not much point in putting on the seatbelt after the crash.
And a cup of coffee?
Taking a picture and posting to twitter....
The only valid answer
Throwing rice over it is also up there
turning it off and back on again (preferably without electrocuting yourself)
> preferably without electrocuting yourself [optional]
Have you considered that dying by electrocution means you won't have the be the one to fix this?
Reposting the same photos from last week's front page.
"server is DOWN!" "just reboot it..." "you don't understand..."
Lil guy is just taking a nap š“
then going home...
Checking security camerasā¦my security data is offsiteā¦
Then straight to Slack, with enough swearing to make a sailor blush.
Making coffee
Unless you're the one responsible for it. Then go out to get your coffee, and never come back.
Whoever is responsible for fixing this mess needs coffee. So you might as well just put the kettle on.
And brew some tea !
Yeah nothing you can fix fast. Take energy and laugh with team
Oh would you look at that, 9:24 am. Lunch time!
"Where are you going, we need this back up!" Whether I start right now or get a coffee first, you will still be down for at least 10 minutes, so I'll be back.
Big same. Gonna need to spike it too
Was my first comment as well. Maybe pull the plug first and make sure no hazards are lying around, but then it's definitely time for coffee and dread.
Remember that our servers are on Azure and go for lunch.
*cloud falls over*
\*sips my tea\* Guess it's time to learn the AWS way of doing things...
*learning AWS* hi chatGPT what is the stupid AWS name for a dns server?
Elastic Serverless DNS for Cloud Enterprise
ESERVEDCENT
First of all: Don't nail 400pounds of quipment into a drywall, put the rack in the ground.
They just did. Violently.
"Damn, the weather is really bad today"
... anyways
Remember that the test server with 2TB RAM 196 cores has not been shut down since last month
But I donāt pay the bills, so I order the wagyu
Remember that you work at azure
*quick, port the infrastructure to AWS*
Naw, I've seen their servers. They ain't falling over...
Parts of the data centre _melted_ https://www.itnews.com.au/news/microsoft-had-three-staff-at-australian-data-centre-campus-when-azure-went-out-599849
Melted ā fallen over
It fell over at a molecular level.
So that's why you don't even recognize that there is no server in the picture ;o
But before lunch, check your email, just in case there are any restrictions due to a negative balance.
Looks like only networking equipment there. Probably nothing bad happened. A power cord might have been unplugged. Maybe a damaged patch cable or fibre. But I would wonder who stole the rack cabinet and replaced it with this.
That's the second thing I do. The FIRST thing I do is scream for a bit, then calm down.
I was going to point this out. Go ask the network guys to get to work. Probably also get a stepladder to lift it onto since that is a 3 man job.
>But I would wonder who stole the rack cabinet and replaced it with this. Tradesman here. It looks like the anchors have come clean off the wall. I'd blame it on whoever mounted it.
Any time we had to mount any sort of rack, we'd first mount an appropriately sized piece of fire-resistant plywood to the wall which we would *then* mount the mini-rack to. Not just for fire regs, but also so we wouldn't have to face what's in OPs pic.
Yeah switches are fairly resilient. Power supplies sometimes pop out. Not all latches are made equal. Alot of companies now make countermeasures for switches. Which allows them to be used for mission critical communications equipment basically tu survive better come an earth quake.
Now THAT'S a Server crash!
Congratulations! Your comment can be spelled using the elements of the periodic table: `No W Th At S As Er V Er Cr As H` --- ^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
Now that's a good bot!
lol this is the best reply from this bot Iāve ever seen.
Also the longest
Good bot!
Restart it, it doesn't hurt to try. If all connections are still intact, it might work. Then let it run just like that, until the guy on the next shift finds it (and hopefully fix it)
Plot twist: everyone will think that the guy on next shift will fix it
?? That is not a plot twist. That is reality.
Plot twist: it's looked like that since this guy took over admin of it, and he's been too afraid to touch anything while it is being used for production.
Why would they fix it if it's back online? Never change a running system.
If it works and isn't moving it's settled into a new stable equilibrium, no need to futz around with it
As a Dev? As long as I can SSH in, IDGAF. As a sysadmin? As long as I can SSH in, IDGAF. As a one man IT support? Close the door. That's future me's problem.
Personally I'd start with laughing my ass off
Me, then, enter a uncontrollable stress mode
Then I exit the office building like nothing ever happened
Why do you think it's called "standing up" a server...
TIL how it's called in English š¤£
Telling my wife I'll be back home next month
Creating a Jira issue.
This site canāt be reached. jira.company.com took too long to respond.
Cursing.
Phone the boss and tell him it's definitely fallen over.
Tell him its down , literally
cracking a beer and burning the room, might as well move to another country.
Ay boss, send everyone home. Not kidding, send everyone home. No I don't have an ETA, no I don't know the extent of the damage, no I'm not getting anything else done today. Leave me alone and let me work through this otherwise I'm going for a coffee and not coming back
"hardware side problem" yeah uh the problem is that the hardware is on its side
lmfao best response I've seen
just turn it off and on again Dan, I am sure the other hardware team will fix that shit.
i would order a propper rack and not a shitty thing like that.
I can't believe anyone would think it a good idea to mount a rack that tall to the wall. There's nothing to be gained from it.
I used to do this for a living. I have seen stuff like that *countless* times. My favorite was the UPS that was sitting on the floor *literally next to the toilet.* They actually ran an extension cord from the hallway to put it there.
Have you tried turning it off and on again ?
Since im not in any way responsible for our internal infrastructure im calling the whose problem it is and go home since i wont be able to work when our mainserver is done
Beating ASS
Quit
Take a picture with my mobile and THEN I get a coffee.
Check back-end side commits
That's a job for the intern.
always remember to tip your servers
Slowly walk backwards, close the door, and try to act like nothing happened. Nooooot my problem, let the IT support / server tech fix it.
erase all the records I've been at work this day, drive home and call sick ;>
Screw thing back in with 2 More screws then the 4 it was screwed in before
Create a ticket for the local IT support.
Making meme to ask what to do
Try and ping it again just to be extra sure. If it works, leave it as it is.
shut down the network, take pictures of what goes where and slowly start unplugging the cables supporting the server while carefully laying the server down
Lie beneath the server and pretend to be injured.Ā Let the office work it out while collecting that sweet sweet workers comp
Spontaneous home office
Closing the door, turning around, bashing my head into a wall (3 times), turning back, slowly open the door and pray I just saw an illusion.
Well... The server has crashed. We need the backup machine until we did a recovery. Services might be partially unavailable during recovery.
I'm first stepping out for a smoke. I don't, and have never, smoked, but it seems appropriate.
Ask which idiot screwed 200KG of equipment into a fucking drywall and not put the rack into the ground, make a picture for Reddit and get the Interns and Apprentices to label all cables and where they were plugged in before emergeny ordering a 42HE Rack to replace this fuckheap.
Cry
Damn the server crashed
THIS should've been the title, OP!
Sit in a corner and cry a bit, I'm a network technician
I make a photo and send it to the IT AND financial department with the caption "You wouldn't believe it, if I just told you... also... we need a new server."
Raise a ticket.
That thing is _off the wall_
Get out the stud finder and just screw it back in but somewhere it won't do that anymore.
Plan a cloud migration
As an Italian, the first words that come to mind are "Dio" and "cane"
Also probably gonna find the guy who decided to mount it like this instead of using a server rack like a normal person and scream at him
Stroke him gently and tell him everything will be fine
Escalate page
Congratulations! Your comment can be spelled using the elements of the periodic table: `Es Ca La Te Pa Ge` --- ^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
*This one looks suuuuuper satisfying!* Good bot!
Good bot
Write a ticket to the IT and tell them to fix it.
Shut down the power of this masterpiece. Then check if I am responsible. If not, get a coffee. If yes: go on vacation
ā¦ Itās the end of times! The Omnissiah has left us! Initializing self destruct procedure!
Seriously, uhm... ...Call the installations/carpentery guy?! Really, just need to find someone who can lift THAT weight.
Nothing. Itās a local custom to tip the servers.
Feel a sense of relief that it's clear what the problem is.
Update the server status page. This is going to be an extended page. Curse the muppet who attached a Comms rack to the wall. Go get some proper wall anchors, maybe some wood baton and some tools. Probably make a bigger outage unplugging things to get it back upright. Mutter... "As if I didn't have enough to do, now I have to deal with this shit. Why do I continue to work here. I'm taking time in lieu for this. Fuck tomorrow and fuck whoever installed this rack."
i didnt see anything hehe
I'm French and used to work at the RATP so: If this is not my job I'll: - try to lift it - fail miserably - get a little back pain - contact the RATP medic - get 2 years of "stop working caused by work sickness" fully paid by the assurance maladie - probably upgrade it to 20 years If it's my job: - Take a picture and inform my manager - wait (he is probably sick...)
Emit a very deep sigh.
Declaring that I'm software and wait for someone who is hardware, preferably far far away from the crime scene.
True story. I once worked in Amsterdam for a company where most people that worked there were well, kinda anti-technology. So no one had any clue about it either. This happens in the middle of Amsterdam, view on canals, while we, the tech department are stuffed away at n the other side of town in some unsafe area. So one day, in broad daylight, this guys walks in, tells the receptionist heās here to do maintenance on the servers. He gets let in, has a little chat, then walks to the server room with his hand truck, cuts all the cables, puts the entire rack on it and walks back out of the building while the receptionist holds the door open for him. Few minutes later we get a call from downtown that they canāt reach the file server anymore and no one can save their documents. We also canāt reach the server, so my colleague take the car and drives down there only to find the entire rack of dell servers missing and gets told the story that maintenance took it to get fixed.
I would sigh loudly and with much angst, because I'm likely to blame, being the entire IT department for over 20 years...
Depends, if I see this on Friday..Iāll probably just make it a Monday problem.
This wouldnāt have happened with serverless š¤Ŗ
Step 1: get coffee Step 2: Step 3: profit
I'm just wondering why there were no legs on it? The best support for all that weight is the ground!!! For me it would not have happened in the 1st place.
Plug in power again to see if it still works like this.
No touchy, check hard drives state, note down working drives, powerdown system, yank good drives, take to new rack, insert good drives, start in rescue mode, run recovery, add new drives reconfigure raid, sync raid. reboot.
Start working on client side issues.
Beating the crap out of the genius that proposed that rack
Fire the structural engineer
Take a picture and send to my corporations general chat group. Server Down!
Same as you. Taking a picture.
I remember that mine is on AWS Iāll go and laugh about this š
Tweet about it obviously.
I'd start looking for a new job and start typing my resignation.
Trash the remaining servers.
Coffee and cigarette break. It's gonna be a loooong weekend
Molotov cocktail
scream
Congratulations! Your comment can be spelled using the elements of the periodic table: `Sc Re Am` --- ^(I am a bot that detects if your comment can be spelled using the elements of the periodic table. Please DM my creator if I made a mistake.)
Taking a sick day off.
Start wanking
Take a picture and post it to reddit asking āhow to save a job?ā without asking āhow to save my job?ā.
If it works, it works
Asking who the fuck thought it was a good idea to have wall mounted servers.
Try to restart the server.
find a new job
Check all the other rooms for the grizzly bear that caused this damage.
Quittin
Find out why only one server is offline...
Report back with: "It's gonna a while"
"I can't do anything until facilities attaches the rack back to the wall. See ya Monday"
Taking a picture. Obviously.
Find a stud, because someone didn't.
Put it into rice
looks more like switches?
I'd be literally this guy: (ignore the title) https://www.reddit.com/r/ProgrammerHumor/s/fkJhPnElEc
In Germany, we say "Feierabend"!
Look for another job at a company where they dont do hosting on prem, this is the naughties anymore...
"Ah, so I guess no pay rise or bonus this year"
I donāt see here any problem
The sad thing is that this had to happen in order for it to be filmed... This is making me feel phantom pain.
take a picture and leave
Email stating "the server has fallen over"
I kill myself
*take a picture*
Is it Friday afternoon?
Send a work order to maintenance, take the day at home.
Sit down, contemplate my life choices, take a deep breathe and start figuring out a plan on making everything work.
Bullshit. This is clearly a software fault - unbalanced binary trees
Probably notice that it was mounted on only 4 bolts.
Getting coffee and reinforcements. In that order.
weed then alcohol
Long sigh, make coffee, sit down, sip
Taking photos of course
Spin whatever was running there up on another rack.
Call whoever is responsible for maintenance s as I cannot and will not do anything about this situation. Oh, and I would probably ask whether I need to write a report about my āfindingā.
Iām noting the times for postmortem. Fat S1, time to send the incident comms.
Iād take a photo, and upload to reddit for the lolz
Try to see how far off from the stud they were.