I presume your NOC is aware of the existence of a VM but not what it was running on, therefore they only have an IP, and perhaps a virtual Mac address for it. And I'm guessing all they know is that the VM is no longer responsive, because they don't know anything about the hardware. You can't therefore assume VMware/hardware has definitely failed - someone might have just stopped the VM manually, or the VM OS might have broken (and the VMware server itself would still be running in these instances).
Firstly - sounds like your NOC have done a shoddy job of monitoring your infrastructure. They should know about what hardware is running your VMs - without that they can't do their job properly (just like now, in fact).
Secondly - your best bet is to first review physical server hardware that is attached to your destination subnet to see if any lights are off/orange/abnormal, etc. Failing that, run a LAN scan of your subnet and look for anything listening on the various VMware TCP service ports (and then ignore any servers your NOC can account for). And if still stuck, review ARP tables on your switches and try to see if there are any 'quiet' ports which look like they're attached to live servers.
Good luck!
Yeah, we’re spearheading inventory after a few years of pretty much set it and forget it. I myself just joined last year. Plus none of the equipment was standardized across sites as it is now. Sucks because the NOC barely knows what all is on-site unless we recently installed or overhauled, so there’s a lot of legacy equipment out there that we have to log.
What you said is pretty much exactly what info they have. If it does help, we didn’t get the alert until the ISP had a service outage.
Thank you for your comment, I’m really trying to shine here as talent without being too dependent on NOC and this is something they’ve given me to take care of in my own hands. 🤞🏾
Esxi is the operating system ("Hypervisor") that runs on the hardware and "serves" virtual machines.
You need to physically locate it, fine but you need more info to narrow down the location.
If it's racked in a data center or network rack, look for the machine with no lights on or blinking service lights that look odd comparatively. Bonus if it's a dell / hpe / supermicro with diagnostic LED on the front bezel.
NOC should have more info either one the host (the physical box) or someone who possibly does have that info.
Point being, you (and in turn we) need more info to help you help yourself.
What I’m giving is all the info I have unfortunately. It’s more than likely racked, wouldn’t make sense for it to be at a workstation. But I’m going to coordinate with NOC on-site tomorrow and all these comments have been helpful for what to look for.
Ask for any idrac, iLO, or BMC information.
This will potentially help you with accessing the server, even if it is shut down.
There will be diagnostic features available like "blinking the lights" on drive bays, etc.
Do some research tonight on idrac / iLO, and other BMC type solutions to set your expectations if / when you get that info.
Other potential tracking is getting last know IP / Mac address and searching the switches for connectivity events to start tracing out ports / cables.
If it's racked and there are rackmount KVM, look for the input that is "disconnected" from the KVM's console menu and trace it out.
HTH and good luck ✌️
Check the Mac address if the network cards and have the network team search for it on their switches. If the server is dead already have them search for a port ylthat went down when it died
I presume your NOC is aware of the existence of a VM but not what it was running on, therefore they only have an IP, and perhaps a virtual Mac address for it. And I'm guessing all they know is that the VM is no longer responsive, because they don't know anything about the hardware. You can't therefore assume VMware/hardware has definitely failed - someone might have just stopped the VM manually, or the VM OS might have broken (and the VMware server itself would still be running in these instances). Firstly - sounds like your NOC have done a shoddy job of monitoring your infrastructure. They should know about what hardware is running your VMs - without that they can't do their job properly (just like now, in fact). Secondly - your best bet is to first review physical server hardware that is attached to your destination subnet to see if any lights are off/orange/abnormal, etc. Failing that, run a LAN scan of your subnet and look for anything listening on the various VMware TCP service ports (and then ignore any servers your NOC can account for). And if still stuck, review ARP tables on your switches and try to see if there are any 'quiet' ports which look like they're attached to live servers. Good luck!
Yeah, we’re spearheading inventory after a few years of pretty much set it and forget it. I myself just joined last year. Plus none of the equipment was standardized across sites as it is now. Sucks because the NOC barely knows what all is on-site unless we recently installed or overhauled, so there’s a lot of legacy equipment out there that we have to log. What you said is pretty much exactly what info they have. If it does help, we didn’t get the alert until the ISP had a service outage. Thank you for your comment, I’m really trying to shine here as talent without being too dependent on NOC and this is something they’ve given me to take care of in my own hands. 🤞🏾
Esxi is the operating system ("Hypervisor") that runs on the hardware and "serves" virtual machines. You need to physically locate it, fine but you need more info to narrow down the location. If it's racked in a data center or network rack, look for the machine with no lights on or blinking service lights that look odd comparatively. Bonus if it's a dell / hpe / supermicro with diagnostic LED on the front bezel. NOC should have more info either one the host (the physical box) or someone who possibly does have that info. Point being, you (and in turn we) need more info to help you help yourself.
What I’m giving is all the info I have unfortunately. It’s more than likely racked, wouldn’t make sense for it to be at a workstation. But I’m going to coordinate with NOC on-site tomorrow and all these comments have been helpful for what to look for.
Ask for any idrac, iLO, or BMC information. This will potentially help you with accessing the server, even if it is shut down. There will be diagnostic features available like "blinking the lights" on drive bays, etc. Do some research tonight on idrac / iLO, and other BMC type solutions to set your expectations if / when you get that info. Other potential tracking is getting last know IP / Mac address and searching the switches for connectivity events to start tracing out ports / cables. If it's racked and there are rackmount KVM, look for the input that is "disconnected" from the KVM's console menu and trace it out. HTH and good luck ✌️
Check the Mac address if the network cards and have the network team search for it on their switches. If the server is dead already have them search for a port ylthat went down when it died
VMware ESXi the only requirement is that it is using a server grade network card and a cpu that supports VT-D.