Upon reviewing the logs of my KMS VM, I was quickly able to determine the issue. What could be the issue? I even tried going back to enable host encryption on my ESXi Hosts however, it didn’t work. After pondering I wondered if my ESXi Host came up before the KMS VM? To be sure I rebooted my ESXi Host doing the actual encryption. You will be happy to know my KMS VM was on a separate cluster than my production VMs on the SuperMicro.
Finally, ensure you have a good backup, including offsite, of the KMS DB!
If you are to have a power outage that takes everything offline, ensure the KMS comes up first, before booting ESXi hosts. The important take away here is to always ensure your KMS is up. Always use HA for KMS, always use separate Hardware if possibly, and always use best practices when it comes to redundancy. This is something that is extremely important to keep in mind when designing a solution using VMware Encryption. Simply put, the KMS VM needs to be online first, before your ESXi host doing the encryption asks for the keys. The purpose of KMS is to protect specified VMs using encryption, and it will do it’s job, even if that means you aren’t able to power up the VM. This, as you could imagine, would be very bad, and lock you out of your own VMs. It’s important to note that the ESXi Host doing the encryption only request keys during certain actions, such as Host reboot, VM reboot, etc. If not, your ESXi Host will not be able to get it’s keys needed to unlock VMs, and since your KMS VM would be on that actual ESXi Host using encryption, it would not be able to get the needed keys to unlock itself. That way, if you have a power outage or any other issue that would cause your ESXi host to be unavailable, your KMS VM can boot, and supply the ESXi Host with the needed keys. When deploying a KMS solution, you always want to have the KMS setup using HA, and have redundancy by ensuring your KMS VM is on a separate cluster than your ESXi host that is doing the encryption. To give a little bit of background, my Home Lab consists of a SuperMicro Server, and an Intel NUC. You may ask, why did powering off your Home Lab accidentally cause a Virtual Machine to display a Locked Alarm? Simple, because KMS was deployed, and it was doing its job. This proved to be a very good learning experience since I was able to learn a lot during my resolution of the issue. When I did this several of my VMs were locked, even though I had powered everything back up, and my KMS was up and running.
Everything was working great until I accidentally powered off my entire Home Lab. Previously I deployed a KMS solution within my VMware Home Lab.