Nested ESXi technology is fascinating. In my recent article Nested vSphere Lab -Ideas? , I was thinking about what hardware, or which way to go for a nested lab. By using nested virtualization It's great way to learn more about those virtualization technologies, without ruining yourself. Not only you can learn about VMware but also Hyper-V.
There might be other hypervizors as well which can be installed as nested, but I think already mastering VMware and Hyper-V for a usual IT guy like me, it's already enough. Learning, testing, playing with more hardware than you own (because it's virtual), it's always fun. But how can you be sure that your server (host) is actually capable of running nested ESXi servers? It's quite easy to test that. To avoid you being disappointed when trying to install and virtual ESXi, plus then installing VM in VM, there is a little trick on how to check if your host is capable to run nested ESXi (and 64bits VMs inside). All you need to know is one command line.
Try this: https://IP_Of_Your_Host/mob/?moid=ha-host
The host will return a page where you'll see many links. At the top there is one link called capability.
When clicked, it opens second page, where you can do a search for nestedHVSupported. You should see this line in the middle of the page.
If you don't see that, you will still probably be able to run ESXi as nested hypervizor (as a VM) but you'll be able to run only 32bit VMs inside of that virtual ESXi.
A quick prerequisites concerning the capability for nested virtualization, when buying new CPU or new PC:
- Look for Intel VT-x or AMD-V – to supports nested 32-bit VMs
- look for Intel EPT or AMD RVI to support nested 64-bit VMs.
In any case, to run nested ESXi servers is unsupported by VMware.
So by checking prerequisites for the CPU, you can at least check that you'll be able to run nested 64bit VMs inside of nested ESXi. But that might not be enough, since there are other drivers and components that might cause problems when installing the ESXi as bare metal.
So when planning to build (buy) your new system, actually the best way is to check on VMware HCL for ESXi 5.x (Hardware compatibility list). You can be absolutely sure that the ESXi 5.1 installs on the bare metal without warnings or driver's problems.
Is the process of creating nested ESXi 5.1 difficult? NO…. -:)
You'll need to create new VM compatible with Virtual Hardware Version 9.
Then, as an OS, select other 64 bit > and personalize the CPU section by checking the box “Expose Hardware Assisted Virtualization to the guest OS”. You can click to enlarge.
Click finish. Then fire up your vSphere Windows client and change the VM settings. Edit the virtual hardware of the VM. We need to change the OS type to ESXi 5.X. Note that this can currently can only be done through the Windows client as in the new vSphere Web client it's grayed out.
Don't forget to give that VM more than the default 512 Mb of RAM (4Gig is necessary for ESXi 5.5 ) …….and that's about it. Now just attach the latest ISO and you can install the ESXi.
Update: Quite important note to add, concerning the number of CPU cores. If you configure your virtual ESXi with only single CPU and with single core only, during the installation you'll run into an error:
<CPU_Core Error: This Host has [1] cpu core(s) which is less than recommended [2] cpu cores>.
So make sure that you choose Two vCPU cores when modifying the VMs properties. If not the installer just throws an error and stops. You can also configure the VM with Two CPUs so you won't see the error either.
Article sources:
- VMware Documentation (NestedHWsupport)
- William Lam's Blog (VMware)
- Nested Virtualization Best Practices (VMware Community)
- Tim Smith's Blog
Tim Smith says
Good writeup. Thanks for the link in the article sources. Running nested ESXi is a great way to lab up a lot of vSphere products for testing and training.
Datto says
ESXi 5.1 nested on ESXi 5.1 physical might be supported — see this post:
http://communities.vmware.com/message/2166303#2166303
Datto
Vladan SEGET says
That’s an interesting find, basically enabling to test the nested virtualization on production servers (ESXi 5.1) and still benefit from support for existing (non nested) workloads.
Thanks for the info.
fadhli nordin says
thanks for the good info 🙂
david gnad says
Very good info. It helped me. Thanks.