Nested vSphere Lab concept is getting more and more popular than physical lab with a couple of whiteboxes and shared storage. In my usual quest for best vSphere homelab I was recently thinking of different solutions. As always, get the best possible price/performance/usability scenario. Currently in my lab I'm running 2 whiteboxes with vSphere 5.1 installed on USB stick. Both boxes, each with 18Gb of RAM, 2 NAS boxes providing shared storage. In addition I added recently 250 Gb Flash drives to each of those boxes as local storage. You might say, good enough, just wait for vVolumes in v 6.0… -:)
I saw few good labs being mounted recently. Both scaling up for All-in-one Box, and destined to run nested workloads only from local SSD storage. The first one, it's been some time ago, which was very briefly showed by Eric Sloof here Gb of RAM, with Intel E5-2603 1.8Ghz Quadcores CPUs, and the other one by Ed Grigson here, 64Gb of RAM, with 2 x Intel Xeon E5-2603, 1.8GHz, 4 cores. Both running on Supermicro Server Motherboards. In addition, those boards provide remote KVM which can be nice feature to play with.
Both of them I imagine replaced their existing whiteboxes. At least Ed did replaced 3 whiteboxes (each with 8Gb of RAM) in his lab…. Concerning the storage, one can go the SSD way, since the prices of SSDs keeps dropping, and we'll have 1Tb SSD's for $600 in very near future. From Crucial and from OCZ. You can use VMDirectPath and spinup some Nexenta or other VSA to provide shared storage, and I don't think that the performance will suffer too much.
Nesting of ESXi hosts as VMs is used in VMware Labs during VMworld since years and also HOL Online are using this technology. I won't solve the equation on which way to go, today, but just laying down some thoughts of which hardware, which CPU, etc… knowing that those Supermicro boards supports up to 512 Gb of RAM with two CPUs. Is that sufficient for running the whole VMware as vCloud in a Box?
What's your thought? Are you rather multiple boxes – shared storage? Or are you planning going All-in-One vSphere Nested? Want to hear from you. React, comment, share…. -:).
Read also my article on How-to Check if your host is capable to run nested ESXi
Hi Vladan !
Nice article that makes me want to sky the limit of my home lab…
i’ll keep you informed of my configuration…
Cheers,
Karim.
The complexity of managing multiple servers and shared storage (space, heat, power) probably means the nested approach is one I will take soon.
Vladan – you mention you have two NAS Servers now – Are these still FreeNas – and how have you found Freenas over the past 6 months?
Yes, one of them is FreeNAS, but running out of 4 SATA drives. I’m using it mostly for storing my ISOs.. I’m trying to run as much workflow as possible from the local SSDs directly in my 2 boxes.
Do you mind if I ask what the Second NAS box is.
I also have FreeNas, but wondered what other alternatives you are using
Sure, The second one is Drobo, older model called Elite. 8bay SATA hooked through iSCSI. The Drobo is rock solid, but still SATA… I’m kind of ani-spinning disks. Especially when you’re in the same room… -:)
Hello,
Good read, imspired me to have a go at doing an All-In-One running pretty sweet now and expanded a little on the simple home lab. Have just finished a write up if you want to check it out.. http://rammstyletech.blogspot.com.au/
Cheers, and thank you!
Amazing post… -:) Really inspiring..
Hi Vladan,
Nice article, Do you have minimum configuration to make vCloud in one box with nested esxi 6?
Thank you in advance..
Best Regards,
Renault
Depends on what exactly you want to run at the same time. Calculate especially the memory requirements… SSDs storage is not “the price” problem any longer today…