Since several years of my IT career every time I decide to expand my lab is to build a new ESXi host or a box which does shared storage for my lab. So it means that I was always left with my laptop to do all the other work. The Samsung laptop I use is still ok, but 16 Gb of RAM is maxing pretty fast those days when using VMware Workstation. And then one day I said to myself – I don't even have a system with a lot of RAM and power to spin more Workstation VMs and encode some videos! How about 64Gb RAM Box With Hexacore CPU?
Well that's why a new system was born -:). This time I went for an architecture based on Intel Core i7-4930k with a motherboard supporting 64 Gb of RAM. A Windows 8.1 box ( with a classic start menu … ) and VMware Workstation 10 installed. A very flexible system using solid state storage disk only.
Update: the host has been transformed to become ESXi host in my lab.
The CPU is a hexacore CPU based on socket 2011, with plenty of power – cpubenchmark.net page. The socket 2011 is an older socket, but as Haswell desktop CPU only supports 32Gb I went this way. So unfortunately the TDP of the CPU is 130W, but the box should not be On all the time (24×7). The Haswell ESXi whitebox I've done last year has only 65W TDP.
VMware Workstation is at the point of technology. The folks in the Workstation team brings new features before anyone else at VMware, and those features are usually found later on on next release of VMware.
So having a system with lots of RAM makes sense as this box is transportable, I can do demos at the client's sites or provide training class for few students. You can do quite a lot with single box beefed with hexacore CPU, 64 Gb of RAM, and few nested ESXi….
Which usage?
- Portable lab for demos, training.
- As a home PC and entertainment box
Specifications:
- CPU: Intel i7-4930k
- Motherboard: Gigabyte X79-UP4
- RAM: 8x8Gb (non ECC) – the mobo supports ECC too..
- Graphics: MSI GTX 760
- Storage SSD 3x Crucial m500 (480Gb) and 1x Samsung 840 Pro (512 Gb)
The underlying W8.1 is ok. The plan is to use the Workstation as a main tool with this beefy system plus install some soft to treat videos, perhaps some games, but I'm not a gamer…. -:). Workstation can also connect remotely to my main lab where I currently run a VSAN cluster.
The graphics card is hooked to two 24′ monitors which gives me quite comfortable working area in my office…-:)
RAM is the first resource which gets exhausted in the lab. If I wanted to give an advice for building ESXi home lab, I think that this time I'd go for a system supporting 128Gb of RAM at least. I know it seems quite overkill, but the CPU is rarely constrained. RAM is also the most expensive investment as 16Gb sticks are costly.
Storage is another story, but the prices of SSDs are falling … I'd say that the end of 2014 and 2015 will be a year of PCIe based Flash storage devices as the 6Gbs SATA is unfortunately becoming a bottleneck. The SATA express motherboards too will start making lots of noise later this year.
Update: I think it might be useful to add the cost of the system, including everything. Some prices has fallen since as for example the Samsung SSD being purchased already last year…. Also this list is more/less for informative purposes, but includes All parts are totalling at €2687 (or if you like in USD, roughly $3493). Or fo our UK readers, it should be about £2126.
I hope you enjoyed this saturday's post and that you will stay tuned for other articles, new, videos and tutorials about VMware virtualization and general IT. You can grab my RSS, follow me on Twitter (@vladan) or other social media networks.
Ather Beg says
Hi Vladan,
I recently did a similar system but with 128GB RAM as you’re also recommending now. RAM is the thing most labs are going to be constrained on so I chose something that acommodates more. I also went for a Dual-Xeon set up because in my previous machine (similar to yours above) was getting a bit CPU-bound as well.
I documented the whole thing in a three-part series, starting from this one:
http://atherbeg.com/2014/06/19/my-new-dual-xeon-based-nested-home-lab-machine-part-i/
Ather
Vladan SEGET says
Thanks for the input Ather. Cool and powerful rig on your side. Two Xeons… wow. If you add the 64Gb of RAM to get the 128Gb, you’ll end up with £3329 (€4205). It starts to be pricey… -:)
Ather Beg says
RAM already added! You’re right, the machine is indeed expensive. That said, one has to pay for all that power and I am hoping it’ll give me a couple of years of usage without needing more.
Ah well! All in the name of developing oneself! 🙂
Ather
Les Garten says
Hi,
I’ve seen some amazing things done with 16GB of RAM. What are you guys doing in a home lab that requires so much RAM?
I am new to ESXi. I just bought an IBM x3650 M3 with two Hex core Xeons and 24 GB of RAM. I thought I had a great box for this untill I saw you guys adding a TB of RAM!! I was considering adding more RAM at some time in the future. This is a home Lab for me.
ROTOR says
Hi!
Did VT-d works with this mobo the GA-X79-UP4, vmdirectpath, vga passthrough?
Can you try it?
Thanks!
Vladan SEGET says
VT-d yes, but haven’t tried the vmdirect path or vga passthrough. I’m using this board as a Desktop system. (Workstation installed). Not as ESXi.