During this year's VMworld in Barcelona 2014 I had an opportunity to be part of Tech Field Day Extra organized by Steven Fosket (@sfosket). It was my first Tech field day (hope not last one) and I was present together with other candidates on this presentation from Diablo Technologies. They're working on Memory Channel Storage (MCS) which is a very high performance storage.
I must say that I'm not a storage nerd. I know storage as a VMware admin or the IT admin but those guys are rather specialists in storage, but even with my limited storage knowledge I was able to follow this exciting technology which is hitting the market early 2015. Currently Diablo Technologies collaborates with some OEM manufacturers (IBM, SuperMicro, SANDISK) to integrate their technology with SSD chips into the newest hardware. They also work closely with VMware.
Today's SSD's the way they are connected to the motherboard are usually through SATA/SAS or PCIe. However those interfaces are still limited in bandwidth and access times resulting some latency, even compared to spinning disks SSDs via SATA/SAS or PCIe offers superiour performance and low latency. Diablo Technologies offers superior performance and lower latency by placing SSD chips via modules which are plugged where usually RAM Memory chips goes. This Memory Channel Storage Solution offers parallel access to the SSD's present in those RAM slots the access times are the fastest ones and the latency lowest.
Deploy Flash within the Memory subsystem
Access flash from within the memory subsystem and the NUMA architecture. The process leveraging VDR bus (not PCI bus) which is the most performant, parallel bus the motherboard offers. It's using rigid protocol which usually communicates with DRAM modules so it has very closely time transactions.
Latency or Throughput – or Both?
Traditionally with PCIe interface there is usually large capacity of flash behind a PCIe bus. While it's possible to get relatively low latency, when the IO throughput is low. But when the IO throughput is high the single ASIC has to do a lot of management tasks (garbage collection etc…) the latency can spike.
Diablo Technologies because it's using the VDR Bus and the Memory Channel Storage technology it's possible to get low latency AND high throughput..
But How the Motherboard knows that it's not a usual RAM module?
In fact the motherboard's bios has to have some microcode and has to be aware which slots are tagged for holding the storage module. The sample looks like this. One would say it's RAM. Diablo works as DDR3 and DDR3 and it's also possible to flash current bios with an update which includes the Diablo's Code.
To the host it's presented as a block device – as a storage device.
What happens if there is a failure?
If there is a failure, the system detects a failure and the slot is marked as failed. But the hardware supports full 10 writes a day for 5 years which is an enterprise grade hardware. SANDISK chips uses guardian technology and in addition, the modules do support RAID config.
Quote from Diablo Technologies PDF:
Diablo’s Memory Channel Storage (MCS) architecture achieves end-to-end parallelism (i.e. no bottleneck) by leveraging the server’s natively parallel memory subsystem. Each MCS module plugs into a standard DDR3 DIMM slot and is directly available to the CPU’s memory controller. Memory controllers were designed to effectively manage massively parallel, time-sensitive, high-speed data access (e.g. to DRAM). MCS DIMMs are populated on multiple memory controller channels and media management tasks are dispersed across those channels in a distributed fashion.
When compared to the traditional storage, we can see the lateness ans disc queue depth
Video Presentation:
MCS and Todays VSAN design benefits:
Update: Diablo Technologies conducted a tests where on one site there was an architecture using PCIe as a flash tier for VSAN, and on the other side the same architecture there was a MCS. The architecture looks like this:
And the results in View Planner shows the differences when opening applications, opening and saving documents.
The Future – Two Tier Flash Strategy for Next Generation of VMware VSAN
The second video shows some cool stuff that VMware is working on (and more). VMware and Diablo are working together and testing full flash VSAN. In fact it's a 2 tier flash strategy for next generation of VMware VSAN where a low cost SSDs (MLC/TLC) are used as tier 2 and high performance Flash (The ones from Diablo) are used for the fastest tier.
The strategy looks like this:
You'll see the two tier flash strategy in this video as well as the schema how this technology works:
The technology looks very cool with modules integrates on the motherboard – as storage devices. But I must say that when using some slots usually destined for DRAM modules, those slots might be missing then when you want to upgrade for more RAM. But it's possible to leverage both by architecting upfront certainly. The main point to retain is the low latency with high throughput which this technology allows. And it's the fastest SSD technology soon to be available.
Disclaimer: Diablo Technologies was a presenter at the Tech Field Day Barcelona. They did not ask for any consideration in the writing of this review nor were they promised any. The conclusions and analysis contained in this post are mine and mine alone.
First Video: Jerome McFarland (Diablo)
Second Video: Ricky Trigalo of Diablo (Diablo) and Daniel Beveridge from VMware
Diablo Technologies
i’m surprised, I thought Sandisk and IBM were shipping ULLtraDIMM in volume now, in 2014?
Also, have you heard of what Netlist is launching in 2015 with HyperVault? It sounds like true converged memory merging DRAM and Flash storage.