ESX Virtualization

VMware ESXi, vSphere, VMware Backup, Hyper-V... how-to, videos....

What Is Log-Structured Write Cache?

By | Last Updated:

Shares

Log-Structured Write Cache (LWC) is composed of RAM but also small SSD pool. This way, an application with random writes, can have optimal performance, compared to a cache using RAM only. But it is also more secure as logs are written to SSD tier and replicated to other nodes within the cluster. StarWind VSAN uses LWC within their software and allows also to optimize the data flow. This post will try to explain What Is Log-Structured Write Cache?

As you know, today, a system without any cache, cannot really satisfy high IO demands from any virtual infrastructure. Even the smallest ones. That's why since over a decade, storage devices are loaded with a cache.

The cache can be RAM, SSD of a different kind (SAS, SATA, NVMe…). Write-back cache data is written to the underlying storage through RAM, usually. If there is a failure, the RAM content is gone, lost.

However, coupled with an SSD, the data are protected and in case a host goes down, the full resync does not even have to be triggered as the logs stored in the SSD pool are not lost. (They would be in case of RAM only).

As such, the architecture is more resilient, clusters can have less maintenance, and the recovery process is much faster without global resync.

Quote:

StarWind Virtual SAN writes the data to the RAM cache first. Then, the data is flushed sequentially to the log device formed out of a tiny portion of your disk storage. The log keeps a track of consistent states for quick recovery in the event of a blackout or a cluster-wide failure Subsequently, all the data is sent to the underlying storage where it eventually resides. In this way, LWC ensures high application performance even under highly randomized I/O loads.

The RAM is still used for caching, but the time the data stays in RAM is fairly small, so the risk is minimized here.

Log-Structured Write Cache (LWC) Requirements

(from “Configuring Log-Structured Write Cache HA device” PDF. A link will be in the sources section at the bottom of this article).

  • HDDs and SSDs – HDD drives as the underlying storage and SSD drives for a cache-disk(s).
  • To collect RAM cache files and their metadata, StarWind requires at least 3GB of RAM per 1TB of storage. In case of the long-term peak loads, required RAM can be over-provisioned.
  • For cache-disk, StarWind service needs at least 16 GB of storage, but it can grow up to 20% of the overall HA device size. In case of the long-term peak loads, cache-disk size can be
    over-provisioned. Size of the storage available for Write log parameter files varies from 10%
    to 20% of the overall underlying storage capacity.

No Full Sync Needed

In case of a failure, you don't need to wait for full synchronization.

Quote:

If all of a sudden a node or entire cluster goes down, the log disk is the only place that nodes need to check to restore data integrity. No full sync is needed. Nodes just quickly match log disks and the cluster is good to go!

How to configure?

You can simply follow the PDF where you'll see the exact sizing that StarWind recommends and which device you'll need to create within StarWind console. And there are also other tweaks and advice there.

Sources:

StarWind continues to innovate and brings solutions to problems with a flexible approach of possibility to create log-structured write cache protected workloads which are more resilient, easy to configure with advantages of a quick recovery in case of hardware problem. The full resync isn't necessary.

StarWind Software has a free version too

StarWind Virtual SAN Free is completely unrestricted: it is allowed for production use, supports all usage scenarios of the commercial version, has a perpetual license, and is not feature- or functionality-limited version of StarWind VSAN.

  • No Capacity Restrictions – you can use as many capacities for your mirrors, as you like (previously restricted)
  • No Scalability Restrictions – as many nodes as you like. (previously limited to 2-nodes only)
  • No Time Limit on License – The Free license if for life. After 30 days, the only management option you’ll have is PowerShell or CLI.
  • Production use – can be used in production, but if anything goes wrong, you will only find support through community forums.
  • PowerShell Scripts – StarWind Virtual SAN Free is shipped with a set of ready to use PowerShell scripts allowing users to quickly deploy the Virtual SAN infrastructure.
  • No StarWind Support – only community-based support.
  • StarWind HA – The shared Logical Unit is basically “mirrored” between the hosts, maintaining data integrity and continuous operation even if one or more nodes fail. Every active host acts as a storage controller and every Logical Unit has duplicated or triplicated data back-end.

Worth to note that StarWind software can be also used in scenarios for on-prem workloads running as a stretched cluster.

More posts about StarWind on ESX Virtualization:

More posts from ESX Virtualization:

Stay tuned through RSS, and social media channels (Twitter, FB, YouTube)

Shares
Vote !

About Vladan SEGET

This website is maintained by Vladan SEGET. Vladan is as an Independent consultant, professional blogger, vExpert x16, Veeam Vanguard x9, VCAP-DCA/DCD, ESX Virtualization site has started as a simple bookmarking site, but quickly found a large following of readers and subscribers.

Connect on: Facebook. Feel free to network via Twitter @vladan.

x