VMware CPU Ready metric is used to see a percentage of time that the virtual machine was ready, but could not get scheduled to run on the physical CPU. CPU ready time is dependent on the number of virtual machines on the host and their CPU loads. Usually, to see the values, an “esxtop” command is used to determine if the ESXi/ESX server is being “overloaded”. This post is called What is VMware CPU Ready and it is no mean to be a deep-dive. There are other resources on the web where you might want to look for much deeper explications.
We will try explain the CPU ready for folks who start with virtualization, so if you're an experimented VMware admin, you might just skip this post.. -:).
Usually, it is normal for a VM to have small values for CPU Ready Time if the hypervisor is not over subscribed or under heavy activity. This is because of the nature of shared scheduling in virtualization.
VMs which are configured with multiple virtual central processor units (vCPUs) the amount of ready time will be higher than for VMs with fewer vCPUs since it needs more resources to schedule or co-schedule the VM when necessary and each of the vCPUs accumulates the time separately. (also called SMP processing).
Symmetric Multiprocessing (SMP) processing is the processing of programs by multiple processors which are sharing OS and memory. In symmetric multiprocessing, the processors share memory and the I/O bus (or data path we say too). But single OS manages all CPUs.
Symmetric multiprocessing – Quote Wikipedia:
Symmetric multiprocessing (SMP) involves a multiprocessor computer hardware and software architecture where two or more identical processors are connected to a single, shared main memory, have full access to all I/O devices, and are controlled by a single operating system instance that treats all processors equally, reserving none for special purposes. Most multiprocessor systems today use an SMP architecture. In the case of multi-core processors, the SMP architecture applies to the cores, treating them as separate processors.
Where to check the CPU Ready values?
Examine the %READY (RDY% in esxtop) field for the percentage of time that the virtual machine was ready but could not be scheduled to run on a physical CPU.
Under normal operating conditions, this value should remain under 5%. If the ready time values are high on the virtual machines that experience bad performance, then check for CPU limiting:
- Make sure the virtual machine is not constrained by a CPU limit set by itself
- Make sure that the virtual machine is not constrained by its resource pool.
There was at one point a recommendation from VMware that anything over 5% ready time per vCPU was something to monitor, but it was a few years back….
When is a host too oversubscribed?
It means when VMs running on a host will perform slower and slower. When too many vCPUs have been allocated per pCPU ratio. So it depends.
Your mileage may vary based on your specific VM workloads, but usually, we can see problems when a host is between 2-2.5 times oversubscribed (server workloads). VDI is another story…
But what can be done?
One of the ways might be to start by doing small evaluating if the VMs need to have the number of configured vCPUs. And if that's the case, some of them might be able to be reconfigured to lower the ratio.
If this cannot be done, then add additional hosts to allow the load to be balanced better and reduce the oversubscription. If the problem is specific to the larger SMP VMs only, such as VMs like SQL Server, Microsoft Exchange Server, you might want to evaluate if you can consolidate those larger VMs onto one or several hosts and move the smaller VMs to the other hosts in order to separate the VMs based on their sizes.
Tools to monitor VMware CPU ready?
As being said, ESXTOP through putty session is one of the easiest and quick ways to check the CPU ready metrics. Then products such as vROPs can be determining while dealing with larger scale architectures.
Wrap up
If you're starting to see 5% or more values, then this should be a warning for you and you can already start to experience performance degradation especially if you're running some high-performance applications within the guest such as SQL Server, or MS Exchange Server.
More from ESX Virtualization
- What is VMware Cluster?
- What is VMware Storage DRS (SDRS)?
- How To Disable vSphere Web Client Inactivity Timeout
- What is The Difference between VMware vSphere, ESXi, and vCenter
- How to Configure VMware High Availability (HA) Cluster
- What is VMware vSAN Caching Tier?
Stay tuned through RSS, and social media channels (Twitter, FB, YouTube)
“we can see problems when a host is between 2-2.5 times oversubscribed”
Was this a general statement, or was it tied to your specific environment?
We are currently 3:1 oversubscribed of vCPU’s to pCPU and have CPU ready less than 0.5%
Our CPU usage is about 20% of the total GHZ, so we are not taxing the environment.
Thanks for your coment, kendal.
Are your VM performing slow or just wondering how “far” you can go? Are your VMs running heavy workloads or just “sitting” there?
It was a general “thumb” rule and every environment is kind of specific concerning the workloads it is running so not possible really to give one rule for all.
Congratulations on the great content.
Can you help me with this doubt?
What calculation do I have to do to scale the amount of vCPUs to my VMs by the CPU capacity of the host?
Exe: I have a host with vSphere 6.0 with the following CPU configuration:
2 Sockets
6 Colors per Socket
24 Logical processors.
How many VMs of 1 vCPU could I run on that host?
Thank you!
Goodnight
I have a client that has on its vROP platform. I made a report where I indicated to validate servers with maximum capacity exceeded, the report tells me that they have vCPU in excess, an example:
a VM with 10 vCPU assigned, VROP tells me that it can recover 5 vCPU, so I recommend leaving the VM with 5 vCPU.
I also have another report with VM with insufficient size, where it tells me that I should assign more memory resources.
Example: VM with 6 GB, vROP recommends me to add 1 GB and otherwise I have a VM with 8 GB and I recommend adding 4 GB.
A VM with workload at the memory level for a maximum of 5 minutes in 103.61% and 99.99% maximum in 1 day.
Is it recommended to assign or remove resources following the vROP recommendations or statistics?
Obviously I have to do a performance study, but the client has indicated that at the performance level their vM do not present any problems.
Please your comments.
What is the difference between ready and readiness in the performance monitor? I am getting up to 90k ms ready, but only 2.6% readiness.