Which EVC mode should I use?

According to Moore’s Law, the number of transistors on an integrated circuit chip will double every two years (http://en.wikipedia.org/wiki/Moore’s_law). This loosely translates into new CPUs released every year, and not only do those CPUs perform faster, but also have more features.

One of the best capabilities of a VMware based cluster of vSphere hosts is that you can add in new hardware without having to perform a ‘rip-and-replace’ or ‘forklift’ upgrade.

New physical hosts can be added to an existing cluster in an iterative way, but this poses a small problem for some VMs. If you power on a virtual machine on a host that has a new CPU, part of the early stages of initialisation of the virtual hardware involves the VM being informed about what CPU capabilities exist on the underlying hardware. Unfortunately, what would happen when a VM is migrated with vMotion to another host that does not have the same capabilities is that the VM would “lose” some CPU functions – normally something that an Operating System does not like. So, Enhanced vMotion Control (available since August 2008 in ESX 3.5 Update 2, and all ESXi releases) performs the function of masking off the features of new CPUs so that all VMs within the cluster are using the same level of features and capabilities.

So, you might think that this is a bad thing. Reducing the capabilities of the virtual CPU? Wouldn’t that mean there is no point in buying a new CPU, because it is going to be reduced to the lowest level of the rest of the cluster? I’ve got a new CPU, why do I want to reduce it’s functionality?

If you have a look at what is being masked off by EVC in the “EVC and CPU compatibility FAQ” http://kb.vmware.com/kb/1005764 and http://kb.vmware.com/kb/1003212, you will see the list of features [instruction sets] for each family of CPU, but note that it is features that are masked off, not gigahertz or cache size, not QPI or HyperTransport, not Turbo functions or memory speed. So, by enabling EVC modes, you are not actually reducing these performance capabilities of the physical CPU.

So, adding a host with an Intel SandyBridge based CPU into a cluster, set at the lowest Intel Merom (Xeon Core2) EVC mode will mask off some features [instruction sets] of SandyBridge, but will not reduce it’s speed. VMs will be able to be migrated to and from the new host without error, but when a VM is on the SandyBridge based host, the VM will operate faster – benefiting from the advances of the newer processor.

So, what is the impact of changing the EVC level on your cluster? One thing to note is that VMs will not get the benefits of the new EVC mode until they are powered off and then powered on again (not a reboot). Unless you have systems that are using the features of a new CPU family – such as advanced functions like multimedia, encryption and the like – you probably won’t notice much difference.
VMware released a whitepaper in August 2012 – “Impact of Enhanced vMotion Compatibility on Application Performance”  where analysis was done on the various EVC modes with real world workloads, and identified that unless you are doing x264 video encoding, AES encryption tasks, EVC modes don’t have much impact on performance.

So, in summary, don’t panic about a “low” EVC mode – the feature enables you to create clusters of miss-matched hosts without sacrificing performance of newer hardware.

I would love to know your views in the comments below