1. Enhanced vMotion Compatibility
Product Support Engineering
VMware Confidential

2. Module 2 Lessons Lesson 1 – vCenter Server High Availability
Lesson 2 – vCenter Server Distributed Resource Scheduler
Lesson 3 – Fault Tolerance Virtual Machines
Lesson 4 – Enhanced vMotion Compatibility
Lesson 5 – DPM - IPMI
Lesson 6 – vApps
Lesson 7 – Host Profiles
Lesson 8 – Reliability, Availability, Serviceability ( RAS )
Lesson 9 – Web Access
Lesson 10 – vCenter Server Update Manager
Lesson 11 – Guided Consolidation
Lesson 12 – Health Status
Agenda Overview
VI4 - Mod Slide

3. Module 2-4 Lessons
Lesson 1 – Overview of Enhanced vMotion Compatibility
Lesson 2 – Compatibility Matrix
Lesson 3 – EVC Baselines
Lesson 4 – Requirements for EVC
Lesson 5 – Enabling EVC on a Cluster
Lesson 6 – Troubleshooting EVC
Agenda Overview: This is the same EVC which was introduced in ESX 3.5 U2 which vSphere 4.0 is compatible with. The new improvements for 4 allows the creation of multiple CPU groups in the datacenters.
VI4 - Mod Slide

4. OEM VMotion Compatibility Matrix
Source: Similar matrices are available for HP and IBM servers
VI4 - Mod Slide

5. Enhanced VMotion Compatibility (EVC)
EVC allows vCenter to enforce VMotion compatibility between all hosts in a cluster by forcing hosts to expose a common set of CPU features (baseline) to Virtual Machines.
EVC automatically configures servers whose CPUs feature Intel FlexMigration and AMD-V Extended Migration technologies to be VMotion-compatible with servers that use older CPUs.
EVC ensures that all hosts in a cluster present the same CPU feature set to Virtual Machines, even if the actual CPUs on the hosts differ.
This prevents migrations with VMotion from failing due to incompatible CPUs.
Beta2 what’s new -
Uses hardware support developed with AMD, Intel.
EVC leverages Intel FlexMigration technology to present the same feature set as the baseline Intel processor.
EVC leverages AMD’s AMD-V Extended Migration technology to present the same feature set as the baseline AMD processor.
All hosts in the cluster must either have hardware live migration support (Intel FlexMigration or AMD-V Extended Migration) or have the CPU whose baseline feature set you intend to enable for the cluster.
VI4 - Mod Slide

6. Detecting CPU Features
OS or application software executes CPUID machine instruction
CPUID instruction reports many system properties:
Vendor (e.g. Intel or AMD)
CPU family, model, stepping
Supported CPU features, e.g.:
NX/XD (No execute; memory protection from malware)
AMD-V/VT-x (Virtualization support in hardware)
SSE3 (CPU instructions to optimize streaming applications)
Number of CPU cores, cache size, and many other properties
VI4 - Mod Slide

7. EVC Benefits Enables VMotion across CPU generations Simple
New CPUs are automatically configured to be compatible with earlier versions.
Makes it much easier to add new hardware to existing clusters.
Simple
No manual CPUID masking required
VI4 - Mod Slide

8. Intel EVC Cluster With Different Generation CPUs
App OS
Intel Core
2 VM
ESX
Intel Core 2 CPU
ESX
Future Intel CPU
ESX
Intel Core 2
45nm CPU
Merom are Intel Core 2 CPUs.
Penryn are Intel Core 2 45 nanometer CPUs.
One of the difficult aspects of EVC is the naming convention – Intel would not allow us to use the codenames like Merom and Penryn so we have to use references like 45 nanometer to distinguish between them.
VM sees Intel Core 2 level CPU features and can migrate to any host in the EVC cluster
VI4 - Mod Slide

9. AMD EVC Cluster With Different Generation CPUs
App OS
Opteron
Rev E VM
ESX
AMD Rev E CPU
ESX
AMD Barcelona CPU
ESX
Future AMD CPU
AMD Barcelona is the third generation of AMD processors, K10 architecture. (K10 supersedes the earlier K8 architecture)
VM sees AMD Opteron Rev E CPU features, can migrate to any host in the EVC cluster.
VI4 - Mod Slide

10. EVC Baselines
Baseline: a set of CPU features that is supported by every host in the cluster
Baseline is the least common denominator of all hosts, or less
In ESX 3.5 u2, one baseline per CPU vendor
Intel: CPU features supported by Merom cores
AMD: CPU features supported in Opteron Rev E/F
In VI4, it is expected that two or more baselines can be defined, e.g.:
Intel: Merom, Penryn (SSE4.1), Nehalem (SSE4.2)
AMD: Rev E/F and Greyhound (SSE4A, ABM)
VI4 - Mod Slide

11. EVC Baselines (ctd) EVC Baseline Compatible CPUs
Intel® Core™ 2 (Merom)
Intel® Core™ 2
Intel® 45nm Core™ 2
Intel® 45nm Core™ 2 (Penryn)
AMD Second Generation Opteron™ (Rev. E/F)
AMD Second Generation Opteron™
AMD Third Generation Opteron™
AMD Third Generation Opteron™ (Barcelona)
At VMworld 2008, Paul Maritz announced Intel's Xeon 7400 Series chips, code-named Dunnington ,which were released the previous day, can "do heterogeneous VMotion across different rev levels [versions] of the chips, whereas now you have to make sure you stick within the same chip family”.
With the release of Dunnington, the six-core Xeon chip officially known as the Xeon 7400, Intel moves away from the Penryn to the Nehalem architecture.
Multiple baselines allow the user to choose:
Greater compatibility (fewer cpu features)
More features (less cpu compatibility)
Additional baselines will be introduced for new CPU generations
VI4 - Mod Slide

12. EVC baselines & CPU Models
VMware KB article lists specific CPU models and which baselines they support
Examples:
Intel Core 2
Intel Merom baseline: 73xx (Tigerton), 51xx (Woodcrest)
Intel Core 2 45nm
Intel Penryn baseline: 74xx (Dunnington), 54xx (Harpertown)
AMD 2nd Generation
Opteron Rev E/F baseline: 2yy, 8yy, 22yy, 82yy
AMD 3rd Generation
Greyhound baseline: 23yy, 83yy
VI4 - Mod Slide

13. EVC Cluster With Intel Core 2 45nm (Penryn) Baseline
ESX
Intel Core 2
45nm CPU
Intel EVC Cluster (Core 2 45nm baseline)
App OS
Core 2
45 nm VM
ESX
Intel Core 2
45nm CPU
ESX
Intel Core 2 CPU
ESX
Future Intel CPU
VM cannot migrate to an Intel Core 2 (Merom) based CPU
It can migrate to an Intel Core 2 45nm (Penryn) based CPU outside the cluster
VI4 - Mod Slide

14. The E54xx CPUs are ‘Harpertown’ processors from the Penryn series
Determine CPU Model
vCenter will display model information for CPUs that already have ESX installed
For new servers that do not have ESX installed, use some of the freeware utilities such as CPU-Z.
The E54xx CPUs are ‘Harpertown’ processors from the Penryn series
VI4 - Mod Slide

15. EVC & CPU Features EVC does not affect CPU features
Number of cores per CPU. For example, a Greyhound (quad core) CPU does not lose 2 cores when it’s added to a Rev E/F (dual core) EVC cluster
Cache size
Hardware virtualization support (VT-x, AMD-V, nested paging)
Clock speed. Thus, EVC does not cause any performance penalties
Worst case scenario from implementing EVC: a VM cannot take advantage of new CPU instructions, e.g. SSE 4.1
VI4 - Mod Slide

16. Requirements For EVC EVC requires ESX 3.5 update 2 or later.
EVC requires Intel CPUs with Core 2 micro architecture and newer, e.g.
Merom: 73xx (Tigerton), 51xx (Woodcrest), 53xx (Clovertown)
Penryn: 74xx (Dunnington), 54xx (Harpertown)
EVC requires AMD second generation Opteron CPUs and newer, e.g.
Rev E/F: models 1yy, 2yy, 8yy, 12yy, 22yy, 82yy
Greyhound: models 13yy, 23yy, 83yy
EVC requires a homogenous cluster; either all Intel or all AMD hosts.
Applications on VMs must be well-behaved.
The applications must be written to use the CPUID machine instruction to discover CPU features.
We intercept and mask CPU features to create the appearance of compatibility. But the CPU features are there and functioning if an application tries to use them anyway. So apps that test for the function (i.e run a commands and see if it works) rather than use CPUID to query for the function might break. If they do not follow the established rule (Use the CPUID operation or else!) they could break if they are in a cluster with a lower baseline that the physical hardware hosting then VM and if they are migrated to a host that does not support those instructions/operations.
VI4 - Mod Slide

17. Enabling EVC On A Cluster (ctd)
VI4 - Mod Slide

18. Using EVC Once EVC is enabled for a cluster:
All hosts already in, or entering the cluster, are automatically configured to match the EVC cluster baseline.
VC will not allow hosts to enter the cluster which are not capable of exactly matching the cluster EVC requirements.
VMotion will never fail due to CPU incompatibility since all hosts present identical features through EVC baselines .
VI4 - Mod Slide

19. EVC Maintains Complete VMotion Compatibility
ESX
AMD CPU
Intel EVC Cluster
(Merom Baseline)
ESX
Intel Pentium 4
Cannot add a host with incompatible hardware
All hosts CPUs must be from the same vendor
CPUs must be on par with the cluster baseline or newer
VI4 - Mod Slide

20. EVC Maintains Complete VMotion Compatibility (ctd)
ESX
Intel Core 2
45nm CPU
App OS
Core 2
45nm VM
Intel EVC Cluster
(Core 2 Baseline)
Cannot add a host with running VMs
The VM could be using CPU features that are not present in all hosts in the cluster.
Must migrate or power off VMs on the ESX that you wish to add to the cluster.
The reason for powering off VMs is that we have no way of knowing if the VM is using instructions that break the baseline.
VI4 - Mod Slide

21. EVC Maintains Complete VMotion Compatibility (ctd)
AMD EVC Cluster
(Rev E/F Baseline)
ESX 3.0.x
Opteron Rev F CPU
Cannot add a host with incompatible ESX version
Host must have ESX 3.5 update 2 or newer installed
ESX 3.5 U2 introduced this feature.
VI4 - Mod Slide

22. Troubleshooting EVC VMware CPUID utility
Bootable CDROM
Reports raw CPUID data and “interesting” features
Intel processor identification utility
CPU-Z, a freeware utility for displaying CPU features
VMotion Info tool
Displays CPU info of servers in a vCenter deployment
VI4 - Mod Slide

23. Troubleshooting EVC
Error: Incompatible CPU: “The following hosts have CPUs that do not support EVC. Remove these hosts from the cluster.”
What troubleshooting steps would you take to solve this problem?
You have to unmask the CPU features of all the ESX servers. If any of the bits are masked, you cannot enabled EVC.
It was discovered during the delta of the course that if you had manually disabled these features in the BIOS of the machine itself and trying to enable EVC that there were odd behaviours; The machines although all having the same configuration still was throwing an error.
The solution was to expose all of the features of the CPU in the BIOS and allow Virtual Center to control the masking/exposure of the CPU features. This anomaly could be fixed for the GA release.
VI4 - Mod Slide

24. Lesson 2-4 Summary Learn how to enable EVC on a Cluster
Learn how to create EVC Baselines
Learn how to troubleshoot EVC
VI4 - Mod Slide

25. Lesson Lab 4
Module 2-4 Lab 4 – VMware vCenter Enhanced vMotion Compatibility
Enable EVC on a Cluster
Checking EVC compatibility
EVC settings
Troubleshooting EVC
VI4 - Mod Slide