Virtualization Performance H. Reza Taheri Senior Staff Eng. VMware

2 Copyright © 2006 VMware, Inc. All rights reserved. Three Key Properties of Virtualization Partitioning Run multiple operating systems on one physical machine Fully utilize server resources Support high availability by clustering virtual machines Encapsulation Encapsulate the entire state of the virtual machine in hardware-independent files Save the virtual machine state as a snapshot in time Re-use or transfer whole virtual machines with a simple file copy Isolation Isolate faults and security at the virtual-machine level Dynamically control CPU, memory, disk and network resources per virtual machine Guarantee service levels

3 Copyright © 2006 VMware, Inc. All rights reserved. Hosted vs. Native Virtualization Native/Hypervisor Hosted Device support is inherited from host operating system for maximum hardware compatibility Virtualization installs like an application rather than like an operating system Can run alongside conventional applications Maximum performance with lowest overhead using certified hardware Highly efficient direct I/O pass-through architecture for network and disk Highly secure micro-kernel virtualization layer— only 100Ks of lines of code versus 10–25 million lines of host operating system code Advanced features like VMotion available (ESX Server) (Workstation, VMware Player, VMware Server, ACE, VMware Server)

4 Copyright © 2006 VMware, Inc. All rights reserved. ESX Server Architecture Hardware VM Device Drivers Storage StackNetwork Stack VM VMM VMkernel POSIX API VMX Peripheral Device Drivers Management Agents and Interfaces Service Console VMX VM VMM Resource Management

5 Copyright © 2006 VMware, Inc. All rights reserved. Virtual Machines in ESX Server (Up to 2 CD-ROMs) 1-4 ports 1-4 adapters 1-4 adapters; 1-15 devices each Up to 16GB RAM 1-2 drives VM Chipset 1 CPU (4 CPUs with VMware SMP)

6 Copyright © 2006 VMware, Inc. All rights reserved. Hot Migration: VMotion ® Technology VMotion Technology moves running virtual machines from one host to another while maintaining continuous service availability - Enables Continuous Workload Consolidation - Enables Zero-Downtime Maintenance

7 Copyright © 2006 VMware, Inc. All rights reserved. Resource Pools Aggregate collections of disparate hardware resources into unified logical resource pools Customer Impact Failed server mean less resources not a failed application Dedicated (virtual) infrastructure for each business unit; central IT retains control over hardware Delegation of resource and virtual machine management down to the business unit Management of an entire SOA application stack as a single entity Servers, Storage, Networking Business Unit Department ADepartment B Aggregate Resources Resource Pool 2 CPU 36GHz, Mem 58GB Priority HIGH Resource Pool 3 CPU 12GHz, Mem 22GB Priority LOW CPU 48 GHz, Mem 80GB

8 Copyright © 2006 VMware, Inc. All rights reserved. VMware HA: Restart VMs if ESX Server fails Virtual Machine X VC

9 Copyright © 2006 VMware, Inc. All rights reserved. Workload Characteristics Consolidation targets are often <30% Utilized Windows average utilization: 5-8% Linux/Unix average: 10-35%

10 Copyright © 2006 VMware, Inc. All rights reserved. Exchange performance in a virtualized environment Many workloads on VM run at/near native H/W speed Let’s focus on something that does not! Study with: Exchange 2003 WS 2K3 EE Loadsim way Dell servers Native supports 1.4X as many users Alternative observation: VM has 1.6X the CPU usage for the same # of users Where does the time go?

11 Copyright © 2006 VMware, Inc. All rights reserved. 2-socket, Woodcrest system Exchange 2003 WS 2K3 EE Loadsim 2003 Perfmon and Vtune on native VMware s/w and h/w monitoring tools on VM

12 Copyright © 2006 VMware, Inc. All rights reserved. Change in h/w profile Native CPU usage is ~60% VM CPU usage is ~90% at ~4% less throughput 73% in the guest+monitor ~20 in hypervisor VM has 1.5X the CPU usage at 0.96% the throughput 1.38X more instructions; 1.09X the CPI The product matches the 1.5X 2-3X increase in TLB misses 40-80% more L1 misses L2: 34% more data misses 90% more imisses, but much fewer than dmisses