1. © 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice In search of a virtual yardstick: Measuring Virtual CPU time Wade Satterfield 2/26/2009

2. Percent busy vs. Percent of capacity used It used to be that the percent of time a CPU was busy was also the percent of the capacity of the CPU’s capacity that was used. This is not true of virtual machines. 21 June 2015

3. Errors in measuring CPU time on a VM The Guest OS counts time it didn’t get The Guest OS doesn’t count time it did get 31 June 2015

4. Counted Time the VM didn’t actually get 41 June 2015 Possible time the Guest OS thinks it used Guest OS finds useful work to do Hypervisor takes CPU from the VM Hypervisor returns CPU to the VM Guest OS becomes idle Time the Guest OS actually used

5. First Experiment Run a tight loop program in a physical server and again on a VM on an otherwise idle VM host 51 June 2015

6. The benchmark #!/bin/sh a=10000000000 while [$a –gt 0] do let a=$a-1 done 61 June 2015

7. Second Experiment Run a tight loop program on a VM on a busy VM host (5 VMs running the same thing on a 4-way host) 71 June 2015

8. Further Experiments Run a tight loop program on a VM on a busy VM host (8 VMs running the same thing on a 4-way host) 81 June 2015

9. What does this mean for %CPU busy? Suppose you have a Physical workload that is normally 20% busy and you make it into a VM on a VM host that is 40% busy. (Assume no IO or other virtualization overhead) How busy will the VM think it is? 20%, 20.5%, 28%, 40% 91 June 2015

10. A constant workload might appear to vary 101 June 2015

11. CPU data that does not get counted 111 June 2015 Physical hardware Host OS (hypervisor) Virtual hardware Guest OS app Virtual hardware Guest OS app

12. Threads within a virtual machine 121 June 2015 Physical hardware Host OS (hypervisor) Virtual hardware Guest OS app CPU thread IO threads The guest OS knows about the threads that represent the virtual CPUs, but it does not know that there are threads representing the NICs, and disks.

13. The benchmark #!/bin/sh a=1000 while [$a –gt 0] do ftp –f getbigfile done 131 June 2015

14. Experiment Run an IO intensive program on a VM on an otherwise idle VM host 141 June 2015

15. How bad can this be? ftp is an extreme example High IO applications like a web server can have throughput reductions to 70% In our testing we have found that the VM can get near 100% utilization, but the IO threads can get time too giving a VM 125% utilization. 151 June 2015

16. What you want to measure Generally you want to measure the CPU time used on your VMs by measuring them at the VM host −Your CPU bottleneck is on the VM host Some people may want to know how much CPU time would be used if the VM were on a physical system 161 June 2015

17. Global memory, disk, network Queues, and process data Vmware 171 June 2015 Hypervisor Guest OS Data Collector “Global” CPU, disk, network Vmware has a WBEM interface on the Vmhost, and on the Virtual Center host

18. All data HP-VM 181 June 2015 Hypervisor Guest OS Data Collector “Global” CPU Integrity VMs have a library call available inside the VM to get the CPU time the VM has used on the VM host.

19. HyperV HyperV does most of its IO processing the parent partition The CPU time used by the VM’s worker process in the parent partition should be added to the time used by the VM HyperV has a WMI interface for a guest’s CPU time 191 June 2015 Global memory, disk, network Queues, and process data Hypervisor Guest OS Data Collector “Global” CPU, DOM0 CPU disk, network Parent partition Worker process CPU data http://blogs.msdn.com/virtual_pc_guy/archive/2008/02/28/hyper-v-virtual-machine-cpu-usage-and-task-manager.aspx

20. Xen Xen does most of its IO processing in DOM0 The CPU time used by DOM0 should be prorated out to the VMs depending on how much CPU time they used. 201 June 2015 Global memory, disk, network Queues, and process data Hypervisor Guest OS Data Collector “Global” CPU, DOM0 CPU disk, network DOM0 CPU data for IO