CSC 660: Advanced Operating SystemsSlide #1 CSC 660: Advanced OS Virtual Machines

CSC 660: Advanced Operating SystemsSlide #2 Topics 1.What is a VM? 2.Process vs System VMs 3.Virtualizing the Processor 4.Virtualizing Memory 5.Virtualizing I/O 6.VM Performance Issues 7.Intel VT-x Technology 8.Paravirtualization

CSC 660: Advanced Operating SystemsSlide #3 What is a VM? A virtualized system that –Provides a consistent ABI to guest programs. –Runs on a host system (software + hardware.) –Controls resources available to guest programs. –May provide different resources than hardware Different Type (ex: JVML in Java VM) Different Quantity (ex: more/fewer CPUs, disks, etc.) –May be of two major types Process: provides VM to a single process. System: emulates an entire machine w/ guest OS.

CSC 660: Advanced Operating SystemsSlide #4 System Models Non-virtual Machine Virtual Machine

CSC 660: Advanced Operating SystemsSlide #5 Why use Virtual Machines? Portability Run software on a different OS. Run software on a different CPU. Aggregation Modern machines are fast and underused. Put multiple servers in VMs on one real machine. Development Complex software environments. Processor testing and simulation. Debugging Can analyze every aspect of hardware behavior. Security VMs provide greater isolation of software than regular OS.

CSC 660: Advanced Operating SystemsSlide #6 Types of VMs

CSC 660: Advanced Operating SystemsSlide #7 Process VMs Multitasking –Each process in a multitasking OS. –VM = System call interface + ISA + VirtMem Emulators –Allow a process to run on a different OS/ISA. –Types: Interpreter Dynamic binary translator High Level Language VMs –ex: Pascal, JVM, CLR

CSC 660: Advanced Operating SystemsSlide #8 HLL VMs HLL Program Intermediate Code Object Code Memory Image HLL Program Host Instructions Virtual Memory Image Byte Code Compiler Front End Compiler Back End Compiler Loader VM Loader VM Dist

CSC 660: Advanced Operating SystemsSlide #9 System VMs Virtual Machine Monitor (VMM) –Provides illusion of multiple isolated machines. –Manages allocation of and access to hardware resources for multiple guest OSes. –Layer between hardware and guest OS. VMM tasks –State management –Resource control

CSC 660: Advanced Operating SystemsSlide #10 System VMs Applications Guest Apps OS Hardware Guest OS VMM Host OS VMM Guest OS Guest Apps a. Traditional OS b. Native VMMc. User-mode Hosted VMM

CSC 660: Advanced Operating SystemsSlide #11 VMM Modes Requirement: guest OS may not change hardware resources to impact other VMs or the VMM. Guest OS runs in user mode. VMM runs in supervisor mode. –Tracks virtual mode of VM. –User programs run in virtual user mode. –OS runs in virtual supervisor mode. Exceptions & interrupts invoke VMM. –VMM can handle directly –or produce a virtual exception for guest OS.

CSC 660: Advanced Operating SystemsSlide #12 System VM Execution 1.Timer Interrupt in running VM. 2.Context switch to VMM. 3.VMM saves state of running VM. 4.VMM determines next VM to execute. 5.VMM sets timer interrupt. 6.VMM restores state of next VM. 7.VMM sets PC to timer interrupt handler of next VM. 8.Next VM active.

CSC 660: Advanced Operating SystemsSlide #13 IBM VM/370 Mainframe VMM OS. –First VM environment in System/ –Control program was a native VMM. –Each user had VM running single-user CMS. –Principles still used in z/VM on IBM zSeries.

CSC 660: Advanced Operating SystemsSlide #14 VMWare x86-based VM. VMWare Workstation –Hosted VMM runs on Linux or Windows. –Any x86 OS can be used as guest OS. VMWare ESX Server –Native VMM runs directly on x86 hardware. –VMotion allows VM migration.

CSC 660: Advanced Operating SystemsSlide #15 Virtualizing the Processor Emulation –VMM examines each guest instruction and emulates on virtualized resources the effects that instruction would have had on real resources. –Only method when host/guest ISA differs. –May be necessary with identical host/guest ISA. Direct Native Execution –Host ISA must be identical to guest ISA. –Most instructions executed directly on CPU. –Some sensitive instructions must be emulated.

CSC 660: Advanced Operating SystemsSlide #16 Privileged Instructions Instrs that trap if executed in user mode. –I/O –Memory Management Can only be executed in supervisor mode.

CSC 660: Advanced Operating SystemsSlide #17 Handling Privileged Instructions Dispatcher Allocator Instructions to change machine resources. Interpreter Routine #1 Interpreter Routine #N Instruction Trap

CSC 660: Advanced Operating SystemsSlide #18 Handling Privileged Instructions 1.Instruction Trap invokes VMM Dispatcher. 2.Dispatcher calls Instruction Routine. 3.Changes mode to supervisor. 4.Emulates instruction. 5.Computes return target. 6.Restores mode to user. 7.Jumps to target.

CSC 660: Advanced Operating SystemsSlide #19 Sensitive Instructions Some instructions cannot be executed directly on the CPU because they would interfere with the VMM or other VMs. Control-sensitive Instructions Attempt to modify resource configuration. Behavior-sensitive Instructions Behavior depends on resource configuration, including user/kernel mode setting.

CSC 660: Advanced Operating SystemsSlide #20 Non-Privileged Sensitive Instructions Example: POPF Pops flag registers from stack. Includes interrupt-enable flag. User mode, POPF modifies all but interrupt flag. Kernel mode, POPF modifies all flags.

CSC 660: Advanced Operating SystemsSlide #21 Efficient VMs If all sensitive instructions are privileged, the processor is considered to be virtualizable. –All sensitive instructions produce a trap. If non-privileged sensitive instructions exist, then the VMM must examine all instructions before execution to trap these instructions. –Emulation –Dynamic Binary Translation (Patching)

CSC 660: Advanced Operating SystemsSlide #22 Memory Virtualization Virtual Memory in a VM –Each guest OS maintains own set of page tables. –Guest OS translates virtual memory locations to real memory locations (“physical memory” of VM.) –Guest OS has swap space on virtual disk. VMM –Translates real memory to physical memory using MMU. –Doubles number of page faults. –VMM has swap space on physical disk.

CSC 660: Advanced Operating SystemsSlide #23 Shadow Page Tables VMM maintains shadow page tables –Map virtual->physical instead of real->physical. –VMM validates guest page table updates. –Replicates guest changes in shadow page table. Virtualize page table pointer register. –VMM manages real page table pointer. –Updates page table ptr when switching VMs.

CSC 660: Advanced Operating SystemsSlide #24 Shadow Page Tables MMU Guest OS Hardware Accessed & dirty bits VMM guest writes guest reads Guest Page Table Shadow Page Table Updates

CSC 660: Advanced Operating SystemsSlide #25 I/O Virtualization VMM must intercept all guest I/O ops. –PC: privileged IN and OUT instructions. –I/O operation may consist of many INs/OUTs. Problem: huge array of diverse hardware –Native VMM needs driver for each device. –Hosted VMM uses host drivers w/ perf penalty.

CSC 660: Advanced Operating SystemsSlide #26 Virtualizing Devices Dedicated Devices –VM has sole control of device. Partitioned Devices –VM has dedicated slice of device, treats as full. –VMM translates virtual full dev parameters to parameters for underlying physical device. Shared Devices –VMM can multiplex devices. –Each VM may have own virtual device state. Nonexistent Devices –Virtual software devices with no physical counterpart.

CSC 660: Advanced Operating SystemsSlide #27 Virtualizing a Network Card

CSC 660: Advanced Operating SystemsSlide #28 VM Performance Why is VM slower than physical hardware? Emulation: Sensitive instructions must be emulated. Interrupt Handling: VMM must handle interrupts, even if eventually passed to guest. Context Switches: VMM must save VM state when controlled transferred to VMM. Bookkeeping: VMM has to do work to simulate behavior of real machine, such as keeping track of time for VMs. Memory: Memory accesses may require access to both shadow and local page tables.

CSC 660: Advanced Operating SystemsSlide #29 VT-x Technology New CPU modes: VMX root/non-root –VMM runs in VMX root. –Guest VMs run in VMX non-root. –Each mode has rings Virtual Machine Control Structure (VMCS) –Guest Area, Host Area. Transitions –VM Entry: root to non-root transition. Load processor state from VMCS guest area. –VM Exit: non-root to root transition. Save state to VMCS guest area, load state from host area.

CSC 660: Advanced Operating SystemsSlide #30 VT-x Technology Instructions –Some sensitive instructions operate on non-root VMX state; others produce a VM exit. –VMM controls which instructions VM exit. Interrupts –External interrupts cause VM exits. –VMM controls which exceptions VM exit.

CSC 660: Advanced Operating SystemsSlide #31 Paravirtualization: Xen Provide VM abstraction similar to hardware. –Modifies guest OS to use Xen/x86 architecture. Memory –Guest has read access to hardware page tables. –Updates batched and validated by Xen VMM. CPU –Guest OS installs direct system call handler. –Sensitive instructions replaced with Xen calls. I/O –Event mechanism replaces hardware interrupts.

CSC 660: Advanced Operating SystemsSlide #32 Xen 1.2 Architecture

CSC 660: Advanced Operating SystemsSlide #33 Xen VMM VMM resides in top 64MB. Protected by segmentation, not page tbl for perf. ring 3 Kernel User 4GB 3GB 0GB Xen S S U ring 1 ring 0

CSC 660: Advanced Operating SystemsSlide #34 Xen System Performance LXVU SPEC INT2000 (score) LXVU Linux build time (s) LXVU OSDB-OLTP (tup/s) LXVU SPEC WEB99 (score) Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

CSC 660: Advanced Operating SystemsSlide #35 References 1.Paul Barham et. al., “Xen and the Art of Virtualization,” 19 th ACM Symposium on Operating Systems Principles, Oct Ian Pratt, “Xen 3.0 and the Art of Virtualization,” Ottawa Linux Symposium John Scott Robin and Cynthia E. Irvine, “Analysis of the Intel Pentium’s Ability to Support a Secure Virtual Machine Monitor,” Proceedings of the 9 th USENIX Security Symposium, Aug Mendel Rosenblum and Tal Garfinkel, “Virtual Machine Monitors: Current Technology and Future Trends,” IEEE Computer, May James E. Smith and Ravi Nair, Virtual Machines, Elsevier, Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, Operating System Concepts, 6 th edition, Wiley, Jeremy Sugerman, et. al., “Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor,” Proceedings of the 2001 USENIX Annual Technical Conference, Rich Uhlig et. al., “Intel Virtualization Technology,” IEEE Computer, May 2005.