2Bibliography“Virtual Machine Monitors: Current Technology And Future Trends”, Mendel Rosenblum and Tal Garfinkel, IEEE Computer, May 2005“Xen and the Art of Virtualization”, P. Barham, R. Dragovic, K. Fraser, S. Hand, T. Harris, A Ho, R. Neugebauer, I. Pratt, A. Warfield, SOSP ’03.The Definitive Guide to the Xen Hypervisor, David Chisnall, Prentice Hall, 2008.“Scale and Performance in the Denali Isolation Kernel”, Andrew Whitaker, Marianne Shaw, and Steven D. Gribble, in System Design and Implementation (OSDI), Boston, MA, DecDenali: Lightweight virtual Machines for Distributed and Networked Applications”, Andrew Whitaker, Marianne Shaw, and Steven D. Gribble, Proc. USENIX annual Technical Conference, June 2002.Xen Homepage:VMWare:
3Outline Overview History of Virtual Machines What is a virtual machine?What is a virtual machine monitor (VMM)?System or application (process) virtual machinesHistory of Virtual MachinesBenefits of Virtual MachinesIssues and ImplementationExamples
4What is it? (1)What is virtualization? an abstraction or simulation of hardware resourcese.g., virtual memoryA virtual machine is an isolated environment that appears to be a whole computer, but actually only has access to a portion of the computer’s resources.Similar to, but much more than, the illusion provided by a multitasking operating system.
5What is it? (2)A virtual machine monitor (VMM) is the software layer that supports one or more virtual machinesEach VM appears to run on bare hardware, giving the appearance of multiple instances of the same computer, but all run on a single machine.VMM is also called a hypervisorGuest operating system: an operating system that runs in a VM, supported by the VMM, rather than directly on the hardware.
6System & Process VMs (1) http://en.wikipedia.org/wiki/Virtual_machine System (hardware) virtual machine - See previous slidesProvides a complete systemEach VM can run its own OS, which in turn can run multiple applicationsProcess or application virtual machine; e.g., JVMRuns inside (under the control of) a normal OSProvides a platform-independent host for a single application at a time (each platform needs a different JVM, however)
7System & Process VMs (2) System virtual machine One machine appears to be multiple identical machines, each running its own operating system which in turn runs user jobs which are compiled to run on the underlying hardwareProcess or application virtual machineSource code is compiled into a “machine” code that represents the instruction set of a virtual (not real) machine.The same byte code can be “executed” by any computer that has the appropriate interpreter/virtual machine, independently of the actual underlying hardwareExamples: Java byte code + JVM, Microsoft Common Language Infrastructure + .NET framework
8System VMMs – Three Types Traditional: VMM is a thin software layer that runs directly on the host machine hardwareMain advantage: better performance than hostedVMWare vSphere, ESXi Servers, Xen, OS370, DenaliAlso called a “bare metal” VMMHosted: VMM runs on top of an existing OS.Main advantage: easier to build; easier to installExamples: User-mode LinuxHosted/Hybrid: shares the hardware with existing OSExample: VMWare Workstation
9Computer System Interfaces/Traditional Model Unprivileged machine instructions: available to any programPrivileged instructions: hardware interface for the OS/other privileged softwareSystem calls: interface to the operating system for applications & library functionsAPI: An OS interface through library function calls from applications.
10Two Ways to VirtualizeProcess Virtual Machine: program is compiled to intermediate code,executed by a runtime systemVirtual Machine Monitor: software layer mimics the instruction set; supports an OS and its applications
13Hosted/Hybrid versus Non-hosted VMM Hosted has 3 advantages VMM is no harder to install than any other applicationThe VMM can use the host OS scheduler, pager, etc. and focus primarily on isolation; (hybrid doesn’t use all host features.)I/O support is better: the VMM can use the device drivers that are designed to work with the host OS rather than having to provide its own. (Hybrid may be limited to using host I/O)
14Hosted versus Non-hosted VMM Disadvantage I/O overhead is “greatly increased”: requests go from guest OS to VMM to host OS and down eventually to the device driver.Too inefficient for serversMore difficult to guarantee complete isolation, so not appropriate for servers from a security perspective.
15Hosted v Non-hosted VMM Conclusion:Hosting is a good approach for individual work stations; reduces effort needed to get VMM up and running; performance isn’t a major issue.Hosting is not advisable for servers. Security issues are the most important concern, followed by added overhead for I/O and any other host OS services that are used.
16VM – How They Work (1)VMM runs in kernel mode (replacing tradtional OS)Guest OS runs in user modeSome modern hardware has a third mode for the guest OSFor the most part, applications run normally and execute machine code directly (direct execution)What about system calls or other attempts by user processes to execute privileged instructions?
17VM – How They Work (2)If the guest OS runs in user mode how can it execute privileged code?It can’t. When it tries to execute a privileged instruction, the VMM traps the operation, and executes in place of the guest OSe.g., when a guest OS appears to execute an I/O system call, the VMM is actually in charge of the actual I/O processing.
18Virtualization versus Emulation Virtualization presents multiple copies of the same hardware system.Direct execution of code on the hardwareEmulation presents a model of another hardware systemInstructions are “emulated” in software – much slower than virtualizationExample: Microsoft’s VirtualPC could run on other chipsets than the x86 family; used on Mac hardware until Apple adopted Intel chips
19Full Virtualization versus Paravirtualization Full virtualization: each virtual machine runs on an exact copy of the actual hardware.Paravirtualization: each virtual machine runs on a slightly modified copy of the actual hardwareBecause some aspects of the hardware can’t be virtualized (see examples later)To present a simpler interface; improve performance.
20History - Why VMM’s?Early computers were large (mainframes) and expensiveVMM approach allowed the machine to be safely multiplexed among many different applicationsAn alternative to multiprogramming
21Virtual Machines - History Early example: the IBM 370VM/370 is the virtual machine monitorAs each user logs on, a new “virtual machine” is createdCMS, a single-user, interactive OS was commonly run as the OS on each VMSeparation of powers:Virtual machine/guest OS interacts with user applicationsVirtual machine monitor manages hardware resources – compare to exokernel concept
22History – 1980s & 1990sAs hardware got cheaper and operating systems became better equipped to handle multitasking, the original motivation went away.Hardware platforms gradually eliminated hardware support for virtualization.And then …
23History – late 90sHitachi MPPMassively parallel processors (MPPs) were developed during the 1990s; they were hard to program and did not support existing operating systemsResearchers at Stanford used virtualization to make MPPs look more like traditional machinesOther research groups explored different approaches to VMsResult: today, virtual machines are very common, although the MPPs of the 90s have been mostly replaced by clusters – and in some areas MPP is now used to refer to multicore chips.
24Example Virtual Machine Systems VMware: commercial products, derived from research done at StanfordXen: open source, Cambridge University, widely used in research and academia; xen.orgDenali: University of Washington, focused on support for Internet servicesNever commercialized
25VMware VMware, a publicly held company, founded by Stanford developers Two lines of products:Desktop : a range of products; advertised as a way for corporations to migrate and upgrade operating systems from a centralized IT centerVMware vSphere hypervisor is a “bare-metal hypervisor” that supports server consolidationVmware also virtualizes datacenters, networks and cloud applications (with Vmware vSphere and vCloud suite)
26Xen: http://xenproject.org/ Xen: open-source VM system for x86, Itanium, ARM & othersOriginated at Cambridge University Computer LabNow supported as an open-source product that has destktop, server, and cloud capabilities (Amazon uses it for its cloud services.)Designed to support execution of Linux, other Unix-like systems (Solaris, BSD), Windows OS’s simultaneously on the same platformObjective of original project: efficient hosting of up to 100 virtual machines
27Hyper-V Hyper_V is Microsoft’s server virtualization software: Each virtual machine (user program + guest OS) is encapsulated in a partition supported by the VMMThree execution modes: Ring 0, 1, 2Requires special hardware to support virtualization.
28Denali Research project – U of Washington Time frame ~Problem addressed: hosting Internet services economicallyGoal: to allow new, untrusted, services to be hosted on third-party servers.Protection provided by VM concept lets servers safely host multiple different services.Encapsulation lets services be swapped in and out of memory easily so multiple services can share one machine
29Reasons for Adopting VMMs Flexibility in choice of operating systemEncapsulation: of an operating system, (virtual) computer system, and one or more applications into a single unitIsolation/Security: supported by encapsulation; systems compromised by internal failure or external attack are isolated and their failure doesn’t affect other VMs.
30OS FlexibilitySupport several operating systems at the same time on a single hardware platformAbility to experiment with new operating systems, or modifications of existing systems, while maintaining backward compatibility with existing systems.Hardware can change faster than software – now you can run an existing application and the OS that supports it on a new computer, thanks to the VMM layer.
31Encapsulation Conventionally, servers ran on dedicated machines. Protects against another server/application crashing the OSBut … wasteful of hardware resourcesEncapsulation means that the complete state of a given VM can be saved to one or a few files – similar to checkpointing an application.Furthermore, the state of one VM is totally separate from the state of any other VM. This is enforced by the VMM’s resource allocation policies.
32IsolationVirtual machines are as separate from each other (isolated) as if they actually were separate computers.Applications in a VM are protected from faults in other VMs, in part because of encapsulation, and because the VMM controls resource allocation and usage by the guest OS’sViruses, buggy applications, other problems that cause crashes or corrupt the OS they run on will not affect other VMs
33Virtualization in Distributed Systems Rosenblum and Garfinkel  point out that encapsulation supports the portability of virtual machines, which in turn means it is easy and safe to move (or replicate) serversThis supports load balancing and maintenanceOr, multiple services can safely share a single computer thanks to encapsulation & isolation.Since many services aren’t frequently used this can a great cost saver.
34Desirable Qualities A good VMM Doesn’t require applications to be modifiedDoesn’t severely affect performanceIs not complex/error prone
35Implementation Issues Virtualize CPUGuest OS runs as if it is executing directly on the hardware CPU, but it isn’tVirtualize memoryGuest OS thinks it is managing memory directly, but it isn’tParavirtualization versus binary translationHardware-assisted virtualization
36CPU Virtualization Basic technique: direct execution As long as it is executing unprivileged instructions the virtual machine (guest OS + applications) executes hardware instructions directly. Note that in emulation direct execution isn’t possible since applications & the OS think they are running on a different ISA.If the guest OS tries to execute a privileged instruction the CPU traps to the VMM which executes the privileged operation.VMM runs in privileged (kernel) mode, guest OS runs in user mode.
37Example: Disable Interrupts  If a guest OS tries to disable interrupts, the instruction is trapped by the VMM which makes a note that interrupts are disabled for that virtual machine only.If interrupts arrive for the VM that disabled them, they are buffered at the VMM layer until the guest OS enables interrupts.Other interrupts are directed to VMs that have not disabled them.
38Direct Execution Not Always Possible Modern CPUs, esp. x86 architectures, were not designed for virtualization.Example: POPF (pop CPU flags from stack)If executed in user mode, no trap – it’s just ignored by the hardwareIn this case, direct execution fails – Guest OS assumes flags have been popped, but they haven’t been because the VMM isn’t notified.
39Two Ways to Handle Non-virtualizable Instructions ParavitualizationXen, DenaliBinary TranslationVMwareBoth use the same basic approach: catch non-virtualizable instructions and emulate them in software at the VMM level.Difference: when they are detected
40ParavirtualizationRewrite portions of the guest OS to replace non-virtualizable instructions with a trap to the VMM, which executes or emulates the instruction on behalf of the guest OSe.g., remove POPFs; substitute a call to the VMMParavirtualization affects the guest OS, but not applications that run on it – the API is unchangedParavirtualization is also used sometimes to replace inefficient operations with more efficient ones.
41Dynamic Binary Translation Dynamic binary translation looks at a short sequence of (binary) source code, translates it, and caches the resulting sequence. [ ]Similar to JIT compilers.During this process VMware’s DBT replaces non-virtualizable instructions with equivalent code that can be virtualized.Compare to static binary translation, done by a compiler, which translates to binary at compile time.
42ComparisonParavirtualization changes the source code of a guest OS; dynamic binary translation generates modified binary code only if needed.Paravirtualization is more efficient, but requires modification to the guest OSParavirtualization also allows more efficient interfaces, in some casesBinary translation is backward-compatible but has some extra overhead of run-time translation the first time an instruction is encountered.
43Hardware-assisted Virtualization AMD-V and Intel VT are architecture extensions to support virtualization on AMD and Intel hardware.New execution modesAllows guest OS to run in a different “ring” than user programs, and VMM in yet a higher privileged modeFlags show which mode the CPU is currently running inEssentially, the trap and emulate mode used in paravirtualization or binary translation is now done in hardware.Does away with need to modify guest OS; is faster than binary translation.
44Memory Virtualization VMM maintains a shadow page table for each virtual machine.When the guest OS makes an entry in its own page table, the VMM makes the same entry in the shadow table.Shadow page table points to actual page frameThe hardware MMU uses the shadow page table when it translates virtual addresses.
45Challenges Let the guest OS decide which of its pages to swap out VMware’s ESX Server used the concept of a balloon process, running inside the guest OS .When the VMM wants to swap out pages from a given VM it notifies the balloon process to allocate more memory to itself.The guest OS must “page out” unused portions of other processes to its virtual disk.The VMM now knows which pages the guest OS thinks it can do without.
46Other Virtual Memory Challenges To share or not to share pages across VM boundaries:VMware tracks duplicate pages in different virtual machines & stores only one copy of the actual page with pointers from the shadow page tables in sharing processes.Copy-on-write policyXen focuses on total isolation of each virtual machine, which means no sharing
47Migrating Virtual Machines A virtual machine encapsulates an entire computing environment.If properly implemented, the VM provides strong mobility since local resources may be part of the migrated environment“Freeze” an environment (temporarily stop executing processes) & move entire state to another machinee.g. In a server cluster, migrated environments support maintenance activities such as replacing a machine.
48Migration of Virtual Machines Example: real-time (“live”) migration of a virtualized operating system with all its running services among machines in a server cluster on a local area network.Presented in the paper “Live Migration of Virtual Machines”, Christopher Clark, et. al.Problems:Migrating the memory image (page tables, in-memory pages, etc.)Migrating bindings to local resources
49Memory Migration in Virtual Machines Three possible approachesPre-copy: push memory pages to the new machine and resend the ones that are later modified during the migration process.Stop-and-copy: pause the current virtual machine; migrate memory, and start the new virtual machine.Let the new virtual machine pull in new pages as needed, using demand pagingClark et.al use a combination of pre-copy and stop-and-copy; claim downtimes of 200ms or less.
50Looking Ahead …How useful will virtual machine technology be for multicore processors and cloud computing???
51Summary & Review (1) A virtual machine is a copy of a real machine Applications don’t know if they are running on real or virtual hardware, other than having fewer resources.A virtual machine is isolated: if several VMs execute on the same hardware they do not interact with each other directly or indirectly.The performance of a virtual machine should be about the same as that of the actual hardware.So most instructions should be directly executed by the hardware as opposed to being emulated.
52Summary and Review (2)Process virtual machines (JVM) virtualize at a higher level, do not necessarily even correspond to real machines.System virtual machines virtualize at the level of the hardware-software interfaceVariations of classic system virtual machine:Hosted (run on another operating systemEmulation (provides virtual hardware and OS, as in Virtual PC) – not really a virtual machine
53Summary & Review (3)Virtual Machine Monitor (hypervisor) runs on a bare machine, implements one or more virtual machines.The VMM allocates resources and controls resource sharing among all VMsOperation:Each VM runs a guest OSVMM runs in kernel modeGuest OS and applications run in user modePrivileged instructions trap to the VMMHypercalls (the VMM equivalent of system calls) may be used by a guest OS to request service from the VMM
54Summary & Review (4)Benefits of VM technology for non-hosted (traditional, or native) VMsIsolation and securityMultiple servers on a single machineEncapsulation of an entire environment: OS and application for the purpose ofMigrationCheckpointingSupporting system maintenanceRunning several OS’s concurrentlyOlder versions, experimental systems, Linux & Windows, …For hosted VMs, the major advantage is the ability to run two or more OS’s at once
55Reading for Next Class Chapter 4 – Communication 9/25: First test Covers: Everything through virtual machines.
57Xen – IntroClaim: virtualization is better than multi-tasking as a way to share hardware.CPU requests, memory demand, disk accesses, other resource needs of one process impact the performance of other processesXen solution: multiplex resources at the OS level instead of the process level.
58Xen implementation of VMM Domain 0 guest has privileged access to the Xen hypervisor and can be used by the system administrator to manage the system.Separation of powersXen only has to worry about multiplexing hardware to multiple guestsDomain 0GuestApplicationDomain UGuest OS2ApplicationDomain UGuest OS3VM1VM2VM3XenHardware layerXen implementation of VMM
59Xen Design PrinciplesVirtualize all architecture features that are required by standard binary interfaces.To support existing applications without modificationSupport multi-application guest operating systemsUse paravirtualization to get improved performance and resource isolation
60Xen HVM (Hardware Virtual Machine) Some versions of Xen are designed to run on Intel VT and AMD-V chips with special virtualizing hardware.Able to run un-modified (no para-virtualization) operating systems. This implementation is known as a hardware virtual machine.Windows requires an HVM environment; Linux, Solaris, and BSD systems don’t.
61Xen Memory ManagementUnlike VMWare and Denali, Xen expects the guest OS’s to manage their own hardware page tables.To support this, each VM receives a fixed allocation of page frames which it can use as it wishes.New page tables must be registered with Xen and updates must be validated by Xen.Make the page table write protected.
62Xen CPU ManagementXen is designed for the X86 architecture which supports 4 rings, or privilege levels.Traditional OS’s execute in ring 0 (most privileged) and applications in ring 3 (least)Xen executes in ring 0 (only level that can execute privileged instructions)Guest OS runs in ring 1, which isolates it from applications.Note: since this paper was written there have been some modifications to X86 to better support virtualization.
63Xen CPU ManagementPrivileged instructions must be validated (is it OK?) and executed by XenExceptions (page faults, system calls, other traps to OS) are handled as much as possible by the guest OS.Exception handlers are registered & validated with XenSystem calls stop at the guest OS; Xen is involved only if the OS executes a privileged instruction.
64Denali Isolation Kernel Authors define Denali as a small-kernel operating system with similarities to microkernels and exokernelsOnce thought to be inefficient, modern hardware has improved performance of this kernel architectureThey expected Denali to support multiple (up to 10,000) untrusted applications that are virtually independent.
65Isolation Kernel Design Principles Expose low-level resources rather than high-level abstractions for greater securityAvoid “layer-below” attacksPrevent direct sharing by exposing only private, virtualized namespacesKeeps one VM from “… even naming the resources of another VM, let alone modifying them”. 
66Isolation Kernel Design Principles Design for scalabilityBe able to support a work load that has a few popular services and many that are accessed infrequently.Modify the virtualized architecture for simplicity, scale and performance.Paravirtualization for reasons other than necessity.They do not believe isolation depends on providing an exact copy of hardware so they provide a hardware version that is modified to be more efficient and secure.
67Zipf’s LawGiven a table that ranks something on the basis of its frequency of occurrence, Zipf’s law states that the most frequent item occurs about twice as often as the next most frequent item, which in turn occurs twice as often as the next item, and so on.Zipf made this observation about words in a natural language. Here, we’re talking about accesses to various web services.
68Statistically Multiplexing Services Studies showed that the popularity of most network services (server requests, document searches, etc) followed a Zipfian distribution.Implications:Most requests go to a small number of servicesMost services aren’t popular, but the total number of requests for unpopular services is non-trivialWith isolation it can be safe and efficient to run hundreds or even thousands of services concurrently on a single platform.
69Proof-of-concept Denali is the virtualized architecture Yakima: a VMM which was designed to run in ring 0 on x86 hardware.Ilwaco: a simple prototype guest OS which provides a full set of abstractions to its applications while hiding the Denali architectureReasonable performance in tests1.4 μsec to 9 μsec context switch time, depending on number of VMsEnd-to-end run times of network apps were “comparable” to those of a traditional operating system.
70Conclusion The Denali research project terminated in the mid-2000’s. The Denali research group was right in supposing that virtual machine technology would be most useful today to enable efficient use of server hardware.