Download presentation
1
Virtualization Technology
Tal Maoz 07/05/2009
2
Outline Background Hardware Virtualization Virtual Machines
The XEN Virtual Machine Virtual Machine Migration
3
Virtual: Webster dictionary defines “virtual” as:
being such in essence or effect though not formally recognized or admitted of, relating to, or using virtual memory of, relating to, or being a hypothetical particle whose existence is inferred from indirect evidence
4
Background 1 - IBM 1963 – IBM builds M44/44X to research time sharing technologies at MIT. Each VM (44X) is an image of the main machine (M44) 1964 – IBM announces System 360. Each VM is a copy of the underlying hardware, running it’s own OS. A hardware component called VMM (Virtual Machine Monitor) managed the VM’s directly on the real hardware.
5
Background 1 - IBM
6
Background 2 – Software At the end of the 1960’s, the VMM took the form of a software abstraction layer Each VM was similar enough to the underlying hardware to run unmodified software Ever growing mainframes gave users the ability to multiplex applications but the cost grew as well For a while, it was good
7
Background 3 - Downhill The 1980’s and 1990’s brought modern multitasking operating systems and strong, faster minicomputers and PC’s Large mainframe VMM’s faded away and gave way to small cheap computers By the late 1980’s neither academics nor industry viewed VMM’s as much more than a historical curiosity In the 1990’s, Stanford researchers looked at VM’s as a way to create cheaper MPPs – Massively Parallel Processing machines
8
Background 4 - Revival Modern stronger machines gave the ability to run more than a few applications at a time VM’s harness the full power of modern CPU’s During the 1990’s VM’s were built for almost every architecture – Emulators were born The biggest step was the emulation or virtualization of the PC’s themselves – VMware Inc. started supplying VMMs for commodity platforms The VMM got a new name: Hypervisor VMM – Running over an existing OS Hypervisor – Running on the bare-metal
10
XEN running 3 XenoLinux VM’s and a NetBSD machine
11
VMWare running 3 virtual machines
12
Background – What Now? Sun’s Sparc and IBM’s G4 have long had support for virtualization technology As the market for VMM’s grew, hardware manufacturers (Intel, AMD …) decided to support VMM’s at the hardware level By late 2005, Intel announced VT (Virtual Technology), codenamed Vanderpool (VT-x & VT-I / VMX) AMD announces the superior Pacifica (AMD-V / SVM), incompatible with Intel’s VT, spelling trouble for developers… HP currently offers it’s HP_UX partitioning technology on Itanium servers Sun introduces the xVM technology based on Xen for x86 and on “Logical Domains” for Sparc
13
Hardware Virtualization
Started with IBM’s System 360 followed by the famous VM/370 – Full Hardware Virtualization Over the years, other architectures such as Alpha, MIPS, PA-RISC and SPARC introduced virtualization friendly features such as software managed TLB’s (Translation Lookaside Buffer) The IA-32 and x86 are less friendly but give 4 privilege “rings”, of which only 2 are in use by current OS’s. Virtualization on x86 can be made efficient by reducing the OS’s privilege mode from ring 0 to ring 1 and running a software VMM in ring 0
14
Intel’s VT (Vanderpool)
First, there were 4 privilege rings, 0 through 3 On the move from 32 bit to 64 bit, Intel decided to drop the extra privilege rings and only keep the 2 in use To solve the problem one has to change the host OS so it only uses ring 3, same as user-level programs. VT introduces a new mode: VMX. Hypervisor runs in VMX root mode, OS and programs run in VMX mode 3 … VMM VMX Mode 1 2 3 3 Normal Mode
15
Intel’s Vanderpool – Contd.
Special commands facilitate the creation of new OS’s (VMPTLRD, VMCLEAR) and switching of OS’s (VMLAUNCH, VMRESUME) A new VMCS (Virtual Machine Control Structure) stores info on the running VMs, including event flags that tell the CPU when to return control to the hypervisor (like a PCB for VMs)
16
AMD’s Pacifica Pacifica does everything VT does but adds 2 key features: MMU modes & a DEV AMD’s MMU (Memory Management Unit) has been moved into the CPU. The MMU manages the Virtual Memory at the hardware level AMD moved much of the MMU’s job from hardware to software when running VM’s. The MMU now gives the VMM the ability to trap VM’s memory calls via the CR3 register, and manage an SPT (Shadow Page Table) for each VM. This provides each guest OS with it’s own little world. Effective but costly! AMD introduces the NPT (Nested Page Tables). The CR3 register is now fully virtualized for each VM by the MMU and each machine’s SPT is controlled by the MMU at the hardware level, under the hypervisor’s supervision. Hardware works much faster!
17
AMD’s Pacifica – Contd. Paged Real Mode – an MMU mode for running guest OS’s that use Real Mode (no virtual memory) TTLB (Tagged Transition Lookaside Buffer) – ID’s are added to TLB addresses so the MMU knows to which entries belong to each VM and can save costly hypervisor flushing work. In effect you have several multiplexed TLB’s, one per VM.
18
AMD’s Pacifica – Contd. DEV (Device Exclusion Vector) is a safety mechanism used when several guest OS’s want to use DMA (Direct Memory Access) DMA allows devices to write directly to the memory with no CPU intervention, This can cause many problems when several VM’s want to use the same device – the device can end up writing to the wrong OS’s page table! The DEV tells each device if it is allowed to access a page of memory or not, thus it can dynamically bind devices to VM’s DEV also helps prevent VM collisions when using HyperTransport (Device-CPU bus) and Cache Coherent HyperTransport (CPU-CPU bus) A new instruction called SKINIT, which is intended for running secure VM’s that are totally invisible to other VM’s
19
Intel – The Next Generation
The Core-i7 microarchitecture moved the MMU from the chipset into the CPU Next, Intel plans to insert a GPU into the same die Hyper-Threading is back… QuickPath replaces the FSB (Front Side Bus) Support for non-ECC DDR3 only! (up to tri-channel)
20
Trends Microsoft included the WoW technology in it’s Windows OS’s
Apple offers integrated virtualization technology to lure Windows users over to OSX (Parallels Desktop) VMware offers “Fusion” for Mac and “Unity” for PC’s – Run graphic applications from various OS’s simultaneously and transparently! Windows 7 will include transparent XP virtualization on hardware-virtualization enabled CPUs Cloud computing farms (Amazon, Yahoo, …) offer virtual clusters on-demand
21
Virtual Machines – Why? Home users who want several OS’s with ease (Windows, Linux, OSX, etc.) Developers who need to develop for several OS’s and want a dedicated development machine Security – one can run a non-secure machine for browsing and games, a more secure machine for work and a very secure machine for sensitive mail and information
22
Virtual Machines – Why? Server farms can use virtual machines to make administration easier Each client gets a dedicated VM where his application can run with it’s own configuration and environment with no compatibility issues VM’s can be migrated from server to server and thus applications can continue running uninterrupted while servers go down for maintenance VM’s allow for much more powerful and efficient Grid economy algorithms and management systems
23
Virtual Machines - Brands
Hardware OS Virtualization: Bochs FAUmachine Microsoft Virtual PC Open CoLinux Parallels QEMU SVISTA TRANGO PL’s twoOStwo UML (User-Mode Linux) Virtuozzo VM from IBM (S360 OS) Denali VMWare XEN MOSIX Virtual OS
24
Open Source Projects Xen VirtualBox QEMU KVM Vserver OpenVZ UML …
25
Virtualization Approaches
Full Virtualization – All the system aspects are virtualized, often while the guest OS’s code is being dynamically rewritten to allow for the VMM’s control (e.g. older VMware ESX Server) – works nicely with x86, allows hosting of unmodified systems but not very efficient and requires shadowing of various system structures Paravirtualization – The hypervisor runs in ring 0 of an x86 architecture while the guest OS is modified to run in ring 1 and use hypercalls so that interrupts are processed through the hypervisor – works with x86, very efficient but requires modified guest OS’s
26
The Xen Hypervisor Developed by the University of Cambridge and supported, amongst others, by Microsoft The Xen virtual machine runs on the x86 architecture and uses Paravirtualization Xen is designed to run up to 100 VM’s simultaneously and efficiently Each VM is called a “Domain”, Control and management software are run on a VM called Domain0 running XenoLinux The Xen project has already ported the Linux OS (XenoLinux) and NetBSD. XenoXP (Windows XP port) was still under work but was abandoned due to the large XP code-base and to Microsoft legal limitations. XP can run unmidified as an HVM (Hardware VM) OpenSUSE, Fedora Core, Debian and NetBSD all include support (standard kernels and utility packages) for various versions of Xen
27
Xen 3.0 Architecture AGP ACPI PCI SMP VT-x 32/64bit
Domain0 VM1 VM2 VM3 Device Manager & Control s/w Unmodified User Software Unmodified User Software Unmodified User Software GuestOS (XenLinux) GuestOS (XenLinux) GuestOS (XenLinux) Unmodified GuestOS (WinXP)) AGP ACPI PCI Back-End Back-End SMP Native Device Driver Native Device Driver Front-End Device Drivers Front-End Device Drivers VT-x Control IF Safe HW IF Event Channel Virtual CPU Virtual MMU 32/64bit Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
28
VMware 5.5 Architecture
29
Paravirtualization in Xen
The Architecture is called xen_x86, similar to x86 but requires Xen Hypercalls for privileged operations Guest OS’s kernel is modified to understand the virtualized environment: Wall-Clock time vs. virtual processor time Expose real resource availability to enable the OS to optimize it’s behavior
30
CPU Virtualization Xen runs in ring 0 (most privileged) while the guest OS runs in rings 1 or 2. Guest apps run in ring 3 Page Faults are generated if the guest OS tries to use privileged instructions TLB Flushed for every VM context switch All exceptions are caught by Xen and forwarded to handlers registered with Xen by the guest OS’s Guest OS’s can register ‘fast’ exception handlers, verified by Xen only when installed and do not have to be indirected via ring 0 Scheduling is done using the BVT (Borrowed Virtual Time) algorithm so to insure fair-sharing
31
Device I/O Xen does NOT emulate hardware devices (e.g. VMware). Xen uses a set of device abstractions I/O data in transferred via Xen using shared-memory asynchronous buffer-rings, Xen performs validation checks Xen uses an event mechanism instead of interrupts The notifications can be held by the guest OS to avoid frequent wake-up notifications
32
Memory Xen does NOT keep shadow page tables! (e.g. VMware)
Guest OS’s use the real page table where updates go through Xen for verification Guest OS’s can send several updates together for optimizations Physical memory is statically partitioned for each VM at the time of creation XenoLinux’s Balloon Driver adjusts a VM’s memory usage by passing pages back and forth between Xen and the page allocator.
33
Network Xen provides a virtual firewall-router (VFR) and guest OS’s can register virtual network interfaces (VIF’s) Domain0 can manipulate the VFR rules I/O rings are used to buffer packet transmission and reception To send packets the guest OS enqueues a buffer descriptor onto the transmit ring. Xen verifies VFR rules and pinnes the relevant page frames Packet reception is done by exchanging unused page frames for each packet
34
Disk Only Domain0 has direct unchecked access to physical disks
VM’s see virtual block devices (VBDs), created and configured from domain0 Xen reorders requests from VBDs for optimization – VBDs look like SCSI devices, guest OS’s can pass down reorder barriers to prevent reordering if needed. Xen performs permission checks on VBD requests and uses Zcopy using DMA for optimization Requests a serviced in batches and in a simple round-robin fashion
35
Performance Relative performance of native Linux (L),
XenoLinux (X), VMware 3.2 (V) and User-Mode Linux (U)
36
Xen 3.0 – The goodies! I/O Spaces – Drivers run in dedicated domains, can enforce security and stability Event channels are used to buffer interrupts New hardware features (Pacifica’s DEV and similar in Silvervale) allow for very efficient security and exclusion of devices, can eliminate paravirtualization and thus run unmodified OS’s Using device channels to pass requests and responses, Xen let’s many VM’s use the same driver instance Xen 3.0 supports SMP 32/64bit hosts and guests, PCI, ACPI and AGP/DRM devices, supports VT-x and Pacifica PPC and IA64 port are now in progress
37
Xen – Use the source, Luke!
The Xen source is a BIG (~150KLOC) patch to the Linux kernel It compiles 3 kernels, combined into a single binary by default: The Xen Hypervizor The Dom0 kernel A DomU kernel The source packages of the standard SUSE, Debian, Ubuntu, Fedora Code and RedHat kernels include the Xen patch The standard Xen distribution is built around the kernel (the standard kernel for SUSE Enterprise, RedHat Enterprise etc.)
38
VM Migration – Why? Avoid many difficulties faced by process-level migration (such as the need for the home node) In-memory state can be transferred consistently A group of mutual processes can be moved together Server farms operator can let the users take care of the VM’s while they take care of the hardware VM Migration can help with cluster load balancing with greater ease
39
VM Migration - Overview
Daneli migrates VM’s by stopping, copying and resuming VMware has developed it’s VMotion OS migration technology Zap migrates process domains via partial OS virtualization NomadBIOS is a virtualization and migration system built on top of the L4 microkernel, using pre-copy MOSIX migrates processes but keeps them connected to the home node’s deputy process
40
Migrating Memory Can be divided to 3 phases:
Push phase – Pages are pushed to the destination machine while the VM keeps running, changed pages must be resent Stop-and-copy-phase – The source VM stops, pages are copied to the destination VM, new VM is started Pull phase – The new VM pulls non-copied pages when they are needed
41
VM Migration – Xen’s way
42
VM Migration - Network A migrating VM retains it’s protocol states (e.g. TCP PCB) Within a subnet, the new host will issue an unsolicited ARP reply advertising the VM’s new location Alternatively, switches can sense that the VM’s MAC has moved to a new port and correct themselves For WAN’s, Xen suggests an indirection layer such as Mobile IP Xen does not handle local storage migration and relies on a NAS (network-attached storage) For block devices, Xen suggests mirroring
43
VM Migration – Optimizations
Xen uses a Dynamic Rate-Limiting algorithm to apply a smart bandwidth usage The algorithm compensates for large WWS’s (Writable Working Sets) by dynamically increasing the used bandwidth with each iterative pre-copy to a certain size threshold Once the threshold has been achieved, Xen moves to the Stop-and-copy phase
46
Summary One can say that the industry is re-embracing the idea of virtualization Virtualization can solve many problems for home users, developers and businesses Virtualization can help with security issues VM migration can solve process migration difficulties VM migration helps with cluster load balancing and maintenance
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.