Software & Services Group A Primer on Virtualization 1

Software & Services Group Agenda Westmere EP on Intel Roadmap Adv. Enc. Std: New Instructions (AES-NI) in Westmere EP Intel® Virtualization Technology – a primer –Evolution of Virtualization –Intel Virtualization Technologies –A Framework for Optimizing Virtualization – improving efficiency and scaling –Reducing Virtualization Overheads: Processor, Memory, I/O –Improving VM scaling –VMM readiness –Improving VMM security through TXT Summary Content condensed from: “ Intel® Virtualization Technology in the Enterprise ”, Rich Uhlig, Intel Fellow, IDF 2009 San Francisco USA.

Software & Services Group Evolution of Virtualization 3 Flexible Resource Management Basic Consolidation No Virtualization Drives Priorities: Lower VMM Overheads Richer Workloads in VMs Seamless VM Migration Power Efficiency Increased Scaling (VMs, vCPUs) Improved Reliability Host OS Host OS Host OS VMM Host OS VMM Host OS VMM Host OS VMM Host VMM Host OS

Software & Services Group Intel® Virtualization Technologies Intel® VT for Directed I/O Reliability and Security through device Isolation I/O performance with direct assignment Intel® VT for Connectivity NIC Enhancement with VMDq Single Root IOV support Network Performance and reduced CPU utilization Intel® I/OAT for virtualization Lower CPU Overhead and Data Acceleration Intel® VT-x Hardware assists for robust virtualization Intel® VT FlexMigration – Flexible live migration Intel® VT FlexPriority – Interrupt acceleration Intel® EPT – Memory Virtualization Network Chipset Processor Intel® VT-x Intel® VT-d Intel® VT-c

Software & Services Group A Framework for Optimizing Virtualization 5 Reduce overheads from virtualization Intel® VT-x Latency Reductions Extended Page Tables Virtual Processor IDs APIC Virtualization (Flex Priority) I/O Assignment via DMA Remapping Intel® Virtualization Technology (Intel® VT) for IA-32, Intel® 64 and Intel® Architecture (Intel® VT-x) Introduce capabilities that increase scaling out across VMs Intel® Hyper-Threading Technology PAUSE-loop Exiting Network Virtualization VMM OS Host OS … 2. … and scale out across VMs 1. Reduce overhead within each VM…

Software & Services Group What makes VM Context Switching expensive 6 Saving-Loading privileged state Compounded by consistency checking Addressing Context changes Translation Lookaside Buffer (TLB) flushes Virtual Machine Ctl Structure (VMCS) – Maintains Guest & Host reg. state – “Backed” by host physical memory – Accessed via architectural VMREAD / VMWRITE – Enables caching of VMCS state on-die Virtual Processor IDs (VPIDs) – Tag µarch structures (TLBs) – Removes need to flush TLBs Reducing VT Latencies

Software & Services Group Intel ® Virtualization Technology (VT-x) VT-x® provides architected assists to allow guest OSes to run directly on hardware On Nehalem and Westmere VT-x is extended with: Extended Page Tables (EPT) Eliminates VM exits to the VMM for shadow page- table maintenance Virtual Processor IDs (VPID) Avoid flushes on VM transitions to give a lower-cost VM transition time Guest Preemption Timer -- lets a VMM preempt a guest OS Aids VMM vendors in flexibility and Quality of Service (QoS) Descriptor Table Exiting –Traps on modifications of guest DTsAllows VMM to protect a guest from internal attack Transition Latency reductions Continuing improvements in microarchitectural handling of VMM round trips

Software & Services Group Issues with abstracting physical memory 8 VMM CPU 0 VM 0 VM n Guest Page Tables TLB Shadow Page Tables Memory Induced VM Exits Remap Address Translation Guest OS expects contiguous, zero- based physical memory VMM must preserve this illusion Page-table Shadowing VMM intercepts paging operations Constructs copy of page tables Overheads VM exits add to execution time Shadow page tables consume significant host memory Guest Page Tables

Software & Services Group How Extended Page Tables help with abstracting Physical Memory 9 Intel® Virtualization Technology (Intel® VT) for IA-32, Intel® 64 and Intel® Architecture (Intel® VT-x) Extended Page Tables (EPT) Map guest physical to host address New hardware page-table walker Performance Benefit A guest OS can modify its own page tables freely and without VM exits Memory Savings A single EPT supports entire VM: instead of a shadow pagetable per guest process

Software & Services Group 10 TLB CR3 PML4 PDP PDE PTE EPTPL4L3L2L1 EPTPL4L3L2L1 EPTPL4L3L2L1 EPTPL4L3L2L1 EPTPL4L3L2L1 Linear Addr Physical Addr Extended Page Table (EPT) Walk 2-level TLB reduces page-table walks VPID tags help to retain TLB entries Paging-structure caches Cache intermediate steps in walk Result: Reduce length of walks Common case: Much better than 24 steps

Software & Services Group Difficulties in virtualizing I/O 11 VMM VM 0 VM n 1 I/O Device Emulation Memory Storage Network Guest Device Driver Device Model Guest Device Driver Device Model 2 Para- virtualization Virtual Device Interface Traps device commands Translates DMA operations Injects virtual interrupts Software Methods I/O Device Emulation Paravirtualize Device Interface Challenges Controlling DMA and interrupts Overheads of copying I/O buffers Physical Device Driver

Software & Services Group Solution: DMA remapping (Intel® VT-d) 12 DMA Requests Device IDVirtual Address Length Memory Access with Host Physical Address DMA Remapping Engine Translation Cache Context Cache Fault Generation Memory-resident Partitioning & Translation Structures Device Assignment Structures Address Translation Structures Device D1 Device D2 Address Translation Structures Bus 255 Bus 0 Bus N Dev 31, Func 7 Dev P, Func 1 Dev 0, Func 0 Dev P, Func 2 4KB Page Frame 4KB Page Tables DMA Requests Device IDVirtual Address Length DMA Remapping Engine

Software & Services Group Intel® VT FlexPriority 13 VMM APIC-TPR access in software VM Guest OS VM Exits Fetch/decode instruction Emulate APIC-TPR behavior Thousands of cycles per exit Without Intel VT FlexPriority VM VMM APIC TPR access in HW Guest OS No VM Exits Instruction executes directly Hardware emulates APIC-TPR access No VM exit in the common case configure With Intel VT FlexPriority APIC Task Priority Register (TPR) Controls interrupt delivery through the APIC Accessed very frequently by some guest OSes

Software & Services Group Technologies to improve VM scaling 14 Hyper-threading Decreasing lock holder preemption impact Network virtualization with Virtual Machine Device Queues Single Root I/O Virtualization (SR-IOV)

Software & Services Group Reducing Lock Holder preemption impact 15 Problem: In an SMP guest, a vCPU holding a lock may get preempted Other vCPUs that attempt to acquire that lock spin for the full quantum Solution: Pause-Loop Exiting (PLE) Spin-locking code typically uses PAUSE instructions in a loop A longer than “normal” loop duration taken as a sign of lock- holder preemption When that happens, HW forces an exit into VMM VMM takes control and schedules some other vCPU spin_lock: attempt lock-acquire; if fail { PAUSE; jmp spin_lock }

Software & Services Group Network Virtualization: HW traffic management 16 Intel® Virtualization Technology (Intel® VT) for Connectivity (Intel® VT-c) Virtual Machine Device Queues (VMDq) Data packets grouped and sorted in HW Packets sent to their respective VMs Round-robin servicing on transmit Tests measure Wire Speed Receive (Rx) Side Performance With VMDq on Intel ® Gigabit Ethernet Controller Source Intel. Throughput (Gbps) w/o VMDq w/ VMDq w/ VMDq Jumbo Frames MAC/PHY NIC w/VMDq Layer 2 Classified Sorter Rxn Rx1 Rx2 Rx1 Rxn Rx1 Idle Tx2 Txn Idle Tx2 Txn Tx1 VM 1 (vNIC) VM 2 (vNIC) VM n (vNIC) Layer 2 Software Switch VMM LAN Rx1 Rx2 Rx1 Rxn Rx1 Rxn Rx1 Rx2 Rxn

Software & Services Group PCI-SIG SR-IOV Description: – PCI-SIG Single Root I/O Virtualization (SR-IOV) Standard: – Allows for I/O devices to be simultaneously shared among VMs – Virtual interfaces can be directly assigned to reduce routing overheads Benefits: – Provide near native performance due to direct connectivity – Allows direct VM control of I/O virtual functions Requirements: – Intel VT-d as the core platform ingredient – BIOS support of SR-IOV

Software & Services Group Westmere: Trusted Execution Technology (TXT) Server Extensions TXT uses features in processor, chipset and TPM to enable more secure platforms TXT works through measurement, memory locking and sealing secrets TXT helps prevent software attacks – such as, attempts to insert rogue VMM (rootkit hypervisor) – and, reset-attacks that are designed to compromise platform secrets in memory – and, BIOS and firmware update attacks Processor Chipset VT Processor TPM Helps prevent hijacking by rootkit TXT technology incorporates multiple components TXT Makes Platforms More Robust Against SW-based Attacks Platform hardware with VT-x support Rootkit Hypervisor VMM Guest VM

Software & Services Group Intel VT Features/VMM Support FeatureVMware ESXMicrosoft Hyper-V Xen OSSRed Hat (RHEL) KVMNovell (SLES) Min Rev DateMin Rev DateMin Rev DateMin Rev DateRevDateMin Rev Date Processor (VT-x)Min Rev for Nehalem 3.5U4NowWin Svr ’08 NowAnyNowAnyNowAny Recent NowAnyNow FlexPriority3.5U4NowWin Svr ’08 Now3.1Now5.2Now2.6.24Now10 SP1 Now FlexMigration3.5U4NowWin Svr ’08 2 Now3.3NowTBD EPT + VPID4.0NowWin Svr ’08 R2 1H’103.3Now5.3Now2.6.26Now10 SP2 Now Chipset (VT-d)VT-d24.0NowTBD 3.3Now5.4Q3’ Now11Now Network (VT-c) SR-IOVTBD Now 3 TBD VMDq3.5U4NowWin Svr ’08 R2 1H’10TBD Power Mgmt (S, P, C, T States) 4.0Now1.0 (S- state TBD) Now3.3NowTBD 11Now SSE NowTBD AnyNowAnyNowAnyNow Turbo4.0Now1.0NowAnyNowAnyNowAnyNow HT (SMT)3.5U4Now1.0NowAnyNowAnyNowAnyNow PWR Other 1 Requires other ecosystem support 2 Product allows capability but robustness improvements in future releases 3 PCI-SIG SR-IOV standard is supported now Notes: Table is based on latest information as of: VT-c : July 09; VT-x and VT-d: May 09 and subject to change; Please contact vendors directly for unreleased products Virtualization

Software & Services Group 20