Urgent Virtual Machine Eviction with Enlightened Post-Copy Yoshihisa Abe†, Roxana Geambasu‡, Kaustubh Joshi, and Mahadev Satyanarayanan† †Carnegie Mellon University, ‡Columbia University, AT&T Research VEE 2016

Virtual Machine (VM) Migration Transfer executing VMs between physical hosts Key operation in cloud resource provisioning – Commonly used for planned activities – e.g., machine maintenance 2

Migration under Contention Multiple VMs co-located for resource efficiency – Performance degrades at peak times Load balancing through resource re-allocation to VMs 3

Objectives Migration for contending VMs under tight time requirement Focus on fast execution transfer and short total duration 4

Current Standard: Live Migration Pre-copy focusing on minimizing down time [Clark ’05, Nelson ’05] 5 [Clark ’05] C. Clark et al. Live Migration of Virtual Machines. NSDI ’05. [Nelson ’05] M. Nelson et al. Fast Transparent Migration for Virtual Machines. USENIX ATC ’05.

Approach to VM Contention Guest OS participates in migration operation – Guest OS is increasingly aware of virtualized environments i.e., para-virtualization Design goals – First priority is resolving contention between loaded VMs Focus on total performance of VMs being affected – Migrating VM should not linger on source Exploit cooperation of guest OS for performance recovery 6

Exploiting Execution Knowledge Enlightenment: knowledge passed from guest to hypervisor [Miłós ’09] Obtain current knowledge explicitly for prioritized memory transfer 7 [Miłós ’09] G. Miłós et al. Satori: Enlightened Page Sharing. USENIX ATC ’09.

Enlightened Post-Copy Exploit enlightenment to identify memory needed for current workload 8

Page Prioritization Rationale: use existing memory management information (in Linux) – Less than 15 lines for tracking kernel/user allocation – Several lines in 16 places for tracking Not Allocated 9

Implementation Guest OS passes enlightenment through bitmaps – Priority bitmap: contains page priorities, written once upon migration – Free bitmap: indicates used/unused pages, updated real-time 10

Experiment Set-Up 11

Example Workloads Two VMs with same workloads under contention VMs: 30 GB memory, 8 (Cassandra) or 4 (Memcached) cores Cassandra: NoSQL database – YCSB with 50% read and 50% read-modify-write – CPU-intensive Memcached: in-memory key-value store – Memslap benchmark with 1:9 set-get ratio – Memory-intensive 12

Results: Cassandra 13

Results: Memcached 14

Migrating Idle VM Enlightened post-copy is designed for busy VMs – Non-busy VMs are amenable to live migration Idle VM with 30 GB memory Approximately 1 GB in use, migrated over 10 Gbps qemu-kvm compresses zeroed-out pages (≈29 GB) – Still needs to scan their contents 15

Conclusion Migration scheme for VMs under contention – Requires fast execution transfer and short total duration – Considers aggregate performance of VMs, rather than target VM only Guest OS participates in migration operation – Allows hypervisor to migrate memory in prioritized manner Encourages optimistic VM co-location – Limit performance penalties by resolving VM contention fast 16