Evaluation of Delta Compression Techniques for Efficient Live Migration of Large Virtual Machines Petter Svärd, Benoit Hudzia, Johan Tordsson and Erik Elmroth Umeå University, Dept of Computing Science VEE 2011, Newport Beach, CA, USA

Live migration “Transfer a VM from one host to another without disrupting services.” The VM:s state (memory pages) is transferred in the background with the VM still running The file system is typically located on a NFS and is not moved

Live migration - Typical algorithm The time between the VM is suspended and resumed is defined as the downtime Our goal is to reduce the downtime

Live migration - Problems with the typical algorithm When migrating memory intensive VM or over slow NW links: 1.Memory pages can be dirtied faster than they are transferred over the network 2.The VM has to be suspended for an extended period of time -> long downtime 3.Network connections time out and drop / triggers fail Leads to disruption of services

Live migration - Problems with the typical algorithm (cont) Problem dirtying rate > migration throughput Possible Solutions Decrease dirtying rate or increase migration throughput Decreasing dirtying rate might hurt server performance and disrupts services Increase migration throughput!

Delta compression - Increasing migration throughput Overall idea: transfer changes to pages instead of the full page contents thus increasing migration throughput Store sent pages in a cache When transferring, if the page is cached, compute an XOR delta page Compress the delta page

Delta compression - continued Wasting time on cache misses Efficient caching scheme and compression algorithm is vital! Vanilla (no compr.) Delta compression

Delta compression - caching Desired properties: Lean Constant seek time regardless of size L2 caching scheme

Delta compression - compression Desired properties: Lean (low cpu usage) Effective (high compression ratio) General purpose The XOR delta page is suitable for RLE compression –(Symbol)(Repetitions) → AAAAABBBCCCCC = 5A3B5C XOR BinaryRunLengthEncoding -> XBRLE

XBRLE compression - Source side algorithm

XBRLE compression - Destination side algorithm

XBRLE compression - conceptual illustration

Implementation Modified version of qemu-kvm userspace code to support the XBRLE migration algorithm. Lean, ~500 LoC Evaluation done on version

Demo Migrating streaming video over 10Mbit/s Before migration:

Demo Migrating streaming video (cont) After migration:

Demo Migrating streaming video (cont)

Evaluation - Test cases Memory write benchmark (lm_bench) –1 GB RAM, 1 vcpu VM –Near ideal case Transcoded HD Video –1 GB RAM, 1 vcpu VM –Real-world, non-ideal case SAP ERP application –8 GB RAM, 4 vcpus VM –Large business application –Relies on transactions and is thus sensitive to extended downtime

Evaluation - Experimental setup Benchmark and HD Video 2x 2,66GHz core2quad 16GB RAM NFS share on source machine 100Mbit/s Network SAP ERP 2x 3,0GHz Xeon dual-core 32GB RAM 16TB Raid 5, 6Gbits/s trunked NFS server 1000Mbit/s Network

Evaluation - Benchmark Downtime reduced by a factor of 100 Throughput increased by 63 %

Evaluation - Streaming video UDP downtime reduced from 8 s to 1 Migration is transparent using XBRLE

Evaluation - SAP ERP The ERP application was non-responsive on resume using the vanilla algorithm but survived using XBRLE “Rule of thumb” is that more than 0.5 s of downtime might hurt the system. Measured downtime was 0.2 for XBRLE and 2 for vanilla. Vanilla XBRLE

Conclusion Delta compression works well migrating VMs running workloads with a highly compressible working set VMs running heavy workloads with large working sets and/or over slow networks (i.e., WANs).

Future work Page priority algorithm Avoid re-sends of pages that are dirtied frequently Promising early results