Exploiting Data Deduplication to Accelerate Live Virtual Machine Migration Xiang Zhang 1,2, Zhigang Huo 1, Jie Ma 1, Dan Meng 1 1. National Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences 2. Graduate University of Chinese Academy of Sciences

Outline Introduction Design Implementation Evaluation Conclusion & Future Work

Live Migration Definition Migrating OS and Apps as a whole to another physical machine without rebooting the VM Advantages Load Balance Services Consolidation Fault Tolerance... Usually a shared storage is deployed Migrating VCPU context and memory image

Pre-Copy Pre-Copy is the default choice in Xen First phase, initial memory pages are copied Second phase, several rounds of incremental synchronization are employed Last phase, VM is suspended, remaining memory image and VCPU context are copied Pre-Copy is reliable

Motivation of Research Performance metrics of migration Total Data Transferred Total Migration Time Downtime Necessity for improving performance of Migration Apps suffer less time of performance degradation Would not miss many migration opportunities Shorter downtime for latency-sensitive Apps

Outline Introduction Design Implementation Evaluation Conclusion & Future Work

Analyzing Migration Data Regularities During the first phase Zero pages are in the majority for lightweight workloads At least 25% of non-zero pages are identical or above 80% similar Ratios of identical and similar pages to reference pages are 8:1 at least During last two phases Little zero pages At least 50% of pages are above 80% similar to their old versions Conclusion Too many redundant data transferred during migration. Migration with Data Deduplication (MDD) Denominator Pecentage of pages whose similarity are above 80% (%) CompilationVODStaticWebBankingEcommerceSupport Non-Zero Pages During the First Phase All Transferred Pages During Last Two Phases

How to Find Identical and Similar Pages (1) HashSimilarityDetector(k, s, c) [21] Hashes (k * s) blocks on the page, and groups them into k groups of s hashes each For each hash fingerprint, c candidates are stored as reference pages HashSimilarityDetector(2, 1, 1), SuperFastHash of 64-byte blocks

How to Find Identical and Similar Pages (2) Similarity is transitive P trans ≈ P old, P hash ≈ P trans, so P hash ≈ P old Need not to cache all the transferred pages Only the privileged domain in source needs to maintain hash table Reference pages are transferred and can be found by their frame numbers in destination

How to Find Identical and Similar Pages (3) Only indexing by hash fingerprints may cause data inconsistency FPHash PxPx b1 P x-old b1 P x-old b1 P x-new b2 P x-new b2 PyPy b1 SourceDestination

How to Find Identical and Similar Pages (4) Double-Hash to eliminate data inconsistency PxPx b1 P x-old b1 FNHash FPHash PyPy b1 b2 P x-old b1 P x-new b2 P x-new b2 SourceDestination

Data Deduplication during Migration In source P parity = P trans ⊕ P ref Encoding P parity with RLE, then migrating In destination Decoding to get P parity P trans = P parity ⊕ P ref Advantages P parity contains less information than P trans Reflects the exact different data at bit level Contains many blocks of continuous zeros, even RLE can compress effectively RLE is one of the fastest encoding algorithm

Outline Introduction Design Implementation Evaluation Conclusion & Future Work

Implementation Do data deduplication parallelly by multi-thread Hash tables are maintained by LRU Extended memcmp() to reduce the overhead of judging zero pages

Outline Introduction Design Implementation Evaluation Conclusion & Future Work

Experimental Setup Experiment platform Cluster composed by six identical servers One storage server, iSCSI protocol, isolated gigabit Ethernet Two servers, which act as the source and destination of migration Three servers work as clients for workloads Server configuration Two Intel Xeon E5520 quad-core CPUs, 2.2GHz 8GB DDR RAM Gigabit LAN Xen and modified Linux Migrated VM is configured with one VCPU and 1GB RAM Migration shares the same network with workloads. Workloads Compilation, VOD, static web server, dynamic web server

Total Data Transferred Transferred data is reduced by 56.60% on average Number of transferred pages is reduced by 48.73% on average (Banking) Compression ratio is 49.27% on average (Banking)

Total Migration Time and Downtime MDD decreases total migration time and downtime by 34.93% and 26.16% on average Less data transferred Number of migration rounds are not reduced

CPU Resource Required Extra CPU resource which MDD requires is 47.21% of a CPU CompilationVODBankingEcommerceSupport Xen MDD DD Average CPU Utilization Ratio of Migration (%)

Influence to Apps Run Apache in migrated VM, and migrate it in normal and adaptive mode respectively The more limited network bandwidth is, the more essential data deduplication is Total Data TransferredTotal Migration TimeDowntime Normal Migration Adaptive Migration Benefits of MDD in Different Migration Mode (%)

Outline Introduction Design Implementation Evaluation Conclusions & Future Work

Conclusion & Future Work Conclusion Study the characteristics of run-time memory image data during migration Present the design and implementation of MDD MDD reduces total data transferred, total migration time and downtime by 56.60%, 34.93% and 26.16% respectively, reduces the influence of migration to Apps. Future work Extend MDD into live whole-system migration in wide-area environment

Thank You! Any Questions?

Related Work Reducing transferred data Post-Copy [7][12] Self-Ballooning [7] Trace and replay [13] Adaptive compression [8] Improving network bandwidth InfiniBand RDMA [14]

Backup EcommerceSupport