Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hypervisor-Assisted Application Checkpointing for High Availability Min Lee Joint work with A. S. Krishnakumar, P. Krishnan, Navjot Singh, Shalini Yajnik.

Similar presentations


Presentation on theme: "Hypervisor-Assisted Application Checkpointing for High Availability Min Lee Joint work with A. S. Krishnakumar, P. Krishnan, Navjot Singh, Shalini Yajnik."— Presentation transcript:

1 Hypervisor-Assisted Application Checkpointing for High Availability Min Lee Joint work with A. S. Krishnakumar, P. Krishnan, Navjot Singh, Shalini Yajnik

2 © 2009 Avaya Inc. All rights reserved. Introduction  Virtualization technology – Gets adopted widely – Proves its usefulness – Most applications run well Natively run  Some important applications don’t run well – Certain operations cannot run natively – Instead they use hypercalls – Our target: Application-checkpointing 2

3 © 2009 Avaya Inc. All rights reserved.3 Xen Virtual Machine Monitor Xen Hypervisor Modified Guest OS Modified Guest OS Modified Guest OS … Virtual machines Virtual hardware (vCpu, vDisk, vNic, vMemory etc.) Physical hardware (Cpu, Disk, Nic, Memory etc.) Applications (Taken/adapted from ‘Xen and co.’ slides)

4 © 2009 Avaya Inc. All rights reserved. High Availability Approaches  Categories – Application-transparent No changes to application or guest Xen-specific: Remus, Kemari – Application-assisted Application implements the checkpointing logic Flexible and light-weight  We are targeting – Application-assisted under virtualization Xen-specific Applicable to general hypervisors 4

5 © 2009 Avaya Inc. All rights reserved. Hypervisor-Assisted Application Checkpointing  Application checkpointing – Provides transactional properties to the traditional heap Make high available heap – Processes survive failures – Has performance issues in Xen  Our technique improves application-checkpointing performance in Xen 5

6 © 2009 Avaya Inc. All rights reserved. High Availability List_add() List_del() Magical mirror changes List_add() Crash Takeover List_add() 6

7 © 2009 Avaya Inc. All rights reserved. Transaction APIs  List of dirty-pages – Written pages  Mprotect() system call – Write-protect – SIGSEGV signal Tstart(); List_add(); Tend(); int declare(addr, size); void undeclare(Tid); void Tstart(Tid); void Tend(Tid, dirty_pages); List_add(); Tstart(); List_add(); List_del(); List_add(); List_del(); Tend(); List_add(); List_del(); List_add(); List_del(); Examples: APIs: 7

8 © 2009 Avaya Inc. All rights reserved. PT – Existing Approach  Get dirty pages List_add(); handler() { mprotect(unprotect); add_to_dirty_pages(); } 5 List_add(); 7 7 Tstart(); Tend(); … Declare() {} Undeclare() {} Process’ view (virtual pages) 8

9 © 2009 Avaya Inc. All rights reserved. PT Call-Flow  Pure User-level User OS Hypervisor Mprotect() Page fault Signal For every dirty page TLB flush 9

10 © 2009 Avaya Inc. All rights reserved. Approaches PT-basedEmulation-basedScan-based Pure user space PT (Exisiting) Hypervisor- assisted 10

11 © 2009 Avaya Inc. All rights reserved. Approaches PT-basedEmulation-basedScan-based Pure user space PT (Exisiting) Emulation Hypervisor- assisted PTxenEmulxenScanxen Our approaches 11

12 © 2009 Avaya Inc. All rights reserved. Our Approaches 12

13 © 2009 Avaya Inc. All rights reserved. Emulation  Under the condition – Most transactions are small List_add(); handler() { emulate(); log_to_write_buffer(); } (Addr1,100) List_add(); (Addr2,200) Tstart() {} Tend(); … Declare(); Undeclare(); Process’ view (virtual pages) 13

14 © 2009 Avaya Inc. All rights reserved. Hypervisor-Assisted:User-to-hypervisor call  Overhead through OS unnecessary – Directly talk to Xen  Move checkpointing to Xen level – Add new interrupt vector 0x80: system call 0x82: hypercall from guest OS 0x84: hypercall from user (Newly added)  Xen-based approaches without any changes to guest OS. 14

15 © 2009 Avaya Inc. All rights reserved. Hypervisor-Assisted:User-to-hypervisor call  User-to-Hypervisor Call 15

16 © 2009 Avaya Inc. All rights reserved. PTxen  Implement PT in Xen List_add(); page_fault() { mprotect(unprotect); add_to_dirty_pages(); } 5 List_add(); 7 7 Tstart() {} Tend(); … Declare(); Undeclare() {} Process1, (1-12) Xen Process’ view (virtual pages) 16

17 © 2009 Avaya Inc. All rights reserved. Emulxen  Emulation in Xen List_add(); Tstart() {} Tend(); … Declare(); Undeclare(); Process1, (1-12) page_fault() { emulate(); log_to_write_buffer(); } (Addr1,100) (Addr2,200) Xen Process’ view (virtual pages) 17

18 © 2009 Avaya Inc. All rights reserved. Scanxen  Idea – Scan page table rather than trapping writes – Hardware marks dirty bit List_add(); 5 7 Tstart() {} Tend(); … Declare(); Undeclare(); Process1, (1-12) Xen = Dirty-bit in page table scan_page_table() { collect_dirty_bit(); add_to_dirty_pages(); } Process’ view (virtual pages) 18

19 © 2009 Avaya Inc. All rights reserved. Microbenchmark transactions 10MB heap size 19

20 © 2009 Avaya Inc. All rights reserved. Microbenchmark  Transactional heap size – For simplicity, whole heap is protected  Transaction – Write per pages (wpp) # of writes per pages – Page per transaction (ppt) # of unique pages written – # of writes = wpp * ppt  Scanxen – Impacted by only heap size – Not wpp, ppt, or transaction size 20

21 © 2009 Avaya Inc. All rights reserved. PT vs PTxen  PTxen shows 10x speedup  PT, PTxen get impacted by ppt 21 PT (wpp = 4, 8, 16 overlapped) PTxen (wpp = 4, 8, 16 overlapped)

22 © 2009 Avaya Inc. All rights reserved. Emulation vs emulxen  Emul-based gets impacted by transaction size  Emulxen shows 4x speedup emul emulxen ppt (wpp=16) : ppt (wpp=8) : ppt (wpp=4) :

23 © 2009 Avaya Inc. All rights reserved. PT Call-Flow  Pure User-level User OS Hypervisor Mprotect() Page fault Signal User OS Hypervisor Page fault declare() For every dirty page TLB flush  Xen-assisted 23

24 © 2009 Avaya Inc. All rights reserved. Evaluation Source from the book “Data Structures and Algorithm Analysis in C (Second Edition),” by Mark Allen Weiss 24

25 © 2009 Avaya Inc. All rights reserved. Data Structures OPS_PER_T=1writespages avgminmaxavgminmax aa (AA-trees)insert delete avl (AVL trees)insert bin (Binomial queues)insert dsl (Deterministic skip list)insert hashquad (Quadratic probing hash)insert hashsepchain (Separate chaining hash)insert leftheap (Leftist heap)insert delete heap (binary heaps)insert delete list (Linked list)insert delete queue (Queues)insert delete rb (Red black tree)insert splay (Splay trees)insert delete tree (Binary search tree)insert delete

26 © 2009 Avaya Inc. All rights reserved. Evaluation Results 1 26 PTXenPT PTXenPT PTXenPT PTXenPT

27 © 2009 Avaya Inc. All rights reserved. Evaluation Results 2 27 PTXen PT PTXenPT PTXenPT

28 © 2009 Avaya Inc. All rights reserved. Evaluation Results 3  Scanxen shows almost constant 2.5sec across all 28 PTXenPTPTXenPT

29 © 2009 Avaya Inc. All rights reserved. Evaluation Summary  Emulxen has up to 4x speedup compared to emulation  PTxen has up to 13x speedup compared to PT 29

30 © 2009 Avaya Inc. All rights reserved. Transaction Aggregation  OPT=1 – A single operation (e.g. an insert or a delete)  OPT=5 – Multiple operations merged into one transaction – # of writes increases linearly – # of unique pages touched remains same in most cases  It should benefit PT-based approaches – Because of their heavy dependence on PPT – Details in the paper 30

31 © 2009 Avaya Inc. All rights reserved. Conclusion  Family of application checkpointing techniques introduced  Emulation-based techniques – Useful for small transactions [fewer # of writes]  Hypervisor-Assisted Application Checkpointing – 4x~13x than userspace implementation 31

32 © 2009 Avaya Inc. All rights reserved.32 Thank you!

33 © 2009 Avaya Inc. All rights reserved. Extra Slides 33

34 © 2009 Avaya Inc. All rights reserved. Emulation vs PT  Emul-based is good for small transaction – Roughly wpp=5 and wpp=1.3 is breakeven point Note scale difference 34

35 © 2009 Avaya Inc. All rights reserved. Scanxen vs PT  For small buffer and large ppt, scanxen might be better – Not the case in our experiments Note scale difference 35 1MB 2MB 3MB 4MB 5MB PT PTxen 40KB 80KB 120KB Scanxen heapsize

36 © 2009 Avaya Inc. All rights reserved. Scanxen vs emulation  Scanxen might be better than emulation – For big transactions 36 Scanxen emul emulxen

37 © 2009 Avaya Inc. All rights reserved.37

38 © 2009 Avaya Inc. All rights reserved.38

39 © 2009 Avaya Inc. All rights reserved.  Operations per transaction – OPT=5, Merging transaction No impact to emulation-based ones Some slowdown for scanxen – Merging transactions Total # of pages written goes down effectively PT and PTxen becomes much better than emul/emulxen Still 13x improvement between PT and PTxen 39

40 © 2009 Avaya Inc. All rights reserved. Evaluation 40

41 © 2009 Avaya Inc. All rights reserved. Bandwidth : Amount Note that tree-insert is which is out of scale.  Emul-based mostly less than 2MB – No ‘diff’ process for emul-based 41

42 © 2009 Avaya Inc. All rights reserved. Bandwidth : Time  Emul-based mostly less than 5ms 42

43 © 2009 Avaya Inc. All rights reserved. Bandwidth : Percentage  Relatively small fraction – Except PTxen --- due to its minimum runtime 43

44 © 2009 Avaya Inc. All rights reserved. Microbenchmark 44 scanxen PT PTxen

45 © 2009 Avaya Inc. All rights reserved.45 emulxen emul scanxen

46 © 2009 Avaya Inc. All rights reserved.46 emulxen PTxen PT emul scanxen

47 © 2009 Avaya Inc. All rights reserved. Microbenchmark 47 writes Transactional heapDirty pages in Transactional heap Tstart() of PT Tend() of PTThree separate mprotect() calls writes Tstart() of PTxen Tend() of PTxenSingle PTxen() call

48 © 2009 Avaya Inc. All rights reserved.48 Main process Diff process diff dirty page Backup process Network


Download ppt "Hypervisor-Assisted Application Checkpointing for High Availability Min Lee Joint work with A. S. Krishnakumar, P. Krishnan, Navjot Singh, Shalini Yajnik."

Similar presentations


Ads by Google