1. Joshua Reich, Oren Laadan, Eli Brosh, Alex Sherman, Vishal Misra, Jason Nieh, and Dan Rubenstein 1 VMTorrent : Scalable P2P Virtual Machine Streaming

2. VM: software implementation of computer Implementation stored in VM image VM runs on VMM –Virtualizes HW –Accesses image VM Basics 2 VMM VM Image VM

3. Where is Image Stored? 3 VMM VM Image VM

4. Traditionally: Local Storage 4 VMM VM Local Storage

5. IaaS Cloud: on Network Storage 5 VMM VM Image Network Storage

6. Can Be Primary 6 VMM VM NFS/iSCSI VM Image Network Storage e.g., OpenStack Glance Amazon EC2/S3 vSphere network storage

7. Or Secondary 7 VMM VM Image VM Network Storage Local Storage e.g., Amazon EC2/EBS vSphere local storage

8. Either Way, No Problem Here 8 VMM VM Image Network Storage

9. Here? 9 VM Image Network Storage Bottleneck!

10. Lots of Unique VM Images 10 Network Storage on EC2 alone 54784 unique images* *http://thecloudmarket.com/stats#/totals, 06 Dec 2012http://thecloudmarket.com/stats#/totals

11. Unpredictable Demand 11 Network Storage Lots of customers Spot-pricing Cloud-bursting

12. Dont Just Take My Word The challenge for IT teams will be finding way to deal with the bandwidth strain during peak demand - for instance when hundreds or thousands of users log on to a virtual desktop at the start of the day - while staying within an acceptable budget 1 scale limits are due to simultaneous loading rather than total number of nodes 2 Developer proposals to replace or supplement VM launch architecture for greater scalability 3 12 1.http://www.zdnet.com/why-so-many-businesses-arent-ready-for-virtual-desktops- 7000008229/?s_cid=e539http://www.zdnet.com/why-so-many-businesses-arent-ready-for-virtual-desktops- 7000008229/?s_cid=e539 2.http://www.openstack.org/blog/2011/12/openstack-deployments-abound-at-austin-meetup- 129http://www.openstack.org/blog/2011/12/openstack-deployments-abound-at-austin-meetup- 129 3.https://blueprints.launchpad.net/nova/+spec/xenserver-bittorrent-imageshttps://blueprints.launchpad.net/nova/+spec/xenserver-bittorrent-images

13. Challenge: VM Launch in IaaS Minimize delay in VM execution Starting from time launch request arrives For lots of instances (scale!) 13

14. Naive Scaling Approaches Multicast –Setup, configuration, maintenance, etc. 1 –ACK implosion –multicast traffic saturated the CPU on [Etsy] core switches causing all of Etsy to be unreachable 2 14 1.[El-Sayed et al., 2003; Hosseini et al., 2007] 2.http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replicationhttp://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication

15. Naive Scaling Approaches P2P bulk data download (e.g., Bit-Torrent) –Files are big (waste bandwidth) –Must wait until whole file available (waste time) –Network primary? Must store GB image in RAM! 15

16. Both Miss Big Opportunity VM image access Sparse Gradual 16 Most of image doesnt need to be transferred Can start w/ just a couple of blocks VM image streaming

17. VMTorrent Contributions Architecture –Make (scalable) streaming possible: Decouple data delivery from presentation –Make scalable streaming effective: Profile-based image streaming techniques Understanding / Validation –Modeling for VM image streaming –Prototype & evaluation 17 not highly optimized

18. Talk Make (scalable) streaming possible: Decouple data delivery from presentation Make scalable streaming effective: Profile-based image streaming techniques VMTorrent Prototype & Evaluation (Modeling along the way) 18

19. Decoupling Data Delivery from Presentation (Making Streaming Possible) 19

20. Generic Virtualization Architecture 20 VM VMM Host Hardware VM Image FS Virtual Machine Monitor virtualizes hardware Conducts I/O to image through file system

21. Cloud Virtualization Architecture 21 VM VMM Hardware VM Image FS Either to download image Network Backend Network Backend Network backend used Or to access via remote FS

22. Custom FS Custom FS VMTorrent Virtualization Architecture 22 VM VMM Hardware VM Image FS Network Backend Network Backend Divide image into pieces But provide appearance of complete image to VMM Introduce custom file system

23. Custom FS Custom FS Decoupling Delivery from Presentation 23 VM VMM Hardware Network Backend Network Backend 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 VMM attempts to read piece 1 Piece 1 is present, read completes

24. Custom FS Custom FS 24 VM VMM Hardware Network Backend Network Backend 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 VMM attempts to read piece 0 Piece 0 isnt local, read stalls VMM waits for I/O to complete VM stalls Decoupling Delivery from Presentation

25. Custom FS Custom FS 25 VM VMM Hardware Network Backend Network Backend 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 FS requests piece from backend Backend requests from network Decoupling Delivery from Presentation

26. 26 VM VMM Hardware Network Backend Network Backend 0 0 Later, network delivers piece 0 Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Read completes Custom FS receives, updates piece VMM resumes VMs execution Decoupling Delivery from Presentation

27. Decoupling Improves Performance 27 VM VMM Hardware Network Backend Network Backend Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Primary Storage No waiting for image download to complete

28. Decoupling Improves Performance 28 VM VMM Hardware Network Backend Network Backend Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 X X Secondary Storage No more writes or re-reads over network w/ remote FS

29. But Doesnt Scale Assuming a single server, the time to download a single piece is t = W + S / (r net / n) W : wait time for first bit r net : network speed S : piece size n :# of clients 29 Transfer time, each client gets r net / n of server BW

30. Read Time Grows Linearly w/ n Assuming a single server, the time to download a single piece is t = W + n * S / r net W : wait time for first bit r net : network speed S : piece size n :# of clients 30 Transfer time linear w/ n

31. This Scenario 31 VM VMM Hardware Network Backend Network Backend Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 cs d

32. Alleviate network storage bottleneck 32 VM VMM Hardware Network Backend Network Backend Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Decoupling Enables P2P Backend Swarm P2P Manager P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Exchange pieces w/ swarm P2P copy must remain pristine

33. 33 VM VMM Hardware Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Space Efficient Swarm P2P Manager P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 FS uses pointers to P2P image FS does copy-on-write 6 6 7 7

34. 34 VM VMM Hardware Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Minimizing Stall Time Swarm P2P Manager P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Non-local piece accesses 6 6 7 7 4?4? 4?4? 4?4? 4?4? 4! Trigger high priority requests

35. Now, the time to download a single piece is t = W(d) + S / r net W(d) : wait time for first bit as function of d : piece diversity r net : network speed S : piece size n :# of peers P2P Helps 35 Wait is function of diversity Transfer time independent of n

36. High Diversity Swarm Efficiency 36

37. Low Diversity Little Benefit 37 Nothing to share

38. All peers request same pieces at same time t = W(d) + S / r net Low piece diversity Long wait (gets worse as n grows) Long download times P2P Helps, But Not Enough 38

39. 39 VM VMM Hardware Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 This Scenario Swarm P2P Manager P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 6 6 7 7 p2p d

40. Profile-based Image Streaming Techniques (Making Streaming Effective) 40

41. How to Increase Diversity? Need to fetch pieces that are Rare: not yet demanded by many peers Useful: likely to be used by some peer 41

42. Profiling Need useful pieces But only small % of VM image accessed We need to know which pieces accessed Also, when (need later for piece selection) 42

43. Build Profile One profile for each VM/workload Ran one or more times (even online) Use FS to track –Which pieces accessed –When pieces accessed Entries w/ average appearance time, piece index, and frequency 43

44. Piece Selection Want pieces not yet demanded by many Dont know piece distribution in swarm Guess others like self Gives estimate when pieces likely needed 44

45. Piece Selection Heuristic Randomly (rarest first) pick one of first k pieces in predicted playback window fetch w/ medium priority (demand wins) 45

46. Profile-based Prefetching Increases diversity Helps even w/ no peers (when ideal access exceeds network rate) 46

47. Profile-based window-randomized prefetch t = W(d) + S / r net High piece diversity Short wait (shouldnt grow much w/ n ) Quick piece download Obtain Full P2P Benefit 47

48. 48 VM VMM Hardware Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 Full VMTorrent Architecture Swarm P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 6 6 7 7 profile p2p p

49. Prototype 49

50. 50 VM Hardware Custom FS Custom FS 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 VMTorrent Prototype BT Swarm P2P Manager 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 6 6 7 7 profile Custom C Using FUSE Custom C++ & Libtorrent

51. Evaluation Setup 51

52. Testbeds Emulab [White, et. al, 2002] –Instances on 100 dedicated hardware nodes –100 Mbps LAN VICCI [Peterson, et. al, 2011] –Instances on 64 vserver hardware node slices –1 Gbps LAN 52

53. VMs 53

54. Workloads 54 Short VDI-like tasks Some cpu-intensive, some I/O intensive

55. Measured total runtime –Launch through shutdown –(Easy to measure) Normalized against memory-cached execution –Ideal runtime for that set of hardware –Allows easy cross-comparison different VM/workload combinations Different hardware platforms Assessment 55

56. Evaluation 56

57. 100 Mbps Scaling 57 Starting to increase

58. Due to Decreased Diversity 58 # peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max

59. Due to Decreased Diversity 59 # peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max

60. Due to Decreased Diversity 60 # peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max We optimized too much for single instance! (choosing demand requests take precedence)

61. (Some) Future Work 61 Piece selection for better diversity Improved profiling DC-specific optimizations Current work orders of magnitude better than state-of-art

62. Demo (video omitted for space) 62

63. See Paper for More Details Modeling –Playback process dynamics –Buffering (for prefetch) –Full characterization of r incorporating impact of centralized and distributed models on W –Other elided details Plus –More architectural discussion! –Lots more experimental results! 63

64. Summary Scalable VM launching needed VMTorrent addresses by –Decoupling data presentation from streaming –Profile-based VM image streaming Straightforward techniques, implementation, no special optimizations for DC Performance much better than state-of-art –Hardware evaluation on multiple testbeds –As predicted by modeling 64