1. Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University http://virtuoso.cs.northwestern.edu

2. 2 Summary Dynamically adapt existing, unmodified applications running on unmodified operating systems in virtual environments to available resources Adaptation mechanisms are application independent and controlled automatically without user or developer help Demonstrate feasibility of adaptation at the level of collection of VMs connected by Virtual Networks Show that its benefits can be significant for two classes of applications

3. 3 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET, VTTIFAdaptive virtual network Evaluation Summary

4. 4 Aim Grid Computing New Paradigm Traditional Paradigm Deliver arbitrary amounts of computational power to perform distributed and parallel computations Problem1: Grid Computing using virtual machines Problem2: Solution How to leverage them? Virtual Machines What are they? 6b 6a 5 4 3b 3a 2 1 Resource multiplexing using OS level mechanism Complexity from resource user’s perspective Complexity from resource owner’s perspective Virtual Machine Grid Computing

5. 5 Virtual Machines Virtual machine monitors (VMMs) Raw machine is the abstraction VM represented by a single image VMware GSX Server

6. 6 The Simplified Virtuoso Model Orders a raw machine User Specific hardware and performance Basic software installation available User’s LAN VM Virtual networking ties the machine back to user’s home network Virtuoso continuously monitors and adapts

7. 7 User’s View in Virtuoso Model User User’s LAN VM

8. 8 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET, VTTIFAdaptive virtual network Evaluation Summary

9. 9 User’s friendly LAN Foreign hostile LAN Virtual Machine VNET: A bridge with long wires Host Proxy X Virtual Networks VM traffic going out on foreign LAN IP network A machine is suddenly plugged into a foreign network. What happens? Does it get an IP address? Is it a routeable address? Does firewall let its traffic through? To any port?

10. 10 Host vmnet0 Ethernet Packet Tunneled over TCP/SSL Connection Ethernet Packet Captured by Interface in Promiscuous mode “Host Only” Network Ethernet Packet is Matched against the Forwarding Table on that VNET First linkSecond link (to proxy) Local traffic matrix inferred by VTTIF Periodically sent to the VNET on the Proxy VNET ethz VM 2 “eth0” VNET ethy IP Network VM 1 “eth0” vmnet0 A VNET Link

11. 11 Virtual Topology and Traffic Inference Framework (VTTIF) Operation Application topology is recovered using normalization and pruning algorithms Ethernet-level traffic monitoring VNET daemons collectively aggregate a global traffic matrix for all VMs

12. 12 Dynamic Topology Inference by VTTIF 1. Fast updates Smoothed Traffic Matrix 2. Low Pass Filter Aggregation 3. Threshold change detection Topology change output VNET Daemons on Hosts VNET Daemon at Proxy Aggregated Traffic Matrix

13. 13 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET, VTTIFAdaptive virtual network Evaluation Summary

14. 14 Monitoring and inference Application performance measure Adaptation algorithm Adaptation mechanisms Adaptation Applications Optimization metric 1.Overlay topology 2.Forwarding rules 3.VM migration 1.Single hop 2.Worst fit 1.BSP 2.Transactional ecommerce 1.Application throughput 1.VTTIF 2.Network monitoring 1.Single metric 2.Combined metric

15. 15 Optimization Problem (1/2) Topology Only Informally stated: Input –Network traffic load matrix of application Output –Overlay topology connecting hosts –Forwarding rules on the topology  Such that the application throughput is maximized The algorithm is described in detail in the paper

16. 16 Foreign host LAN 1 User’s LAN Host 2 + VNET Proxy + VNET IP network Host 3 + VNET Host 4 + VNET Host 1 + VNET Foreign host LAN 3 Foreign host LAN 4 Foreign host LAN 2 VM 1 VM 4 VM 3 VM 2 Resilient Star Backbone Merged matrix as inferred by VTTIF Illustration of Topology Adaptation in Virtuoso Fast-path links amongst the VNETs hosting VMs

17. 17 Evaluation Reaction time of VNET Patterns: A synthetic BSP benchmark Benefits of adaptation (performance speedup) –Eight VMs on a single cluster, all-all topology –Eight VMs spread over WAN, all-all topology CMU VM 7 University of Chicago VM 8 Northwestern VM 1 DOT Network VM 6 VM 5 … Wide-Area testbed Proxy

18. 18 Reaction Time

19. 19 Benefits of Adaptation Benefits accrued as a function of the number of fast-path links added Patterns has an all-all topology Eight VMs are used All VMs are hosted on the same cluster

20. 20 Patterns has an all-all topology Eight VMs are used VMs are spread over WAN Benefits of Adaptation Benefits accrued as a function of the number of fast-path links added

21. 21 Informally stated: Input –Network traffic load matrix of application –Topology of the network Output –Mapping of VMs to hosts –Overlay topology connecting hosts –Forwarding rules on the topology  Such that the application throughput is maximized Optimization Problem (2/2) Topology + Migration The algorithm is described in detail in the paper

22. 22 Evaluation Applications –Patterns: A synthetic BSP benchmark –TPC-W: Transactional web ecommerce benchmark Benefits of adaptation (performance speedup) –Adapting to compute/communicate ratio –Adapting to external load imbalance

23. 23 Effect on BSP Application Throughput of Adapting to Compute/Communicate Ratio

24. 24 Effect on BSP Application Throughput of Adapting to External Load Imbalance

25. 25 TPCW Throughput (WIPS) With Image Server Facing External Load No TopologyTopology No Migration1.2161.76 Migration1.42.52

26. 26 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET, VTTIFAdaptive virtual network Evaluation Summary

27. 27 Summary Dynamically adapt existing, unmodified applications running on unmodified operating systems in virtual environments to available resources Adaptation mechanisms are application independent and controlled automatically without user or developer help Demonstrate feasibility of adaptation at the level of collection of VMs connected by Virtual Networks Show that its benefits can be significant for two classes of applications

28. 28 Future Work –Free network measurement (Wren) – Collaboration with CS, W&M –Applicability of a single optimization scheme Related Talk at HPDC 2005 –J. Lange, A. Sundararaj, P. Dinda, “Automatic Dynamic Run-time Optical Network Reservations” –Wednesday, July 27, 2:00 P.M. Please visit –Prescience Lab (Northwestern University) http://plab.cs.northwestern.edu –Virtuoso: Resource Management and Prediction for Distributed Computing using Virtual Machines http://virtuoso.cs.northwestern.edu VNET is publicly available from above URL For More Information

29. 29 Backup slides start from here…

30. 30 Isn’t It Going to Be Too Slow? ApplicationResourceExecTime (10^3 s) Overhead SpecHPC Seismic (serial, medium) Physical16.4N/A VM, local16.6 1.2% VM, Grid virtual FS 16.8 2.0% SpecHPC Climate (serial, medium) Physical9.31N/A VM, local9.68 4.0% VM, Grid virtual FS 9.70 4.2% Experimental setup: physical: dual Pentium III 933MHz, 512MB memory, RedHat 7.1, 30GB disk; virtual: Vmware Workstation 3.0a, 128MB memory, 2GB virtual disk, RedHat 2.0 NFS-based grid virtual file system between UFL (client) and NWU (server) Small relative virtualization overhead; compute-intensive Relative overheads < 5%

31. 31 Isn’t It Going To Be Too Slow? Synthetic benchmark: exponentially arrivals of compute bound tasks, background load provided by playback of traces from PSC Relative overheads < 10%

32. 32 Isn’t It Going To Be Too Slow? Virtualized NICs have very similar bandwidth, slightly higher latencies –J. Sugerman, G. Venkitachalam, B-H Lim, “Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor”, USENIX 2001 Disk-intensive workloads (kernel build, web service): 30% slowdown –S. King, G. Dunlap, P. Chen, “OS support for Virtual Machines”, USENIX 2003 However: May not scale with faster NIC or disk

33. 33 User’s friendly LAN Foreign hostile LAN Virtual Machine Why VNET? A Scenario IP network User has just bought

34. 34 User’s friendly LAN Foreign hostile LAN Virtual Machine VNET: A bridge with long wires Host Proxy X Why VNET? A Scenario VM traffic going out on foreign LAN IP network A machine is suddenly plugged into a foreign network. What happens? Does it get an IP address? Is it a routeable address? Does firewall let its traffic through? To any port?

35. 35 Host vmnet0 Ethernet Packet Tunneled over TCP/SSL Connection Ethernet Packet Captured by Interface in Promiscuous mode “Host Only” Network Ethernet Packet is Matched against the Forwarding Table on that VNET First linkSecond link (to proxy) Local traffic matrix inferred by VTTIF Periodically sent to the VNET on the Proxy VNET ethz VM “eth0” VNET ethy IP Network VM “eth0” vmnet0 A VNET Link

36. 36 Host vmnet0 Ethernet Packet Tunneled over TCP/SSL Connection Ethernet Packet Captured by Interface in Promiscuous mode “Host Only” Network Ethernet Packet is Matched against the Forwarding Table on that VNET First linkSecond link (to proxy) Local traffic matrix inferred by VTTIF Periodically sent to the VNET on the Proxy VNET ethz VM 2 “eth0” VNET ethy IP Network VM 1 “eth0” vmnet0 A VNET Link

37. 37 User’s LAN Foreign LAN 1 Host 2 + VNET Proxy + VNET VNET startup topology IP network Host 3 + VNET Host 4 + VNET Host 1 + VNET Foreign LAN 3 Foreign LAN 4 Foreign LAN 2 VM 1 VM 4 VM 3 VM 2 TCP Connections

38. 38 VTTIF Traffic characterization and topology inference for applications Ethernet-level traffic monitoring VNET daemons collectively aggregate a global traffic matrix for all VMs Application topology is recovered using normalization and pruning algorithms

39. 39 VTTIF Operation Synced Parallel Traffic Monitoring Traffic Filtering and Matrix Generation Matrix Analysis and Topology Characterization

40. 40 Reaction time of VTTIF

41. 41 Benefits of Adaptation Benefits accrued as a function of the number of fast-path links added Patterns has an all-all topology Eight VMs are used All VMs are hosted on the same cluster

42. 42 Patterns has an all-all topology Eight VMs are used VMs are spread over WAN Benefits of Adaptation Benefits accrued as a function of the number of fast-path links added

43. 43

44. 44

45. 45 Adaptation Algorithms Topology adaptation –Implied traffic intensity between VNET daemons –Links established in order of decreasing traffic intensity –Cost constraint “c” Migration –A worst-fit algorithm Combining algorithms –Migration algorithm is run first –The overlay topology is next determined –Finally the forwarding rules are computed

46. 46 Present and Future Demonstrated the feasibility of adaptation at the level of collection of VMs connected by VNET Showed that its benefits can be significant for two classes of applications Studying the computational complexity of the generic incarnation of adaptation problem Exploring the applicability of a single optimization scheme for a wide-range of distributed applications

47. 47 Summary Dynamically adapt applications in virtual environments to available resources Demonstrate the feasibility of adaptation at the level of collection of VMs connected by Virtual Networks Show that its benefits can be significant for two classes of applications Exploring the applicability of a single adaptation scheme for a wide-range of distributed applications