Emerging applications in cloud High performance computing E-Commerce Media hosting Web hosting Content delivery... –from Amazon AWS survey 1 Emulated network experiments –Cloudlab, Roar [SIGCOMM09] You name it …

VM Virtualization in cloud data centers 2 Virtualization Infrastructure VM Virtualization for flexible and cost-effective sharing Cloud: a large scale virtualized environment e.g. Amazon EC2, GoGrid

This work First measurement study on the network performance of commercial cloud –Amazon EC2 from user’s perspective Understand the impact of virtualization on network performance 3

Why do I care? Critical to cloud service applications. Different from traditional clusters or enterprise networks. –Virtualization has impacts, but how? Insights for both users and service providers. 4

Background: Amazon EC2 Leasing VMs at per instance hour rate –Use Xen for virtualization 5 Different types of VM instances Small (default): 1 EC2 compute unit, 1.7GB memory and 160 GB storage, EC2 compute unit: “equivalent to a GHz 2007 Opteron or Xeon processor” Large, Extra Large, High-CPU Medium, High-CPU Extra Large etc. 4 to 20 compute unit, much more memory and storage

Background: Xen virtualization 6 Physical Machine Hardware IO & platform device (disk, LAN, USB …) CPU & memory Xen Hypervisor Domain 1 Virtual driver User Software Guest OS Domain 0 Physical driver Guest OS Control Software Domain 2 Virtual driver User Software Guest OS virtual I/O path direct I/O path Potential processor sharing I/O sharing through Domain0

Measurement methodology Metrics –Processor share –End-to-end delay –TCP/UDP throughput Instance types –Small instances (default) –High-CPU medium instances (for comparison purpose) Experiments –Spatial: 750 pairs small, 150 pairs medium instances covering 177 subnets in EC2 us-east clouds –Temporal: 6 pairs small, 3 pairs medium instances continuously run for 1 week. 7

Processor sharing Inferring processor sharing 8 main() { timestamps=malloc(MAX_LOOP*sizeof(double)); for(i = 0; i < MAX_LOOP; i++) { gettimeofday(&now, NULL); timestamps[i] = now.tv_sec+(double)now.tv_usec/ ; } //dump timestamps; } main() { timestamps=malloc(MAX_LOOP*sizeof(double)); for(i = 0; i < MAX_LOOP; i++) { gettimeofday(&now, NULL); timestamps[i] = now.tv_sec+(double)now.tv_usec/ ; } //dump timestamps; } A simple cputest.c Running cputest as the only user process to estimate processor sharing based on timestamps trace.

Small instance Processor sharing Timestamp trace demo 9 CPU share trace plot Non-virtualized computer: AMD Dual Core Opteron GHz Medium instance Non-virtualized EC2 Medium Instance: Intel Xeon Dual Core 2.4 GHz

Processor sharing Cumulative distribution 10 CPU share distribution Restricted processor sharing on small instances Small instances Medium instances

TCP/UDP throughput Cumulative distribution 11 (1)TCP/UDP throughput distribution in spatial experiment ~200Mbps gap Small instances Medium instances

(1) Small Instance TCP/UDP throughput 12 Unstable bandwidth on small instances due to processor sharing Fine-grained throughput (2) Medium Instance TCP UDP

End-to-end delay Measuring RTT using 5000 probes 13 Abnormal delay variations on both small and medium instances. (1)Distribution of delay metrics (2) Raw RTT measurement Non-virtualized Medium instance Small instance Min Median Max StdDev

Potential reasons of large delay variations 14 Xen VM2 VM1 P P Sender ReceiverDomain 0 VM2 VM3 P P Processor sharing I/O sharing

Implications of unstable network Example: Packet loss estimation using Badabing –Estimate loss episode using one way delay [SIGCOMM05] 15 network delay = propagation delay + queuing delay Queue capacity Max OWD Loss episode One-way delay (OWD) Time

Badabing packet loss estimation Badabing one way delay trace 16 Skewed results: Badabing reports >10% packet loss frequency! Badabing one way delay results

Implications How to balance the resource sharing and performance interference in cloud? When the world become virtual… –how to adapt and migrate applications to cloud? –how to measure and diagnose the virtual network? –how to adjust protocol design for good performance? 17