1. The Only Constant is Change: Incorporating Time-Varying Bandwidth Reservations in Data Centers Di Xie, Ning Ding, Y. Charlie Hu, Ramana Kompella 1

2. Cloud Computing is Hot 2 Private Cluster

3. Key Factors for Cloud Viability Cost Performance 3

4. Performance Variability in Cloud BW variation in cloud due to contention [Schad10 VLDB] Causing unpredictable performance 4

5. Reserving BW in Data Centers SecondNet [Guo10] – Per VM-pair, per VM access bandwidth reservation Oktopus [Ballani11] – Virtual Cluster (VC) – Virtual Oversubscribed Cluster (VOC) 5

6. How BW Reservation Works 6... Virtual Cluster Model Time Bandwidth N VMs Virtual Switch 1. Determine the model 2. Allocate and enforce the model 0T B Only fixed-BW reservation Request

7. Network Usage for MapReduce Jobs Hadoop Sort, 4GB per VM Hadoop Word Count, 2GB per VM Hive Join, 6GB per VM Hive Aggregation, 2GB per VM 7 Time-varying network usage

8. Motivating Example 4 machines, 2 VMs/machine, non-oversubscribed network Hadoop Sort – N: 4 VMs – B: 500Mbps/VM 1Gbps 500Mbps Not enough BW 8

9. Motivating Example 4 machines, 2 VMs/machine, non-oversubscribed network Hadoop Sort – N: 4 VMs – B: 500Mbps/VM 9 1Gbps 500Mbps

10. Under Fixed-BW Reservation Model 10 1Gbps 500Mbps Job3 Job2 Virtual Cluster Model Job1 Time 0 5 10 15 20 25 30 500 Bandwidth

11. Under Time-Varying Reservation Model 11 1Gbps 500Mbps TIVC Model Job1 Time 0 5 10 15 20 25 30 500 Job2 Job3 Job4 Job5 J1J2J3J4J5 Bandwidth Doubling VM, network utilization and the job throughput Hadoop Sort

12. Temporally-Interleaved Virtual Cluster (TIVC) Key idea: Time-Varying BW Reservations Compared to fixed-BW reservation – Improves utilization of data center Better network utilization Better VM utilization – Increases cloud providers revenue – Reduces cloud users cost – Without sacrificing job performance 12

13. Challenges in Realizing TIVC 13... Virtual Cluster Model Time Bandwidth N VMs Virtual Switch 0T B Request Time Bandwidth 0T B Request Q1: What are right model functions? Q2: How to automatically derive the models? Q1: What are right model functions? Q2: How to automatically derive the models?

14. Challenges in Realizing TIVC 14 Q3: How to efficiently allocate TIVC? Q4: How to enforce TIVC? Q3: How to efficiently allocate TIVC? Q4: How to enforce TIVC?

15. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 15

16. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 16

17. How to Model Time-Varying BW? 17 Hadoop Hive Join

18. TIVC Models 18 Virtual Cluster T 11 T 32

19. Hadoop Sort 19

20. Hadoop Word Count 20 v

21. Hadoop Hive Join 21

22. Hadoop Hive Aggregation 22

23. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 23

24. Possible Approach White-box approach – Given source code and data of cloud application, analyze quantitative networking requirement – Very difficult in practice Observation: Many jobs are repeated many times – E.g., 40% jobs are recurring in Bings production data center [Agarwal12] – Of course, data itself may change across runs, but size remains about the same 24

25. Our Approach Solution: Black-box profiling based approach 1.Collect traffic trace from profiling run 2.Derive TIVC model from traffic trace Profiling: Same configuration as production runs – Same number of VMs – Same input data size per VM – Same job/VM configuration 25 How much BW should we give to the application?

26. Impact of BW Capping 26 No-elongation BW threshold

27. Choosing BW Cap Tradeoff between performance and cost – Cap > threshold: same performance, costs more – Cap < threshold: lower performance, may cost less Our Approach: Expose tradeoff to user 1.Profile under different BW caps 2.Expose run times and cost to user 3.User picks the appropriate BW cap 27 Only below threshold ones

28. From Profiling to Model Generation Collect traffic trace from each VM – Instantaneous throughput of 10ms bin Generate models for individual VMs Combine to obtain overall jobs TIVC model – Simplify allocation by working with one model – Does not lose efficiency since per-VM models are roughly similar for MapReduce-like applications 28

29. Generate Model for Individual VM 1.Choose B b 2.Periods where B > B b, set to B cap 29 BW Time B cap BbBb

30. Maximal Efficiency Model Enumerate B b to find the maximal efficiency model 30 BW Time B cap BbBb

31. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 31

32. TIVC Allocation Algorithm Spatio-temporal allocation algorithm – Extends VC allocation algorithm to time dimension – Employs dynamic programming Properties – Locality aware – Efficient and scalable 99 th percentile 28ms on a 64,000-VM data center in scheduling 5,000 jobs 32

33. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 33

34. Enforcing TIVC Reservation Possible to enforce completely in hypervisor – Does not have control over upper level links – Requires online rate monitoring and feedback – Increases hypervisor overhead and complexity Observation: Few jobs share a link simultaneously – Most small jobs will fit into a rack – Only a few large jobs cross the core – In our simulations, < 26 jobs share a link in 64,000-VM data center 34

35. Enforcing TIVC Reservation Enforcing BW reservation in switches – Avoid complexity in hypervisors – Can be implemented on commodity switches Cisco Nexus 7000 supports 16k policers 35

36. Challenges in Realizing TIVC What are the right model functions? How to automatically derive the models? How to efficiently allocate TIVC? How to enforce TIVC? 36

37. Proteus: Implementing TIVC Models 37 1. Determine the model 2. Allocate and enforce the model

38. Evaluation Large-scale simulation – Performance – Cost – Allocation algorithm Prototype implementation – Small-scale testbed 38

39. Simulation Setup 3-level tree topology – 16,000 Hosts x 4 VMs – 4:1 oversubscription Workload – N: exponential distribution around mean 49 – B(t): derive from real Hadoop apps 39 50Gbps 10Gbps … … … 1Gbps … 20 Aggr Switch 20 ToR Switch 40 Hosts ………

40. Batched Jobs Scenario: 5,000 time-insensitive jobs 40 42%21%23%35% 1/3 of each type Completion time reduction All rest results are for mixed

41. Varying Oversubscription and Job Size 41 25.8% reduction for non-oversubscribed network

42. Dynamically Arriving Jobs Scenario: Accommodate users requests in shared data center – 5,000 jobs, Poisson arrival, varying load 42 Rejected: VC: 9.5% TIVC: 3.4% Rejected: VC: 9.5% TIVC: 3.4%

43. Analysis: Higher Concurrency Under 80% load 43 7% higher job concurrency 28% higher VM utilization Rejected jobs are large 28% higher revenue Charge VMs VM

44. Tenant Cost and Provider Revenue Charging model – VM time T and reserved BW volume B – Cost = N (k v T + k b B) – k v = 0.004$/hr, k b = 0.00016$/GB 44 12% less cost for tenants Providers make more money Amazon target utilization

45. Testbed Experiment Setup – 18 machines – Tc and NetFPGA rate limiter Real MapReduce jobs Procedure – Offline profiling – Online reservation 45

46. Testbed Result 46 TIVC finishes job faster than VC, Baseline finishes the fastest TIVC finishes job faster than VC, Baseline finishes the fastest Baseline suffers elongation, TIVC achieves similar performance as VC Baseline suffers elongation, TIVC achieves similar performance as VC

47. Conclusion Network reservations in cloud are important – Previous work proposed fixed-BW reservations – However, cloud apps exhibit time-varying BW usage We propose TIVC abstraction – Provides time-varying network reservations – Uses simple pulse functions – Automatically generates model – Efficiently allocates and enforces reservations Proteus shows TIVC benefits both cloud provider and users significantly 47

48. Backup slides 48

49. Adding Cushions to Model 49 Without cushion With 60s cushion

50. Network Utilization VC reserves 26.4% abs. more bandwidth But less actual utilization (8.9% vs. 20.1%) 50

51. BW Variability on Cloud 51 [Ballani11]

52. Model Refinement Can we further reduced BW for low efficiency pulses without elongation? – This allows us potentially fit more jobs 52 Hadoop Hive Join

53. Model Refinement (cont.) If efficiency of a pulse < γ lower the cap so that efficiency = α γ = 8%, α = 20% 53