Cost- and Energy-Aware Load Distribution Across Data Centers Presented by Shameem Ahmed Kien Le, Ricardo Bianchini, Margaret Martonosi, and Thu D. Nguyen (HotPower 2009) Rutgers University and Princeton University

2

Motivations  Large org has multiple Data Centers (DC)  Business distribution  High availability  Disaster tolerance  Uniform access times to widely distributed client sites  Problems  Consumes lots of energy  Financial and environmental cost  How can we exploit the geographical distribution of DCs for optimizing energy consumption? 1. Different & variable electricity prices (hourly pricing) 2. Exploit DCs in different time zones (peak/off-peak demand price) 3. Exploit DCs located near sites that produce “green” electricity 2

Assumptions  Multi-DC Internet services (e.g. Google, iTunes)  DCs are behind a set of front-end devices  Each service has single SLA (Service Level Agreement) with customers  SLA (L,P) = At least P% req must complete in <L time  Req can be served by 2 or 3 mirror DC  Further replication increases state-consistency traffic But no meaningful benefit in availability or performance 4 Is it True?

Contributions  Framework for optimization-based request distribution policy  What % of client req should be directed to each DC  Front-ends periodically solve optimization problem  After % computation, front-ends abide by them until they are recomputed  A greedy heuristic policy for comparison  Same goal and constraints  First exploits DC with best power efficiency  Then exploits DC with cheapest electricity 5

Prior Research  Energy management on a single data center  A. Qureshi. HotNets 2008  Shut down entire data centers when electricity costs are relatively high  K. Le et al. Middleware 2007  Did not address energy issues, time zones, or heuristics 6

Request Distribution Policies 7

Principles and Guidelines  Only minimizing energy cost is not enough  Must also guarantee high performance and availability  Respect these requirements by having the front-ends:  Prevent DC overloads  Monitor response time of DCs and adjust req distribution accordingly  Each DC reconfigures itself by  Leaving as many servers active as necessary + 20% slack for unexpected load increase  Other servers are turned off 8

“base” energy cost (servers are idle) energy cost of processing the client req Optimization Based Distribution (1/4)  Problem Formulation  Policy EPrice: Leveraging time zones & variable electricity prices Doesn’t distinguish DCs based on energy source SymbolMeaning f i (t)% req to be forwarded to DC i LT(t)Expected total # of req Cost i (t)Avg cost ($) of a req at DC i BCost i (offered i, t)Base energy cost ($) of DC i under offered i load LR(t)Expected Peak service rate (req/s) offered i LR(t) * f i (t) LCiLoad Capacity (req/s) of DC i CDF i Expected % of req that complete within L time, given offered i load

Optimization Based Distribution (2/4)  Problem Formulation  Policy GreenDC: Leveraging DCs powered by green energy 10 energy cost of processing the client req“base” energy cost that is spent when active servers are idle Assumptions: 1.DCs will increasingly be located near green energy source 2.Green energy supply may not be enough to power DC entire period; Need backup (regular electricity) Assumptions: 1.DCs will increasingly be located near green energy source 2.Green energy supply may not be enough to power DC entire period; Need backup (regular electricity)

 Instantiating parameters  Typical approach: front ends communicate & coordinate  Proposed approach:  Each front end independently solves optimization problem  LT(t), LR(t), and offered i are defined for each front-end  Load capacity (LC) of each DC is divided by # of front-ends  CDF i instantiation  CDF i = Expected % of req that complete within L time  Each Front end  Collects recent history of response time of DCi  Maintains a table of for each DC  Similar table for BCost i : SymbolMeaning f i (t)% req to be forwarded to DC I Cost i (t)Avg cost ($) of a req at DC I BCost i (offered i, t)Base energy cost ($) of DC i under offered i load LC i Load Capacity (req/s) of DC i LT(t)Expected total # of req LR(t)Expected Peak service rate (req/s) offered i LR(t) * f i (t) CDF i Expected % of req that complete within L time, given offered i load Optimization Based Distribution (3/4) 11 Does this approach satisfy the constraints globally?

 Solving Optimization Problem  Electricity price prediction: Ameren  Load intensity prediction: ARMA  CDFi prediction: Current CDFi tables  Can’t use LP solvers  Solving for entire day at once involves non-linear functions (e.g. BCosti, CDFi)  Use Simulated Annealing  Divide the day into six 4-hour epochs  Solution recomputation (e.g. data center becomes unavailable) Optimization Based Distribution (4/4) 12

Heuristic-Based Request Distribution (1/2)  Cost-aware but simple  For each epoch (1 hr), each front-end computes R = P x E  P = % of req must complete within L time (SLA)  E = # of req front-end expects in next epoch (use ARMA)  R = # of req that must complete within L time  Each front-end orders DCs that have CDF i (L, LC i )>= P according tofrom lowest to highest ratio  Remaining DCs are ordered by same ratio  Concatenate two lists of DC to create final list (MainOrder) 13

Heuristic-Based Request Distribution (2/2)  Request forward policy  Req are forwarded to first DC in MainOrder until its capacity is met  New req is forwarded to next DC on the list and so on  After front-end has served R req within L time, it can disregard MainOrder and start forwarding req to cheapest DC until capacity is met  What will happen if prediction is inaccurate?  Adjusts R for next epoch 14

Optimization-based vs Heuristics-based CharacteristicsOptimization (EPrice and GreenDC) CA-heuristic Accounting Period1 day Epoch length4 hrs1 hr Load PredictionsPer front-end for entire dayPer front-end for next hour Energy price predictions Entire dayNext hr Recomputation decision Epoch boundary Comm w/ DCsYes 15

Evaluation 16

Methodology (1/4)  Implemented a simulator for large Internet service  Simulate only a single front-end (East US)  Front-end distributes req to 3 DC 17 Data CenterBrown energy (cents/KWh) Green Energy (cents/KWh) Capacity (reqs/s) DC1 (West US) (solar)125 DC2 (East US) DC3 (Europe) (wind)125

Methodology (2/4)  Request Trace  Day-long trace received by Ask.com  Trace doesn’t include response time  To generate realistic DC response times:  Installed a simple service on 3 PlanetLab machines  Req are made from a machine at Rutgers (front-end)  Assumption: avg processing time of each req = 200 ms  How to mimic effect of load intensity and network congestion  5% increase in response time for each 25% increase in load 18 ARMA

Methodology (3/4)  Electricity Prices, Sources, and time zones  Three price scheme  Constant rate, two rate (on/off peak), hourly prices  How to mimic different brown electricity prices for each DC?  Shift default prices 3 hrs earlier or 6 hrs later  Assumptions  Electricity price for Green DC is constant  Green energy at each green site is enough to process 25% req 19 Ameren

Methodology (4/4)  Other parameters  Assumptions  A req consumes 60 J to process by 2 machines (including cooling, conversion, and delivery overheads)  SLA requires 90% of req to complete in 700 ms  Cost-unaware distribution policy  Used for comparison basis  Approach: similar to CA-heuristic  Orders DCs according to performance [CDF i (L, LC i )] from highest to lowest  Req are forwarded to first DC until its capacity is met  New req are forwarded to next DC and so on 20

 Effect of cost-awareness and pricing scheme (brown electricity)  No cost for base energy Result (1/4) 21 (1)Both cost-aware policies reduce costs even under constant pricing (2)On/Off and Dynamic schemes reduce cost significantly (3)EPrice always achieves lowest cost EPrice: Optimization-based distribution (No green energy) CA-Heuristic: Cost-Aware Heuristic (consider Costi/CDFi) CU-Heuristic: Cost-Unaware Heuristic (consider CDFi) EPrice: Optimization-based distribution (No green energy) CA-Heuristic: Cost-Aware Heuristic (consider Costi/CDFi) CU-Heuristic: Cost-Unaware Heuristic (consider CDFi)

Result (2/4) 22 Why does EPrice behave better than CA-Heuristic? Data CenterBrown energy (cents/KWh)Capacity (req/s) DC1 (West US) DC2 (East US) DC3 (Europe) Can Compensate DC3’s poor performance during future periods of low load.

 Effect of green DC  Considers only dynamic pricing  Results are normalized against EPrice results w/ dynamic pricing  No cost for base energy Result (3/4) 23 Brown energy consumption is reduced by 35% by using green DC (3% cost increase) Why do heuristic policies have higher cost than GreenDC?

Result (4/4)  Effect of Base Energy  Assumption:  Server consumes power even when idle  No DC consumes green energy 24 Base energy Cost savings

Conclusion  Optimization framework for request distribution in multi-DC  To reduce energy consumption and cost  To respect SLAs  Policies take advantage of time zones, variable electricity prices, and green energy  Propose a heuristic for achieving the same goal  Evaluation using a day-long trace from a commercial service 25

Discussions  Only used 1 Front-end in experiment  More front-ends will satisfy global constraints?  How to ensure end-to-end QoS guarantee  Can we combine SLA guarantee with QoS requirement provided by clients?  How to handle services with session state  Soft state: Only lasts a user’s session with the service  All req of a session must be sent to same DC  Can we apply the similar concept for multi-cloud structure?  Optimize power  Optimize monetary cost for online service provider  In multi-cloud computing, is it good to assume that data will be available in clouds beforehand?  Pros and Cons 26