Peter X. Gao, Andrew R. Curtis, Bernard Wong, S. Keshav Cheriton School of Computer Science University of Waterloo August 15,

2 = CO 2 of 280,000 cars ~1M servers

Datacenters and Request Routing DC 2 DC 1 Dynamic DNS 3

Where to route? DatacenterLatencyElectricity PriceCarbon footprint DC1 (Texas)LowHigh DC2 (Washington)HighLow 4

Where to route? DatacenterLatencyElectricity PriceCarbon footprint DC1 (Texas)LowHigh DC2 (Washington)HighLow 5 DatacenterLatencyElectricity PriceCarbon footprint DC1 (Texas)LowHighLow DC2 (Washington)HighLowHigh A.M. P.M. Electricity carbon footprint in California

How to split? 6 DC 1 DC 2 80% 20%

FORTE and its Contributions FORTE: Flow Optimization based framework for Request- routing and Traffic Engineering Contributions: – Principled framework for managing the three-way trade-off between access latency, electricity cost, and carbon footprint Green datacenter upgrade plans – Impact of carbon taxes on datacenter carbon footprint reduction 7

Surprising Results FORTE can reduce datacenter carbon footprint by 10% with no increase in electricity cost and access latency Carbon Tax is not effective because taxes are only about 5% of electricity price 8

Outline Model P1: Assigning users to datacenters P2: Assigning data objects to datacenters P3: Datacenter upgrade Evaluation 9

Model User Groups: u i Datacenters: n j Data Objects: d k Requests r(u i, d k ) NY LA DC 10 Carbon emission: c(n j ) Electricity price: e(n j ) Capacity: cap(n j )

P3 P2P1 Model User Groups: u i Datacenters: n j Data Objects: d k Requests r(u i, d k ) serves is placed at NY LA DC 11 Access latency: l(u i, n j, d k ) Carbon emission: c(n j ) Electricity price: e(n j ) Capacity: cap(n j )

Latency Cost Function l max latency latency cost: l(u i,n j ) 12

Outline Model P1: Assigning users to datacenters P2: Assigning data objects to datacenters P3: Datacenter upgrade Evaluation 13

P1 User Groups: u i Datacenters: n j Data Objects: d k 14 Assigning Users to Datacenters

Objective Function 15 n3 n1 d2 User Groups: u i Datacenters: n j Data Objects: d k u1 Access latency: l(u i, n j, d k ) n1 Weighted Carbon Cost: λ 1 c(n j ) + Weighted Electricity Cost: λ 2 e(n j ) } + Weighted Latency Cost: λ 0 l(u i, n j, d k ) Minimize: ∑ Carbon emission: c(n j ) Electricity price: e(n j )

16 Demand Satisfaction Constraints n3 n1 d2 User Groups: u i Datacenters: n j Data Objects: d k Requests r(u i, d k ) u1 Access latency: l(u i, n j, d k ) n3 n1 Carbon emission: c(n j ) Electricity price: e(n j )

17 Datacenter Capacity Constraints n4 User Groups: u i Datacenters: n j Data Objects: d k u2 u3 Capacity: cap(n j ) n4

Scale of Linear Program Evaluation problem size: – Over 1 million variables – FORTE can solve it in approximately 2 min Actual problem: – Can be over 1 billion variables 18

Fast-FORTE Greedy Heuristic Running time O(N logN) vs Simplex O(~N 6 ) Reduces running time from 2 minutes to 6 seconds 0.3% approximation error 19

Outline Model P1: Assigning users to datacenters P2: Assigning data objects to datacenters P3: Datacenter upgrade Evaluation 20

P2 Assigning Data Objects to Datacenters 21 User Groups: u i Datacenters: n j Data Objects: d k

Assigning Data Objects to Datacenters User Groups Datacenters Data Objects requests Σ Flow size = 100 Σ Flow size = 1 22 Σ Flow size = 101

Outline Model P1: Assigning users to datacenters P2: Assigning data objects to datacenters P3: Datacenter upgrade Evaluation 23

P3 Using FORTE for upgrading datacenters 24 User Groups: u i Datacenters: n j Data Objects: d k

Using FORTE for upgrading datacenters Datacenter operators need to decide: – Which datacenters should be upgraded? – How many servers in that datacenter should be upgraded? The upgrade decisions are based on: – Estimation of future traffic demands – Annual budget on upgrading – Trade-off between cost and benefit 25

Using FORTE for upgrading datacenters 26 User Groups Datacenters Data Objects requests Can also be used for selecting new datacenter locations by adding zero size datacenters into the network

Outline Model P1: Assigning users to datacenters P2: Assigning data objects to datacenters P3: Datacenter upgrade Evaluation 27

Datasets Akamai traffic data – Akamai delivers about 15% - 20% Internet traffic – 3 weeks coarse-grained data in U.S. – Aggregated every 5 minutes U.S. Energy Information Administration – Carbon footprint – Electricity cost Data Objects: Synthetic with long-tail popularity, 10% latency tolerant 28

Different Level of Carbon Reduction 29 Latency Only Small Reduction Medium Reduction Large Reduction

Three-way Tradeoff Tradeoff between carbon emissions, average distance, and electricity costs. 30

Two-way Tradeoff between Carbon Emission and Electricity Cost 31 (987, 5.83) (1010, 5.73) Electricity Cost ($/hour)

Will Carbon Taxes or Credits Work? Akamai uses ~2 * 10 8 kWh per year Electricity cost of 2 * 10 8 kWh: 2 * 10 8 kWh * 11.2c/kWh = $22.4 M “Carbon cost” of 2 * 10 8 kWh : 2 * 10 8 kWh * 500g/kWh = 10 5 t 10 5 t * $10/t = $1 M 32

Green Upgrades 33 WA1 CA1 CA2 TX1 NY1 NJ1 NJ2 Year1 Year 2 Year 3 Reduces carbon emission by ~25% compare to carbon oblivious plan Use Green Energy Reduce Access Latency Use Green Energy Reduce Access Latency Low Electricity Price

Related Work Qureshi et. al., Cutting the electric bill for internet-scale systems, SIGCOMM 09 Doyle et. al., Server Selection for Carbon Emission Control, GreenNet 11 Other related work can be found in our paper FORTE: – Considers data allocation problem – Supports datacenter upgrade – Explores the three-way trade-off 34

Conclusions FORTE is a request routing framework that can reduce carbon emissions by ~10% without affecting latency and electricity cost Surprisingly, carbon taxes do not provide sufficient incentives to reduce carbon emissions A green upgrade plan can further reduce carbon emissions by ~25% over 3 years 35

Acknowledgement We thank Prof. Bruce Maggs for providing us access to Akamai traces We thank our shepherd Prof. Fabian Bustamante and the reviewers for their insightful comments 36