Carbon-Aware Energy Capacity Planning for Datacenters Chuangang Ren, Di Wang, Bhuvan Urgaonkar, Anand Sivasubramaniam Computer Science and Engineering The Pennsylvania State University Aug. 9 th, POWER

Energy Consumption of Datacenters Increase in number/size of datacenters due to heavy reliance on Internet services Datacenters, if treated as a country, fifth in the world for electricity use Datacenter electricity usage expected to double in next 5 years and requires 12 new power plants Environmental Cost Environmental Cost Economic Cost Economic Cost 2

Electricity Cost of Datacenters Sources: Koomey 2011 and IDC 2009 Year Electricity Cost for Datacenters (billion US $) 3

Monthly Costs of a 10MW Datacenter 4 4 Servers Power Infrastructure Utility Bill Other 8% 30.5% 24% 37.5% All cost are amortized at a monthly granularity Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2)

Monthly Costs of a 10 MW Datacenter Servers Power Infrastructure Utility Bill Other 5 8% 30.5% 24% 37.5% All cost are amortized at a monthly granularity Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2) Power draw (W) Energy consumption (area under this curve) Month Peak power draw

Environmental Cost of Datacenters CO 2 Emission by Country in 2007 (megatons CO 2 per year) Datacenter 2007 Datacenter Sources: Mckinsey on Business Technology 2008 Trend: Cap the carbon footprint of large electricity consumers (including datacenters) Regulations: Emission Capping and Trading Schemes (e.g. cap-and trade) Carbon Tax Penalties if one fails to comply with the carbon regulations 6

Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Ways to Reduce Datacenter CO2 Emissions 7

Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Renewable energy penetration – Oklahoma Wind to Power Google Data Center – Facebook’s solar-powered datacenter Ways to Reduce Datacenter CO2 Emissions 8

Poor Renewable Usage by Datacenters Today 9 Sources: Greenpeace International 2012

Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Renewable energy penetration – Oklahoma Wind to Power Google Data Center – On-site solar based renewable generation at Google’s headquarter Ways to Reduce Datacenter CO2 Emissions 10 Smarter electricity sourcing strategies

Datacenter Power Sourcing Datacenter Utility Energy On-site Off-site Grid 11

Datacenter Power Sourcing Datacenter Diesel Generator Utility Energy On-site Off-site Grid 12

Datacenter Power Sourcing Datacenter Diesel Generator Utility Energy On-site Off-site Grid 13

On-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 14 Grid + Negligible transmission and distribution loss + Power peak shaving + Tolerate grid outage - Real-estate concerns may limit its efficacy - Not necessarily the best location with the right renewable energy potential (low kWh/sq)

Off-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 15 Grid Off-site Renewable $ $ Wheeling Fee $ $ Banking Fee

Off-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 16 Grid Off-site Renewable $ $ Wheeling Fee $ $ Banking Fee + Good renewable energy potential (higher kWh/sq) + Grid reduces intermittency of renewable energy -Transmission losses -Wheeling and banking charges

Renewable Energy Products Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 17 Grid Off-site Renewable Energy Products

Renewable Energy Products Power Purchase Agreement (PPA): – A long-term contract to buy electricity output from a renewable energy provider, e.g., Google, Microsoft. – Giving renewable power project developers the access to financing they need to build new projects that contribute to new green power to the grid 18

Renewable Energy Products Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 19 Grid Off-site Renewable Energy Products Carbon Offsetting Market

A REC is a certificate that represents 1 MWh of electricity generated from renewable energy Two products: commodity electricity and REC – REC can be sold or traded separately from commodity electricity Renewable Energy Certificate (REC) + REC 20

Datacenter Power Sourcing Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products Off-site Renewable 21 On-site Renewable + Eliminate the need of capex and opex investment + Immunity to the renewable intermittency -Expensive

Question Given all renewable energy options, how to achieve a target carbon footprint at minimal cost? 22

Talk Outline 1 1 Motivation 2 2 Renewable Power Sources 3 3 Framework for Provisioning 4 4 Evaluation 5 5 Conclusions 23

Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 24 Power (W) Time t=10min

Peak Power Impact on Utility Bill 25 5 c/KWh Power draw (W) Energy consumption (area under this curve) Month Duke Utility Tariffs (12 $/KW, 5 c/KWh) 15-min Average draw Peak power draw Peak to Average ratio 3:1 12 $/KW Note: Tariff rates collected from Duke Energy Utility.

Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 26 Power (W) Time t=10min

Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 27 Power (W) Time t=10min

Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 28 Power (W) Time t=10min

Minimize: UtilityBill + OnSiteCost + OffSiteCost + MarketCost + ESD Cost + DGOpEx Optimization Problem: Objective 29

Optimization Problem: Constraints Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 30

Optimization Problem: Constraints On-site Off-site On-site Off-site On-site Constraints 31 Power (W) Time t=10min

Optimization Problem: Constraints On-site Off-site On-site Off-site On-site Constraints 32

Optimization Problem: Constraints Grid Carbon Offsetting Market On-site Off-site Renewable Energy Products 33

Optimization Problem: Constraints Carbon Constraint 34

Talk Outline 1 1 Motivation 2 2 Renewable Power Sources 3 3 Framework for Provisioning 4 4 Evaluation 5 5 Conclusions 35

Realistic Power Profiles 36

Realistic Power Profiles 37

Renewable Trace Profiles Source: NREL, Western Wind and Solar Integration Study 38 On-site Off-site On/Off-site

Datacenter Configuration: Baseline Datacenter Utility Energy On-site Off-site Grid 39

Datacenter Configuration: Base-DG Datacenter Diesel Generator Utility Energy On-site Off-site Grid 40

Datacenter Configuration: Base-DG-ESD Datacenter Diesel Generator Utility Energy On-site Off-site Grid 41 ESD

Datacenter Configuration: All Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site On-site Renewable Energy Products Off-site Renewable 42

Datacenter Cost Optimization Facebook Datacenter Power Cost (No Carbon Requirement) Total Cost per Day ($) 43 On-site renewables help reduce peak power drawn, hence lower cost On-site renewables can supplement ESD and DG in their role in peak reduction at a lower cost

Impact of Carbon Target Facebook Datacenter Power Cost Total Cost per Day ($) Carbon Reduction Goal (%) add baseline here $ 44

Impact of Carbon Target (cont.) 45 On-site only: Base-DG-ESD + on-site renewable generation Off-site only: Base-DG-ESD + off-site renewable generation Market only: Base-DG-ESD + REC/PPA Carbon Reduction Goal (%) Total Cost per Day ($)

Impact of Carbon Target (cont.) 46 On-site only: Base-DG-ESD + on-site renewable generation Off-site only: Base-DG-ESD + off-site renewable generation Market only: Base-DG-ESD + REC/PPA Hybrid solution as high carbon reduction increases Renewable penetration goes beyond carbon offsetting, even lower cost

Concluding Remarks Framework for energy capacity planning to achieve carbon footprints target at minimal cost Key findings: – Renewable penetration lower carbon footprints and costs – On-site renewables helpful for peak power reduction – Hybrid solution depending on carbon footprint targets 47

Penn State University Thank you! 48

Backup Slides 49

Electricity Cost of Datacenters Sources: Koomey 2011 and IDC 2009 Year Power and Cooling Cost for Datacenters (billion US $) Servers Power Infrastructure Utility Bill Other 8% 30.5% 24% 37.5% Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2) 50

Renewable Energy Renewable energy provides 16% of global energy consumption Wind energy grew by 11% per year Solar energy grew by 28% between 2000 and 2010 Sources: REN21 Renewable Energy in Data Center Wind: Google, Microsoft … Solar: Facebook, Apple … 51

Datacenter Power Sourcing Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site On-site Renewable Energy Products Off-site Renewable Typical Datacenter Power Sourcing 52

On-site Renewable Energy Power Plant Utility Power Grid $ $ POWER Datacenter POWER Off-siteOn-site  No transmission and distribution loss  Helpful for power peak shaving  Power supply intermittency  Not the right location for renewable energy  Other constraints, e.g. real estate No Wind 53

Off-siteOn-site  Good renewable energy potentials  Wheeling and Banking effectively eliminate the intermittency of renewable energy  Wheeling charge is sometimes expensive, depending on the location of off-site facility (e.g. avg. 15$/MWh for wind) Off-site Renewable Energy Power Plant Utility Power Grid $ $ POWER Datacenter POWER No Wind Good Wind Beautiful sunlight POWER $ $ Wheeling Fee 54

Renewable Energy Certificate (REC) Third-party Renewable Energy Producer Power Plant Utility REC Supplier Power Grid Green Datacenter $ $ $ $ POWER REC Datacenter 55

Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 56

Impact of Carbon Target Facebook Datacenter Power Cost 57 Carbon Reduction Goal (%) Total Cost per Day ($)

Impact of Renewable Capacity Factor 58 Capacity factor plays a crucial role in selecting renewable sources/locations On-site becomes less cost-effective than off-site when CF is less than 24%

Impact of Renewable Energy Price 59 Wheeling charge plays a crucial role in selecting renewable sources Off-site wind penetration can go beyond carbon reduction, even lower costs if CF and wheeling charge is friendly

Wind and Solar Sources Most prominent sources of renewable energy today – Wind (google, microsoft, etc): 62%; Solar (Facebook, Apple, etc): 13% Highly time-varying power output – Wind speed, solar irradiance Lower Capacity factor than conventional power plants – Wind: 20-45%; Solar photovoltaic: 14-24% Much lower carbon emission – Occur during manufacturing, transportation, installation, recycling 60

ESD for peak shaving 61 Time Power consumption (W) Peak Energy Storage Device (ESD) Power Cap New draw Original draw