1. Carbon-Aware Energy Capacity Planning for Datacenters Chuangang Ren, Di Wang, Bhuvan Urgaonkar, Anand Sivasubramaniam Computer Science and Engineering The Pennsylvania State University Aug. 9 th, 2012 1 POWER

2. Energy Consumption of Datacenters Increase in number/size of datacenters due to heavy reliance on Internet services Datacenters, if treated as a country, fifth in the world for electricity use Datacenter electricity usage expected to double in next 5 years and requires 12 new power plants Environmental Cost Environmental Cost Economic Cost Economic Cost 2

3. Electricity Cost of Datacenters Sources: Koomey 2011 and IDC 2009 Year Electricity Cost for Datacenters (billion US $) 3

4. Monthly Costs of a 10MW Datacenter 4 4 Servers Power Infrastructure Utility Bill Other 8% 30.5% 24% 37.5% All cost are amortized at a monthly granularity Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2)

5. Monthly Costs of a 10 MW Datacenter Servers Power Infrastructure Utility Bill Other 5 8% 30.5% 24% 37.5% All cost are amortized at a monthly granularity Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2) Power draw (W) Energy consumption (area under this curve) Month Peak power draw

6. Environmental Cost of Datacenters CO 2 Emission by Country in 2007 (megatons CO 2 per year) Datacenter 2007 Datacenter 2020 142 146 178 80 340 Sources: Mckinsey on Business Technology 2008 Trend: Cap the carbon footprint of large electricity consumers (including datacenters) Regulations: Emission Capping and Trading Schemes (e.g. cap-and trade) Carbon Tax Penalties if one fails to comply with the carbon regulations 6

7. Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Ways to Reduce Datacenter CO2 Emissions 7

8. Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Renewable energy penetration – Oklahoma Wind to Power Google Data Center – Facebook’s solar-powered datacenter Ways to Reduce Datacenter CO2 Emissions 8

9. Poor Renewable Usage by Datacenters Today 9 Sources: Greenpeace International 2012

10. Power demand reduction – Numerous endeavors on energy-proportional computing technologies – Smart cooling system control Renewable energy penetration – Oklahoma Wind to Power Google Data Center – On-site solar based renewable generation at Google’s headquarter Ways to Reduce Datacenter CO2 Emissions 10 Smarter electricity sourcing strategies

11. Datacenter Power Sourcing Datacenter Utility Energy On-site Off-site Grid 11

12. Datacenter Power Sourcing Datacenter Diesel Generator Utility Energy On-site Off-site Grid 12

13. Datacenter Power Sourcing Datacenter Diesel Generator Utility Energy On-site Off-site Grid 13

14. On-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 14 Grid + Negligible transmission and distribution loss + Power peak shaving + Tolerate grid outage - Real-estate concerns may limit its efficacy - Not necessarily the best location with the right renewable energy potential (low kWh/sq)

15. Off-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 15 Grid Off-site Renewable $ $ Wheeling Fee $ $ Banking Fee

16. Off-site Renewable Generation Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 16 Grid Off-site Renewable $ $ Wheeling Fee $ $ Banking Fee + Good renewable energy potential (higher kWh/sq) + Grid reduces intermittency of renewable energy -Transmission losses -Wheeling and banking charges

17. Renewable Energy Products Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 17 Grid Off-site Renewable Energy Products

18. Renewable Energy Products Power Purchase Agreement (PPA): – A long-term contract to buy electricity output from a renewable energy provider, e.g., Google, Microsoft. – Giving renewable power project developers the access to financing they need to build new projects that contribute to new green power to the grid 18

19. Renewable Energy Products Datacenter Diesel Generator Utility Energy On-site Off-site On-site Renewable 19 Grid Off-site Renewable Energy Products Carbon Offsetting Market

20. A REC is a certificate that represents 1 MWh of electricity generated from renewable energy Two products: commodity electricity and REC – REC can be sold or traded separately from commodity electricity Renewable Energy Certificate (REC) + REC 20

21. Datacenter Power Sourcing Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products Off-site Renewable 21 On-site Renewable + Eliminate the need of capex and opex investment + Immunity to the renewable intermittency -Expensive

22. Question Given all renewable energy options, how to achieve a target carbon footprint at minimal cost? 22

23. Talk Outline 1 1 Motivation 2 2 Renewable Power Sources 3 3 Framework for Provisioning 4 4 Evaluation 5 5 Conclusions 23

24. Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 24 Power (W) Time t=10min

25. Peak Power Impact on Utility Bill 25 5 c/KWh Power draw (W) Energy consumption (area under this curve) Month Duke Utility Tariffs (12 $/KW, 5 c/KWh) 15-min Average draw Peak power draw Peak to Average ratio 3:1 12 $/KW Note: Tariff rates collected from Duke Energy Utility.

26. Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 26 Power (W) Time t=10min

27. Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 27 Power (W) Time t=10min

28. Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 28 Power (W) Time t=10min

29. Minimize: UtilityBill + OnSiteCost + OffSiteCost + MarketCost + ESD Cost + DGOpEx Optimization Problem: Objective 29

30. Optimization Problem: Constraints Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 30

31. Optimization Problem: Constraints On-site Off-site On-site Off-site On-site Constraints 31 Power (W) Time t=10min

32. Optimization Problem: Constraints On-site Off-site On-site Off-site On-site Constraints 32

33. Optimization Problem: Constraints Grid Carbon Offsetting Market On-site Off-site Renewable Energy Products 33

34. Optimization Problem: Constraints Carbon Constraint 34

35. Talk Outline 1 1 Motivation 2 2 Renewable Power Sources 3 3 Framework for Provisioning 4 4 Evaluation 5 5 Conclusions 35

36. Realistic Power Profiles 36

37. Realistic Power Profiles 37

38. Renewable Trace Profiles Source: NREL, Western Wind and Solar Integration Study 38 On-site Off-site On/Off-site

39. Datacenter Configuration: Baseline Datacenter Utility Energy On-site Off-site Grid 39

40. Datacenter Configuration: Base-DG Datacenter Diesel Generator Utility Energy On-site Off-site Grid 40

41. Datacenter Configuration: Base-DG-ESD Datacenter Diesel Generator Utility Energy On-site Off-site Grid 41 ESD

42. Datacenter Configuration: All Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site On-site Renewable Energy Products Off-site Renewable 42

43. Datacenter Cost Optimization 3761 3700 3678 3669 Facebook Datacenter Power Cost (No Carbon Requirement) Total Cost per Day ($) 43 On-site renewables help reduce peak power drawn, hence lower cost On-site renewables can supplement ESD and DG in their role in peak reduction at a lower cost

44. Impact of Carbon Target Facebook Datacenter Power Cost Total Cost per Day ($) Carbon Reduction Goal (%) add baseline here 3761 3669 3682 37003717 3841 $ 44

45. Impact of Carbon Target (cont.) 45 On-site only: Base-DG-ESD + on-site renewable generation Off-site only: Base-DG-ESD + off-site renewable generation Market only: Base-DG-ESD + REC/PPA 1 1 2 2 3 3 Carbon Reduction Goal (%) Total Cost per Day ($)

46. Impact of Carbon Target (cont.) 46 On-site only: Base-DG-ESD + on-site renewable generation Off-site only: Base-DG-ESD + off-site renewable generation Market only: Base-DG-ESD + REC/PPA 1 1 2 2 3 3 Hybrid solution as high carbon reduction increases Renewable penetration goes beyond carbon offsetting, even lower cost

47. Concluding Remarks Framework for energy capacity planning to achieve carbon footprints target at minimal cost Key findings: – Renewable penetration lower carbon footprints and costs – On-site renewables helpful for peak power reduction – Hybrid solution depending on carbon footprint targets 47

48. Penn State University Thank you! 48 http://csl.cse.psu.edu/

49. Backup Slides 49

50. Electricity Cost of Datacenters Sources: Koomey 2011 and IDC 2009 Year Power and Cooling Cost for Datacenters (billion US $) Servers Power Infrastructure Utility Bill Other 8% 30.5% 24% 37.5% Chart: Source: Book by Barroso et al., Assumption: 20,000 servers, 1.5 PUE, 15$/W Cap-ex, Duke Energy Op-ex, 4yr server & 12 yr infrastructure amortization (Tier-2) 50

51. Renewable Energy Renewable energy provides 16% of global energy consumption Wind energy grew by 11% per year Solar energy grew by 28% between 2000 and 2010 Sources: REN21 Renewable Energy in Data Center Wind: Google, Microsoft … Solar: Facebook, Apple … 51

52. Datacenter Power Sourcing Grid Datacenter Diesel Generator Utility Energy Carbon Offsetting Market On-site Off-site On-site Renewable Energy Products Off-site Renewable Typical Datacenter Power Sourcing 52

53. On-site Renewable Energy Power Plant Utility Power Grid $ $ POWER Datacenter POWER Off-siteOn-site  No transmission and distribution loss  Helpful for power peak shaving  Power supply intermittency  Not the right location for renewable energy  Other constraints, e.g. real estate No Wind 53

54. Off-siteOn-site  Good renewable energy potentials  Wheeling and Banking effectively eliminate the intermittency of renewable energy  Wheeling charge is sometimes expensive, depending on the location of off-site facility (e.g. avg. 15$/MWh for wind) Off-site Renewable Energy Power Plant Utility Power Grid $ $ POWER Datacenter POWER No Wind Good Wind Beautiful sunlight POWER $ $ Wheeling Fee 54

55. Renewable Energy Certificate (REC) Third-party Renewable Energy Producer Power Plant Utility REC Supplier Power Grid Green Datacenter $ $ $ $ POWER REC Datacenter 55

56. Methodology Grid Utility Energy Carbon Offsetting Market On-site Off-site Renewable Energy Products 56

57. Impact of Carbon Target Facebook Datacenter Power Cost 57 Carbon Reduction Goal (%) Total Cost per Day ($)

58. Impact of Renewable Capacity Factor 58 Capacity factor plays a crucial role in selecting renewable sources/locations On-site becomes less cost-effective than off-site when CF is less than 24%

59. Impact of Renewable Energy Price 59 Wheeling charge plays a crucial role in selecting renewable sources Off-site wind penetration can go beyond carbon reduction, even lower costs if CF and wheeling charge is friendly

60. Wind and Solar Sources Most prominent sources of renewable energy today – Wind (google, microsoft, etc): 62%; Solar (Facebook, Apple, etc): 13% Highly time-varying power output – Wind speed, solar irradiance Lower Capacity factor than conventional power plants – Wind: 20-45%; Solar photovoltaic: 14-24% Much lower carbon emission – Occur during manufacturing, transportation, installation, recycling 60

61. ESD for peak shaving 61 Time Power consumption (W) Peak Energy Storage Device (ESD) Power Cap New draw Original draw