1. GREEN COMPUTING CS 595 LECTURE 12 4/1/2015

2. Number one concern of large scale data centers? Efficiency affects: accessibility reliability sustainability costs Can we affect datacenter efficiency with software? - yes!

3. OUTLINE Power consumption in large datacenters Alternative energy sources Green hardware design best practices Server level hardware DC/DC, AC/DC conversions Rack power distribution UPS Cooling Building power distribution Green software design best practices Resource virtualization Storage Computational

4. POWER CONSUMPTION IN LARGE DATACENTERS United States: 91 Billion kWh (2013) Datacenter power usage effectiveness (PUE): ESIS – Consumption for supporting infrastructure EITS – Consumption for IT power substations ETX – Building Transformer losses EHV – High voltage cable losses ELV – Low voltage cable losses EF – Consumption from fuel (natural gas, oil) ECRAC – CRAC consumption EUPS – Loss at UPS Enet1 – Network room energy from type 1 unit substitution PUE? For every (PUE)Watt consumed at the datacenter, 1 Watt is delivered to the IT load

5. 2014 Global Datacenter average PUE: 1.7 Amazon: 1.45 Microsoft: 1.13 Google: 1.1 Yahoo: 1.07 Facebook: 1.07

6. POWER CONSUMPTION IN LARGE DATACENTERS Datacenter energy breakdown: Computing: 52% CPU – 15% Server Power supply – 14% Other Server Components – 15% Storage – 4% Networking – 4% Support Systems: 48% Cooling – 38% UPS – 5% Building Power Distribution/Transformer – 3% PDU – 1% Lighting – 1%

7. POWER CONSUMPTION IN LARGE DATACENTERS Average and maximum power densities: 10 years ago: 100W – 175W per sq ft. Modern data center: Average: 225W Maximum: 400W Previous slide: 5,000 sq ft. datacenter 2mW of energy to dissipate! Is this a large datacenter? 2011: Clemson University completes its new datacenter: 50,000sq ft. 25,000sq ft. -office space 25,000sq ft. - rack space 4.5mW at full capacity (current)

8. ALTERNATIVE ENERGY SOURCES SOLAR Solar Constant: amount of incoming solar electromagnetic radiation ~1.36 kW/m 2 (maximum) Average solar panel efficiency: 14% 19w/ft 2 (best condition) Closer to 10w/ft 2 5,000 sq ft. datacenter: 2mW  200,000 sq ft of solar panels! (4.59 acres) lol? Semprius (solar startup): 33.9% 1.89 acres Energy storage? Datacenter placement? Cloudy?

9. ALTERNATIVE ENERGY SOURCES WIND Advantages: Generate 250W – 1.8MW per turbine Green, renewable energy Disadvantages: Unpredictable Noise Aesthetics

10. ALTERNATIVE ENERGY SOURCES GEOTHERMAL Cooling, Cooling, Cooling! Approach 1: Climate cooling Strategic placement of datacenter that allows outside air temperature to help regulate inside temperature. Approach 2: Ground cooling Average temperature at 20ft below earth’s surface: 54F Datacenters can use closed loop venting ducts underground to dissipate the heat produced by equipment.

11. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - CPU CPUs with lower TDP: Intel Ivy Bridge Xeon: 22nm manufacturing process E3-1280 – 4 core w/HT (8 threads) Max TDP: 69W E5-2600 V2 – 10 core w/HT (20 threads) Max TDP: 130 AMD Piledriver Opteron: 32nm manufacturing process 6366 HE – 16 core Max TDP: 85W 6386 SE – 16 core Max TDP: 140W Goal: Increase Flops/Watt

12. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - CPU Performance per watt: rate of computation that can be delivered by a CPU for every Watt of power consumed. www.green500.org: www.green500.org #1: 5.27 GFlops/W #1 - #10 average: 3.91 GFlops/W #500: 22.11 MFlops/W Is this green? Google Nexus 6: ~100 MFlops/W CPU + GPU + Android System

13. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - STORAGE HDD vs SSD: WD Caviar Blue 500 GB SATA 6GB/s Read/Write: 6.8W Idle: 6.1W Read/Write performance: 130MB/s Seagate Cheetah 15k 146GB SAS Read/Write: 17.2W Idle: 14.4W Read/Write performance: 186MB/s OCZ Vertex 3 240GB SATA 6GB/s Read/Write: 1.25W Idle: 0.53W Read/Write performance: >500MB/s Cost per GB? HDD: ~$0.05/GB SSD: ~$0.47/GB

14. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - RAM DDR2 vs DDR3 DDR2 – 1.8v 1066 MT/s 80nm manufacturing process DDR3 – 1.25v – 1.65v 2133 MT/s 20nm (Samsung, 2012) DDR4 (September 2012): 1v – 1.2v Up to 4266 MT/s (megatransfers/sec) 20nm current manufacturing process

15. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - MOTHERBOARD 85% of power consumed by motherboard: Voltage Regulators Converts voltage provided by power supply into appropriate voltages needed by the CPU, RAM, Chipsets, etc. Also used to “clean up” DC voltage from the power supply. Linear voltage regulator: Solid state semiconductor to regulate voltages ~60% efficient Switching voltage regulator: Circuit that transitions between full regulated out, low-current output, and off Generates less heat  ~95% efficient Use motherboard with switching voltage regulators for higher efficiency.

16. GREEN HARDWARE DESIGN BEST PRACTICES SERVER LEVEL HARDWARE - NETWORK Use of power efficient networking components: Switches Routers NICs Devices can be scaled depending on demand Device low power mode Turn off ports Goal: Increase Bandwidth/Watt in communication components

17. GREEN HARDWARE DESIGN BEST PRACTICES AC/DC, DC/DC CONVERSIONS AC/DC conversions: Translates non-polarized alternating current (building power feed) to polarized direct current (used by IT components) Not 100% efficient Heat generated equates to loss Also must be dissipated requiring more power consumption! AC/DC conversion locations: Building feed to UPS Server Power supply EPA Energy Star rating – 80% efficiency across all rated power output. Switching voltage regulator (previous slides)

18. GREEN HARDWARE DESIGN BEST PRACTICES RACK POWER DISTRIBUTION Problem with current rack deployment? Rack mount servers connected to UPS Power transforms at UPS and again at the server Solution (Google): Remove unneeded AC/DC, DC/AC transformations AC from building line feed connects to UPS in the rack UPS performs AC/DC transformation Servers connect directly to UPS’s DC output

19. GREEN HARDWARE DESIGN BEST PRACTICES UPS Uninterruptible Power Supply: Stores power to ensure IT component operations in the case of short power outages Also used to “clean” electricity from outside source Efficiency: 92% - 95% Arranged in a 2N configuration for fault tolerance. Example: 20,000sq ft. datacenter at 120w/sq ft. Electricity cost: $0.10/kWh 95% efficient vs 92% efficient UPS $72,000 annual difference in electricity costs Generated $72,000 worth of heat!!

20. GREEN COMPUTING CS 595 LECTURE 12 4/1/2015

21. GREEN COMPUTING Current system extremely wasteful Need energy to power Need energy to cool 1000 racks, 25,000 sq ft, 10MW for computing, 5MW to dissipate heat Need a system more efficient, less expensive strategy with immediate impact on energy consumption

22. DATA CENTERS Focus by green computing movement on data centers (SUVs of the tech world) 6,000 data centers in US Cost: $4.5 B (more than used by all color TVs in US) In 2007, DOE reported data centers 1.5% of all electricity in US Greenhouse gas emission projected to more than double from 2007 to 2020

23. DATA CENTERS Within a few years, cost of power for data center was expected to exceed cost of original capital investment

24. GOAL Fed. Gov. wanted data center energy consumption to be reduced by at least 10% Same as energy consumed by 1M average US households

27. DATA CENTER METRICS Metrics SPECPowerjbb benchmark and DCiE from Green Grid Green Grid – group of IT professionals Power Usage Effectiveness PUE PUE = Total facility power/IT equipment power Data Center infrastructure Efficiency metric DCiE 1/PUE

28. FUTURE VISION Sources of computing power in remote server warehouses Located near renewable energy sources – wind, solar Usage shifts across globe depending on where energy most abundant

29. CURRENT APPROACHES Some “low hanging fruit” approaches Orient racks of servers to exhaust in a uniform direction Higher fruit - Microsoft Built near hydroelectric power in WA Built in Ireland - can air cool, 50% more energy efficient Countries with favorable climates: Canada, Finland, Sweden and Switzerland

30. CURRENT APPROACHES Google – trying to reduce carbon footprint Carbon footprint includes direct fuel use, purchased electricity and business travel, employee commuting, construction, server manufacturing According to Google, its data centers use ½ industry’s average amount of powerGoogle How? Ultra efficient evaporative cooling (customized) Yahoo (are they back??) Data centers also carbon-neutral because of use of carbon offsets

31. CURRENT APPROACHES US government EPA has phase-one of Energy Star standards for servers Measure server power supply efficiency and energy consumption while idle Must also measure energy use at peak demand Green Grid consortium Dell, IBM, Sun VM-Wear AMD Green500 – 500 most green supercomputers

32. DATA CENTER PRODUCT SPECIFICATION COMPLETION 2009 Servers v1.0 2011 Data Center Buildings Program 2012 UPS v1.0 (uninterruptable power supply) 2013 Servers 2.0 Storage v1.0 2014 Large Network Equipment v1.0 2015 Data Center Cooling Equipment v1.0

33. CURRENT APPROACHES Replace old computers with new more energy-efficient But manufacturing through day-to-day uses energy Dell - reducing hazardous substances in computers, OptiPlex 50% more energy efficient Greenest computer company – VirtualBoxImages What is “Greenest computer ever” ? Is MacBook air (pro) greenest? https://www.apple.com/macbook-pro/environment/

34. GOALS FOR FUTURE 1. Consider energy to manufacture, operate, dispose of wastes 2. Sense and optimize world around us 3. Predict and respond to future events by modeling behavior (grown in performance) 4. Benefit of digital alternative to physical activities E-newspapers, online shopping Personal energy meter??

35. THE CASE FOR ENERGY- PROPORTIONAL COMPUTING BARROSO AND HOLZLE (GOOGLE)

36. INTRO Energy proportional computing primary design goal for servers Cooling and provisioning proportional to average energy servers consume Energy efficiency benefits all components Computer energy consumption lowered if: Adopt high-efficiency power supplies Use power saving features already in equipment

37. INTRO More efficient CPUs on chips based on multiprocessing has helped But, higher performance means increased energy usage

38. SERVERS Servers Rarely completely idle Seldom operate at maximum 10-50% of max utilization levels 100% utilization not acceptable for meeting throughput, etc. – no slack time

40. SERVERS Completely idle server waste of capital Difficult to idle subset of servers Servers need to be available Perform background tasks Move data around Help with crash recovery Applications can be restructured to create idle intervals Difficult, hard to maintain Devices with highest energy savings, highest wake-up penalty, e.g disk spin up

41. ENERGY EFFICIENCY AT VARYING UTILIZATION LEVELS Utilization – measure of performance normalized to performance at peak loads Energy efficient server still consumes ½ power when doing almost no work Power efficiency – utilization/power value Peak energy efficiency occurs at peak utilization and drops as util. decreases At 20-30% utilization, efficiency drops to less than ½ at peak performance

43. TOWARD ENERGY-PROPORTIONAL MACHINES Mismatch between servers’ high-energy efficiency characteristics and behavior Designers need to address this Design machines that consume energy in proportion to amount of work performed No power when idle (easy) Nearly no power when little work (harder) More as activity increases (even harder)

44. CPU POWER Fraction of total server power consumed by CPU changed since 2005 CPU no longer dominates power at peak usage, trend will continue Even less when idle Processors close to energy-proportional Consume < 1/3 power at low activity (70% of peak) Power range less for other components < 50% for DRAM, 25% for disk drives, 15% for network switches

45. DYNAMIC RANGE Processors can run at lower voltage frequency mode without impacting performance No other components with such modes Only inactive modes in DRAM and disks Inactive to active mode transition penalty (even it only idle to submilliseconds) Servers with 90% dynamic range could cut energy by ½ in data centers Lower peak power by 30% Energy proportional hardware reduce need for power management software

47. DISKS - INACTIVE/ACTIVE Penalty for transition to active from inactive state makes it less useful Disk penalty 1000 higher for spin up than regular access latency Only useful if idle for several minutes (rarely occurs) More beneficial to have smaller penalty even if higher energy levels Active energy savings schemes are useful even if higher penalty to transition because in low energy mode for longer periods

48. CONCLUSIONS CPUS already exhibit energy proportional profiles, other components less so Need significant improvements in memory and disk subsystems Such systems responsible for larger fraction of energy usage Need energy efficient benchmark developers to report measurements at nonpeak levels for complete picture

49. GREEN INTROSPECTION BY K. CAMERON

50. HISTORY OF GREEN In the 1970s Energy crisis High gas prices Fuel shortages Pollution Education and action Environmental activism Energy awareness and conservation Technological innovation

51. GIFTS FROM THE 70S Energy crisis subsided In the meantime advances in computing responsible for: Innovation for energy-efficient buildings and cars Identified causes and effects of global climate change Grassroots activism, distributing info about energy consumption, carbon emission, etc. The same computing technologies pioneered by hippie geeks (???) are the problem now

52. WHAT HAPPENED NEXT Call to action within IT community (what about the 80s??) In 1990s General-purpose microprocessors built for performance Competing processors ever-increasing clock rates and transistor densities fast processing power and exponentially increasing power consumption Power wall at 130 watts Power is a design constraint

53. BETTER, BUT ALSO WORSE? To reduce power consumption Multicore architectures – higher performance, lower power budgets But Users expect performance doubling every 2 years Developers must harness parallelism of multicore architectures Power problems ubiquitous – energy-aware design needed at all levels

54. MORE PROBLEMS Memory architectures consume significant amounts of power Need energy-aware design at systems level Disks, boards, fans switches, peripherals Maintain quality of computing devices, decrease environmental footprint Can’t rely on nonrenewable resources or toxic ingredients

55. THOSE DATA CENTERS IT helping in data centers Reducing energy with virtualization and consolidation Need to address chip level device to heating/cooling of building Need metrics

56. TRADE-OFF How often to replace aging systems? 2% of solid waste comes from consumer electronic components E-waste fastest growing component of waste stream In US 130,000 computers thrown away daily and 100 M cell phones annually Recycle e-waste (good luck) Use computers as long as possible?