1. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)1 Chapter x: Green Datacenter Infrastructures in the Cloud Computing Era 1 Sergio Ricciardi, 2 Francesco Palmieri, 3 Jordi Torres-Viñals, 2 Beniamino Di Martino, 1 Germán Santos-Boada, 1 Josep Solé-Pareta 1 Technical University of Catalonia, Spain 2 Second University of Naples 3 Barcelona Supercomputing Center (BSC), HANDBOOK ON GREEN INFORMATION AND COMMUNICATION SYSTEMS

2. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)2 Introduction ICT energy consumption 7% worldwide produced electrical energy (ICT industry has the same energy demand of the aviation industry) [2] Demand Source: 20% from manufacturing, 80% equipment use [3] ICT energy consumption growth rate will double in 2020

3. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)3 Introduction Humans activities have severe impacts on the environment Resource exploitation: Energy-consumption Pollution: GHG emissions, global warming & climate changes Human ecological footprint measures the humanitys demand on the biosphere 1,4 planet Earths [2006] Carbon footprint Measures the total set of GHG emissions Three dimensions Energy consumption (Wh) GHG emissions (kg CO 2 ) Cost () Source: [1]

4. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)4 Data Centers energy consumption Standard computer servers can consume between 1,200 to 8,600 kWh annually. Annual source energy use of a 2MW data center is equal to the amount of energy consumed by 4,600 typical cars in one year. = 4,600 typical cars 1 million vehicles A single 2MW data center All the US data center

5. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)5 BSC MareNostrum Power consumption: 1.2 MW ~ 1,200 houses 1.100.000 /year ~ 10000 Servers It is not the most powerful supercomputer in the world, but it is the most beautiful (Fortune, Sept. 2006)

6. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)6 Data center energy use doubled to more than 120 billion kWh from 2006 to 2011, equal to annual electricity costs of $7.4 billion. Data Centers energy costs

7. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)7 Where energy goes? Cooling 34% Server/Storage 47% Conversion 7% Network 10% Lighting 2% Energy distribution within the DC l The ICT Vicious cycle Watt Heat Cooling l Power Usage Effectiveness (PUE): ~ 2 [38]

8. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)8 Improving energy efficiency in distributed Data Centers can: What we can gain? If we do nothing to change our data center consumption, 10 more power plants need to be built (over the next four years) to the tune of $2 billion to $6 billion each and their cost is ultimately going to get passed on to IT through increased utility bills. -Ken Brill, Forbes Magazine Reduce business risk Become more socially responsible Lower utility bills

9. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)9 What can we do? Global warming: great challenge sensibilize People sensibilize Governments sensibilize Industries sensibilize Energy Providers sensibilize Academia sensibilize Internet Service Providers Avoid wastes not increasing the offer but decreasing the demand Develop energy-efficient architectures, energy-aware algorithms & protocols, use renewable energy

10. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)10 Replace physical servers with virtual servers that allow consolidation and resource sharing Use thin clients, mobile phones or other low energy devices Transfer network presence to a proxy and use wake on LAN Not bringing the electrical power to data centers (power losses) but moving the data centers to the source of the green power and connect them with long reach fiber optic cables (ICT industry is the only business sector that has this inherent capability) attenuation(light) < impedance(electric) Virtualization & decentralization

11. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)11 Consolidation Example 200 Server Virtualization $49,000/yr Dollar Savings Energy Savings 980,000 kWh/yr Physical servers Virtual server 25 Storage Source: BC Hydro

12. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)12 Follow-the-sun/wind/… in complex clouds 12

13. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)13 Much of the time our systems are idle but on What we seek is the ability to do nothing well… 13 Sleep mode: doing nothing well Source: [40]

14. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)14 14 Sleep mode: doing nothing well Sources: [41][42] Consumption is driven by on-times, not by usage PC savings potential is most of current consumption

15. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)15 15 Sleep mode Generic devices Load balancing Time consuming Start-up & configuration problem + peak in power usage Lifetime (MTBF) Economic CAPEX & OPEX Per-interface sleep mode / Adaptive rate / Low Power Idle [39] / STOP-START Energy proportional computing / Downclocking Grid sites / data centers / Clouds Modular structure with hierarchical devices CE WN 1 WN n... DPM SE 1 SE m...

16. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)16 The Facilities Power on procedure executed on SE 1

17. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)17 The Facilities Power off procedure executed on SE 1

18. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)18 Performance Analysis Results Time t 1 : core 1core 2core 3core 4 server 1 server 2 server 3 server 4 x server 5 x server 6 x Time t 2 : 9 incoming jobs: core 1core 2core 3core 4 server 1 server 2 server 3 server 4 x server 5 x server 6 x => first-fit uses up to 2x more servers than best-fit In multicore servers job aggregation is possible: Best-fit vs First-fit, Workload scheduler: 1 job 1 core Time t 2 : 9 incoming jobs: core 1core 2core 3core 4 server 1 xxxx <= +4 first-fit server 2 xxxx <= +4 first-fit server 3 x <= +1 first-fit server 4 x server 5 x server 6 x Time t 2 : 9 incoming jobs: core 1core 2core 3core 4 server 1 xxxx <= +4 first-fit server 2 xxxx <= +4 first-fit server 3 x <= +1 first-fit server 4 xxxx <= +3 best-fit server 5 xxxx <= +3 best-fit server 6 xxxx <= +3 best-fit

19. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)19 19 Energy-aware data center control plane Capacity-demand mismatch leads to resource and energy wastes [8] Traffic fluctuation s Overprovisionin g IDEA: exploit traffic fluctuations to aggregate jobs on a subset of servers and turn-off the idle ones

20. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)20 20 Theoretical provisioning elasticity concept Safety Margin d Ideal case IDEA: exploit traffic fluctuations to aggregate jobs on a subset of servers and turn-off the idle ones Energy-aware data center control plane

21. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)21 21 Service-demand matching algorithm Theoretical energy savings upper-bound:Actual energy saving: Real case Energy-aware data center control plane

22. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)22 22 Energy-aware data center control plane Server energy model: the power consumption varies linearly with the CPU load. Probability Density Functions (PDFs) and Cumulative Distribution Functions (CDFs) of jobs duration

23. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)23 23 Energy-aware data center control plane Energy consumption of day one with and without the service-demand matching algorithm.

24. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)24 24 Energy-aware data center control plane Energy consumption of day n with and without the service-demand matching Algorithm and queued jobs that have to wait due to a peak in the traffic load.

25. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)25 Performance Analysis Results Energy, CO 2 emissions and Costs with varying d values (large data center)

26. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)26 Conclusions Farms usually over-provisioned + Fluctuations in the traffic load Job aggregation and sleep mode to save Energy, CO2 and service-demand matching algorithm job aggregation capabilities respects both the demand requirements and the logical and physical dependencies Resource allocation efficiency : 20% ~ 68% Significant energy, cost and emissions savings

27. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)27 References 1.BONE project, 2009, WP 21 Topical Project Green Optical Networks: Report on year 1 and updated plan for activities, NoE, FP7-ICT-2007-1 216863 BONE project, Dec. 2009. 2.An inefficient Truth by the Global Action Plan, http://www.globalactionplan.org.uk/upload/resource/Full- report.pdf. 3.SMART 2020: Enabling the low carbon economy in the information age, The climate group, 2008. 4.The Green Grid, The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE, Technical Committee White Paper, 2008. 5.Jordi Torres, Green Computing: the next wave in computing, Ed. UPCommons, Technical University of Catalonia (UPC). February 2010. Ref. http://hdl.handle.net/2099.3/33669. 6.Sergio Ricciardi, Alessandra Doria, Gianpaolo Carlino, Salvatore Iengo, Leonardo Merola, Maria Carla Staffa, Powerfarm: a power and emergency management thread-based software tool for the ATLAS Napoli Tier2, proceedings of Computing in High Energy Phisics (CHEP) 21 - 27 March 2009, Prague, Czech Republic, Journal of Physics: Conference Series (JPCS), IOP Publishing 7.Sergio Ricciardi, Davide Careglio, Ugo Fiore, Francesco Palmieri, Germán Santos-Boada, Josep Solé- Pareta, "Saving Energy in Data Center Infrastructures", submitted to e-Energy 2011, New York, U.S., 21/1/2011. 8.B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma, S. Banerjee, N. McKeown, Elastictree: Saving energy in data center networks, in Proceedings of the 7th USENIX Symposium on Networked System Design and Implementation (NSDI), pages 249--264. ACM, 2010. 9. L.A. Barroso, L. A., Hölzle, U., The Case for Energy-Proportional Computing, IEEE Computer, vol. 40, 33- 37, 2007.

28. © Technical University of Catalonia - Second University of Naples - Barcelona Supercomputing Center (BSC)28 Thanks for your attention!