1. © University of Ottawa, ON, Canada1 Chapter 11: Energy-Efficient Cloud Computing: A green migration of the traditional IT Hussein T. Mouftah and Burak Kantarci School of Electrical Eng. and Computer Science University of Ottawa HANDBOOK ON GREEN INFORMATION AND COMMUNICATION SYSTEMS

2. © University of Ottawa, ON, Canada2 Outline Introduction to Cloud Computing Green ICT and Energy-Efficient Cloud Computing Motivation for Green Data Centers Motivation for Green ICT Energy-Efficient Storage and Processing in Cloud Computing Energy-Efficient Processing in Data Centers Energy-Efficient Storage Monitoring Thermal Activity in Data Centers Optimal Data Center Placement Energy-Efficient Transport of Cloud Services Summary and Challenges

3. © University of Ottawa, ON, Canada3 Introduction to Cloud Computing (1/4) Cloud computing infrastructure of business enterprises Software as a Service (SaaS) e.g., web services, multimedia, etc.e.g., web services, multimedia, etc. Platform as a Service (PaaS) e.g., software framework, storage, etc.e.g., software framework, storage, etc. Infrastructure as a Service (IaaS) e.g., Virtual machinee.g., Virtual machine

4. © University of Ottawa, ON, Canada4 Introduction to Cloud Computing (2/4) Various deployment models exist Public clouds serve through the Internet backbone and operate based on the pay-as-you-go fashion Private clouds are dedicated to an organization where the files and tasks are hosted within the corresponding organization

5. © University of Ottawa, ON, Canada5 Introduction to Cloud Computing (3/4) Various deployment models exist (Cont d ) Community clouds enable several organizations to access a shared pool of cloud services forming a community of a special interest Hybrid clouds are a combination of the public, private and the community clouds with the objective of overcoming the limitations of each model

6. © University of Ottawa, ON, Canada6 Introduction to Cloud Computing (4/4) Challenges in cloud computing Security Privacy Reliability Virtual Machine Migration Automated Service Provisioning Energy Management

7. © University of Ottawa, ON, Canada7 Green ICT and Energy-Efficient Computing (1/3) Motivation for Green ICT ICT consumes 4% of the electricity and expected to be doubled (8%) Telecommunication networks contribute a big portion of the CO 2 emissions of ICTs GHG emission contribution of the telecom networksGHG emission contribution of the telecom networks –PCs and monitors (40%) –Servers (23%) –Fixed Line Telecommunications (15%) –Mobile Telecommunications (9%) –LAN and Office Communications (7%) –Printers (6%)

8. © University of Ottawa, ON, Canada8 Green ICT and Energy-Efficient Computing (2/3) Motivation for Green Data Centers Cloud Computing infrastructure is housed in Data Centers US Data Centers consume 1.7%~2.2% of the total electricity consumed in the country (61 billion kWh in 2006, doubled in 2007) Worldwide data centers consume 1.1%~1.5% of all electricity consumed in the world Proper Power Management in the data centers can lead to significant energy savings Virtualization of computing resourcesVirtualization of computing resources Sleep schedulingSleep scheduling Shared Servers and Storage Units Energy savings possible if users migrate IT services towards remote resources Increase in the network traffic and the associated network energy J. Baliga et al., Green Cloud Computing: Balancing Energy in Processing, Storage, and Transport, Proceedings of the IEEE, vol. 99, issue-1, pp. 149-167, 2011.

9. © University of Ottawa, ON, Canada9 Green ICT and Energy-Efficient Computing (3/3) Motivation for Green Data Centers (Cont d) A key metric to evaluate how green is a data center Power Usage Efficiency (PUE) Data Center Efficiency (DCE) A good DCE is 0.625A good DCE is 0.625 A reasonable DCE target is 0.5A reasonable DCE target is 0.5 C. Belady, The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE,Whitepaper, [Online] http://www.thegreengrid.org/, 2008. Google-Data Center Efficiency, [Online] http://www.google.com/about/datacenters/inside/efficiency/powerusage. html, accessed in Oct. 20011. http://www.google.com/about/datacenters/inside/efficiency/powerusage

10. © University of Ottawa, ON, Canada10 Energy-Efficient Processing in Cloud Computing (1/6) Thermal-Aware Workload Placement Minimum Heat Recirculation (minHR) Objective: Minimizing the heat recirculation (δQ) in the data center with n servers Specific heat of air [Watt-s/Kg K] Air flow rate at server-i [kg/s] Inlet temperature of server-i Temperature supplied by CRAC Distribute the power (P i ) proportional to the ratio of the heat produced (Q i ) to the heat recirculated (δQ i ) at server-i.

11. © University of Ottawa, ON, Canada11 Energy-Efficient Processing in Cloud Computing (2/6) minHR Set a reference heat value (Q ref ) and a reference heat recirculation value (δQ i ) Group of adjacent servers are considered to be a pod, e.g., p pods in the data center Determine the power level in each pod by p iterations Set the power level of servers in pod-j to maximum Calculate Heat Recirculation Factor for pod-j (HRF j ) Calculate power level for pod-j Adjust CRAC supply tepmperature (T adj ) J. Moore et al., Making Scheduling Cool:Temperature-Aware Workload Placement in Data Centers, in Usenix Ann. Technical Conf., 2005, pp. 61–74.

12. © University of Ottawa, ON, Canada12 Energy-Efficient Processing in Cloud Computing (3/6) Cooling and thermal-aware workload placement No cooling and thermal- awareness Cooling and thermal-aware job management

13. © University of Ottawa, ON, Canada13 Energy-Efficient Processing in Cloud Computing (4/6) Cooling and thermal-aware workload placement A. Banerjee et al., Integrating cooling awareness with thermal aware workload placement for HPC data centers, Sustainable Computing: Informatics and Systems, vol. 1(2), pp. 134–150, June. 2011. Temporal Job Scheduling First Come First Serve (FCFS) Earliest Deadline First (EDF) Spatial Job Scheduling Thermal-Aware Job Scheduling Minimum Re-circulated Heat (MRH) Cooling-aware Job Scheduling Highest Thermostat Setting (HTS)

14. © University of Ottawa, ON, Canada14 Energy-Efficient Processing in Cloud Computing (5/6) Highest Thermostat Setting (HTS): A Cooling and Thermal- Aware Workload Placement scheme Temporally schedule the jobs EDF / FCFS Server Ranking According to the requirement of thermostat set temperature to meet the redline for 100% utilization Spatial scheduling Place jobs to the available servers with the lowest rank Obtain power distribution vector P h Set thermostat setting to the highest possible value (T th high ) For details of the determination process see: A. Banerjee et al., Integrating cooling awareness with thermal aware workload placement for HPC data centers, Sustainable Computing: Informatics and Systems, vol. 1(2), pp. 134–150, June. 2011.

15. © University of Ottawa, ON, Canada15 Energy-Efficient Processing in Cloud Computing (6/6) A. Banerjee et al., Integrating cooling awareness with thermal aware workload placement for HPC data centers, Sustainable Computing: Informatics and Systems, vol. 1(2), pp. 134–150, June. 2011. Benefits of coordinated workload scheduling Without turning off idle servers With turning off idle servers

16. © University of Ottawa, ON, Canada16 Energy-Efficient Storage in Cloud Computing (1/2) Solid State Disks (SSDs) A storage device consisting of NAND flash memory and a controller. An alternative to the conventional hard disk drives (HDDs) due to: Being light weight, having small form factor, having no moving mechanical parts and lower power consumption Issues to be addressed Write reliability A single level SSD cell bit introduces write penalty after 100,000 writes Cost / GB $1.80/GB for an SSD while a HDD costs approximately $0.11/GB (as of 2011) Massive Arrays of idle Disks (MAIDs) Large amount of hard disk drives that are used for nearline storage a hard disk drive spins up whenever an access request arrives for the data stored in it and the rest of the storage consists of a large number of spun down disks. trade-off between energy-efficiency and performance spinning up takes more time than data access does.

17. © University of Ottawa, ON, Canada17 Energy-Efficient Storage in Cloud Computing (2/2) Storage Virtualization A logical storage pool which is independent of the physical location of the disks Unused storage segments can be consolidated in logical storage units increasing the storage efficiency. Servers are connected to the physical resources through SAN switches; hence a global storage pool is available to each server. Whenever a storage block is required to be allocated, a logical unit number is assigned to the allocated virtual space D. Barrett and G. Kipper, Virtualization Challenges, in Virtualization and Forensics, pp. 175 – 195. Syngress, Boston, 2010.

18. © University of Ottawa, ON, Canada18 Monitoring Thermal Activity in Data Centers (1/2) Project Genome J. Liu et al., Project Genome: Wireless Sensor Network for Data Center Cooling, The Architecture Journal, Microsoft, vol 18, pp. 28-34, 2008. RACNet: Wireless Sensor Networks (WSNs) in Data Centers Wireless sensor network developed for Microsoft Research Data Center Genome project Provides fine-grained and real- time visibility to data center cooling behaviour ~700 sensors deployed in a MMW data center Hierarchical topology Master and slave sensor nodesMaster and slave sensor nodes Large-scale sensor network Multiple slave sensors for collecting temperature, humidityMultiple slave sensors for collecting temperature, humidity Several master sensors providing connectivitySeveral master sensors providing connectivity Uses IEEE 802.15.4 radios

19. © University of Ottawa, ON, Canada19 Monitoring Thermal Activity in Data Centers (2/2) J. Liu et al., Project Genome: Wireless Sensor Network for Data Center Cooling, The Architecture Journal, Microsoft, vol 18, pp. 28-34, 2008. Project Genome (Cont d) RACNet Challenges High BER of IEEE 802.15.4 radioHigh BER of IEEE 802.15.4 radio Tough RF environment in the Data Center due to high metal contents of servers, racks, cables, railings, etc.Tough RF environment in the Data Center due to high metal contents of servers, racks, cables, railings, etc. High density of wireless nodes in RACNet increases the likelihood of packet collisions.High density of wireless nodes in RACNet increases the likelihood of packet collisions. Solution reliable Data Collection Protocol (rDCP)reliable Data Collection Protocol (rDCP) –Uses Three Key Technologies Channel Diversity: rDCP coordinates among multiple base stations to use 16 concurrent channels in the 2.4 GHz ISM band concurrentlyChannel Diversity: rDCP coordinates among multiple base stations to use 16 concurrent channels in the 2.4 GHz ISM band concurrently Adaptive bidirectional collection tree: On each wireless channel, a collection tree is built to adapt to link quality changes.Adaptive bidirectional collection tree: On each wireless channel, a collection tree is built to adapt to link quality changes. Coordinated data retrieval: Data is polled by the base station. Only one data retrieval stream on an active channel at a given timeCoordinated data retrieval: Data is polled by the base station. Only one data retrieval stream on an active channel at a given time

20. © University of Ottawa, ON, Canada20 Optimal Data Center Placement (1/3) X. Dong, T. El-Gorashi, and J. M. H. Elmirghani, Green IP over WDM with Data Centers, IEEE/OSA Journal of Lightwave Technology, vol. 29/12, pp. 1861–1880, June 2011. Location of data centers has significant impact on the transport energy of cloud services in the Internet backbone. Objectives Minimum power consumption Utilize renewable resources, e.g., solar panels, wind farms, etc. IP-over-WDM network at the backbone Employ optical-bypass Power consuming components IP router ports Transponders EDFAs OXCs (De)multiplexers

21. © University of Ottawa, ON, Canada21 Optimal Data Center Placement (2/3) X. Dong, T. El-Gorashi, and J. M. H. Elmirghani, Green IP over WDM with Data Centers, IEEE/OSA Journal of Lightwave Technology, vol. 29/12, pp. 1861–1880, June 2011. Constraints Constraints of energy-minimized design of IP over WDM network are inherited Additional constraints Renewable energy consumption at the router ports, transponders., cooling and computing equipments in a data center cannot exceed the power supplied by a single wind farm. Solar power that is available to a backbone node sets an upper bound for the renewable energy consumed by the router ports and the transponders of the corresponding node. total renewable power consumption of all data centers is constrained to the total power supply of the wind farms.

22. © University of Ottawa, ON, Canada22 Optimal Data Center Placement (3/3) X. Dong, T. El-Gorashi, and J. M. H. Elmirghani, Green IP over WDM with Data Centers, IEEE/OSA Journal of Lightwave Technology, vol. 29/12, pp. 1861–1880, June 2011. optimal placement of data centers by incorporating non-bypass routing power savings between 4.4% and 12.7% multi-hop bypass routing-based design power savings between 1.7% and 6.3%. multi-hop bypass routing already consumes less power when compared to non-bypass routing Optimal placement is a critical design parameter when number of data centers is limited and/or the backbone topology is irregular

23. © University of Ottawa, ON, Canada23 Energy-Efficient Transport of Cloud Services (1/7) Conventional network services (s, d) Unicast (s, d) (s, D) Multicast (s, D) Cloud computing services Anycast Energy-Delay Optimal Routing (EDOR) Algorithm X. Dong, et al., Green IP over WDM with Data Centers, IEEE/OSA Journal of Lightwave Technology, vol. 29/12, pp. 1861–1880, June 2011. Manycast Evolutionary Algorithm for Green Light-Tree Establishment (EAGLE) B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, Dec. 2011.

24. © University of Ottawa, ON, Canada24 Energy-Efficient Transport of Cloud Services (2/7) Cloud-over-NSFNET Transport medium: Wavelength Routed (WR) Network During off-peak hours, WR nodes can enter the sleep mode Can add traffic Can drop traffic No pass-through traffic Demand Provisioning Lightpath Light-tree Problem: Energy-Efficient Light-Tree (EELT) selection B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, pp. SAC06.6.1–SAC06.6.5, Dec. 2011. (Considering manycast)

25. © University of Ottawa, ON, Canada25 Energy-Efficient Transport of Cloud Services (3/7) B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, pp. SAC06.6.1–SAC06.6.5, Dec. 2011. Objective Maximize the number of sleeping nodes Minimize total energy consumption Minimize maximum resource (channel) consumption Solving a manycast-based ILP model may lead to significantly long runtimes. Any faster solution? Heuristics: Evolutionary Algorithm for Green Light-tree Establishment (EAGLE)

26. © University of Ottawa, ON, Canada26 Energy-Efficient Transport of Cloud Services (4/7) NO Sort the manycast demands in decreasing order Find an initial solution space I Solution Space P = I Select two candidate solutions in P w.r.t fitness proportionate Crossover on two solutions. Obtain new two individuals End condition reached? Channel assignment on the new individuals New solutions valid? Mutate new individuals with probability of γ Compute a fitness function for each solution in P Add solutions to P Age Solutions in P NO YES Evolutionary Algorithm for Green Light-tree Establishment (EAGLE)Evolutionary Algorithm for Green Light-tree Establishment (EAGLE)

27. © University of Ottawa, ON, Canada27 Energy-Efficient Transport of Cloud Services (5/7) B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, pp. SAC06.6.1–SAC06.6.5, Dec. 2011. Crossover in EAGLE

28. © University of Ottawa, ON, Canada28 Energy-Efficient Transport of Cloud Services (6/7) B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, pp. SAC06.6.1–SAC06.6.5, Dec. 2011. Fitness Functions in EAGLE Maximize the number of sleeping nodes Minimize the maximum channel index Minimize the total consumed energy β of the idle power is consumed in the sleep mode

29. © University of Ottawa, ON, Canada29 Energy-Efficient Transport of Cloud Services (7/7) B. Kantarci and H. T. Mouftah, Energy-efficient cloud services over wavelength-routed optical transport networks, in in Proc. of IEEE GLOBECOM, pp. SAC06.6.1–SAC06.6.5, Dec. 2011. Cloud service demands arrive in four time zones EST, CST, MST, PSTEST, CST, MST, PST Size of the destination set : {3,4} Crossover prob. 0.20 Mutation ratio: 0.01 Solution space: 100 solutions Energy consumption of EAGLE throughout the day PST MST CST EST

30. © University of Ottawa, ON, Canada30 Summary Energy Efficient cloud computing Balance between process, storage and transport Processing and Storage Workload placementWorkload placement –Thermal-aware –Cooling-aware –Thermal-and-cooling-aware highest thermostat setting Thermal activity monitoring of data centers by WSNsThermal activity monitoring of data centers by WSNs –rDCP for data collection by the WSN inside the data center Transport –Energy-efficient anycasting/manycasting of cloud service demands

31. © University of Ottawa, ON, Canada31 Research Directions Energy-Efficient Cloud Computing Processing and Storage Enhanced Data Center MonitoringEnhanced Data Center Monitoring Further reduction in PUE of the data centersFurther reduction in PUE of the data centers PUE enhancement vs SLA guaranteesPUE enhancement vs SLA guarantees Solutions for public and private cloudsSolutions for public and private clouds Transaction specific energy consumptionTransaction specific energy consumption Holistic solutions considering servers, storage and non-IT equipment

32. © University of Ottawa, ON, Canada32 Thanks for your attention! Questions? mouftah@site.uottawa.camouftah@site.uottawa.ca, kantarci@site.uottawa.cakantarci@site.uottawa.ca