GreenSoftware: Managing Datacenters Powered by Renewable Energy Íñigo Goiri, William Katsak, Md E Haque, Kien Le, Ryan Beauchea, Jordi Guitart, Jordi Torres,

Slides:

Advertisements

Similar presentations

Network II.5 simulator ..

Advertisements

Parasol and GreenSwitch: Managing Datacenters Powered by Renewable Energy Íñigo Goiri, William Katsak, Kien Le, Thu D. Nguyen, and Ricardo Bianchini Department.

University of Minnesota Optimizing MapReduce Provisioning in the Cloud Michael Cardosa, Aameek Singh†, Himabindu Pucha†, Abhishek Chandra

GreenSlot: Scheduling Energy Consumption in Green Datacenters Íñigo Goiri, Kien Le, Md. E. Haque, Ryan Beauchea, Thu D. Nguyen, Jordi Guitart, Jordi Torres,

Locality-Aware Dynamic VM Reconfiguration on MapReduce Clouds Jongse Park, Daewoo Lee, Bokyeong Kim, Jaehyuk Huh, Seungryoul Maeng.

Supply and Demand Coordination in Energy Adaptive Computing (invited talk) Dr. Krishna Kant Intel/GMU M. Murugan, U/Minn 1.

Walter Binder University of Lugano, Switzerland Niranjan Suri IHMC, Florida, USA Green Computing: Energy Consumption Optimized Service Hosting.

1 MemScale: Active Low-Power Modes for Main Memory Qingyuan Deng, David Meisner*, Luiz Ramos, Thomas F. Wenisch*, and Ricardo Bianchini Rutgers University.

Power Management in Cloud Computing using Green Algorithm -Kushal Mehta COP 6087 University of Central Florida.

Cloud Computing Resource provisioning Keke Chen. Outline  For Web applications statistical Learning and automatic control for datacenters  For data.

Energy-efficient Virtual Machine Provision Algorithms for Cloud System Ching-Chi Lin Institute of Information Science, Academia Sinica Department of Computer.

Multifaceted Resource Management in Virtualized Providers Íñigo Goiri PhD Defense June 14th, 2011 Advisors: Jordi Guitart and Jordi Torres.

Green Cloud Computing Hadi Salimi Distributed Systems Lab, School of Computer Engineering, Iran University of Science and Technology,

A Grid Resource Broker Supporting Advance Reservations and Benchmark- Based Resource Selection Erik Elmroth and Johan Tordsson Reporter ： S.Y.Chen.

Energy Efficient Prefetching – from models to Implementation 6/19/ Adam Manzanares and Xiao Qin Department of Computer Science and Software Engineering.

Energy Efficient Prefetching with Buffer Disks for Cluster File Systems 6/19/ Adam Manzanares and Xiao Qin Department of Computer Science and Software.

Register Packing Exploiting Narrow-Width Operands for Reducing Register File Pressure Oguz Ergin*, Deniz Balkan, Kanad Ghose, Dmitry Ponomarev Department.

What is Solar Power? A Simple Example How Does Solar Work? Where do we use Solar Power? What is Photovoltaic Power? What are the Components of a Solar.

Present By : Bahar Fatholapour M.Sc. Student in Information Technology Mazandaran University of Science and Technology Supervisor:

Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1.

Undergraduate Poster Presentation Match 31, 2015 Department of CSE, BUET, Dhaka, Bangladesh Wireless Sensor Network Integretion With Cloud Computing H.M.A.

A Survey of Home Energy Management Systems in Future Smart Grid Communications By Muhammad Ishfaq Khan.

CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 34 – Media Server (Part 3) Klara Nahrstedt Spring 2012.

MULTICOMPUTER 1. MULTICOMPUTER, YANG DIPELAJARI Multiprocessors vs multicomputers Interconnection topologies Switching schemes Communication with messages.

CoolAir Temperature- and Variation-Aware Management for Free-Cooled Datacenters Íñigo Goiri, Thu D. Nguyen, and Ricardo Bianchini 1.

GreenHadoop: Leveraging Green Energy in Data-Processing Frameworks Íñigo Goiri, Kien Le, Thu D. Nguyen, Jordi Guitart, Jordi Torres, and Ricardo Bianchini.

WINDOWS AZURE Scott Guthrie Corporate Vice President Windows Azure Application Platform.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2007 (TPDS 2007)

XI HE Computing and Information Science Rochester Institute of Technology Rochester, NY USA Rochester Institute of Technology Service.

PARAID: The Gear-Shifting Power-Aware RAID Charles Weddle, Mathew Oldham, An-I Andy Wang – Florida State University Peter Reiher – University of California,

1 Distributed Operating Systems and Process Scheduling Brett O’Neill CSE 8343 – Group A6.

1 Optimal Power Allocation and AP Deployment in Green Wireless Cooperative Communications Xiaoxia Zhang Department of Electrical.

LOGO Scheduling system for distributed MPD data processing Gertsenberger K. V. Joint Institute for Nuclear Research, Dubna.

Cloud Computing Energy efficient cloud computing Keke Chen.

Leveraging Renewable Energy in Data Centers Ricardo Bianchini on tour 2012.

Building Green Cloud Services at Low Cost Josep Ll. Berral, Íñigo Goiri, Thu D. Nguyen, Ricard Gavaldà, Jordi Torres, Ricardo Bianchini.

Parasol: A Solar-Powered µDatacenter Íñigo Goiri Ricardo Bianchini, Thu D. Nguyen Team: Josep Lluis Berral, Md Haque, Bill Katsak, Kien Le Department of.

1 A Framework for Data-Intensive Computing with Cloud Bursting Tekin Bicer David ChiuGagan Agrawal Department of Compute Science and Engineering The Ohio.

Challenges towards Elastic Power Management in Internet Data Center.

Papers on Storage Systems 1) Purlieus: Locality-aware Resource Allocation for MapReduce in a Cloud, SC ) Making Cloud Intermediate Data Fault-Tolerant,

Joint Power Optimization Through VM Placement and Flow Scheduling in Data Centers DAWEI LI, JIE WU (TEMPLE UNIVERISTY) ZHIYONG LIU, AND FA ZHANG (CHINESE.

Quantifying and Improving I/O Predictability in Virtualized Systems Cheng Li, Inigo Goiri, Abhishek Bhattacharjee, Ricardo Bianchini, Thu D. Nguyen 1.

Copyright © 2011, Performance Evaluation of a Green Scheduling Algorithm for Energy Savings in Cloud Computing Truong Vinh Truong Duy; Sato,

Chapter 8-2 : Multicomputers Multiprocessors vs multicomputers Multiprocessors vs multicomputers Interconnection topologies Interconnection topologies.

A dynamic optimization model for power and performance management of virtualized clusters Vinicius Petrucci, Orlando Loques Univ. Federal Fluminense Niteroi,

Towards Dynamic Green-Sizing for Database Servers Mustafa Korkmaz, Alexey Karyakin, Martin Karsten, Kenneth Salem University of Waterloo.

VGreen: A System for Energy Efficient Manager in Virtualized Environments G. Dhiman, G Marchetti, T Rosing ISLPED 2009.

Using Map-reduce to Support MPMD Peng

An Energy-efficient Task Scheduler for Multi-core Platforms with per-core DVFS Based on Task Characteristics Ching-Chi Lin Institute of Information Science,

An Adaptive Energy-Efficient and Low- Latency MAC for Data Gathering in Wireless Sensor Networks Gang Lu, Bhaskar Krishnamachari, and Cauligi S. Raghavendra.

We hear much about energy problems; supply shortages, pollution issues and high prices, but the solutions to these problems are here now in the form of.

XI HE Computing and Information Science Rochester Institute of Technology Rochester, NY USA Rochester Institute of Technology Service.

Performance Performance is about time and the software system’s ability to meet timing requirements.

1 Adaptive Parallelism for Web Search Myeongjae Jeon Rice University In collaboration with Yuxiong He (MSR), Sameh Elnikety (MSR), Alan L. Cox (Rice),

ApproxHadoop Bringing Approximations to MapReduce Frameworks

Scheduling MPI Workflow Applications on Computing Grids Juemin Zhang, Waleed Meleis, and David Kaeli Electrical and Computer Engineering Department, Northeastern.

Accounting for Load Variation in Energy-Efficient Data Centers

Optimizing Power and Data Center Resources Jim Sweeney Enterprise Solutions Consultant, GTSI.

A Two-phase Execution Engine of Reduce Tasks In Hadoop MapReduce XiaohongZhang*GuoweiWang* ZijingYang*YangDing School of Computer Science and Technology.

Ensieea Rizwani An energy-efficient management mechanism for large-scale server clusters By: Zhenghua Xue, Dong, Ma, Fan, Mei 1.

Zeta: Scheduling Interactive Services with Partial Execution Yuxiong He, Sameh Elnikety, James Larus, Chenyu Yan Microsoft Research and Microsoft Bing.

Hello Cloud… Mike Benkovich

IIS Progress Report 2016/01/11. Goal Propose an energy-efficient scheduler that minimize the power consumption while providing sufficient computing resources.

Input and Output Optimization in Linux for Appropriate Resource Allocation and Management James Avery King.

Asst. Prof. Dr. Sameer Saadoon Algburi

System Control based Renewable Energy Resources in Smart Grid Consumer

The Management of Renewable Energy

EE5900: Cyber-Physical Systems

Chapter 16: Distributed System Structures

Presentation transcript:

GreenSoftware: Managing Datacenters Powered by Renewable Energy Íñigo Goiri, William Katsak, Md E Haque, Kien Le, Ryan Beauchea, Jordi Guitart, Jordi Torres, Thu D. Nguyen, Ricardo Bianchini Department of Computer Science

Motivation Datacenters consume large amounts of energy High energy cost and carbon footprint – Brown electricity: coal and natural gas Connect datacenters to green sources: solar, wind Apple DC in Maiden, NC 40MW solar farm 2

Challenges and opportunities Scheduling workload/energy sources – Lower costs: brown energy, peak brown power, capital Study opportunities in green datacenters – Build hardware/software Power Time Load Variable 3 Solar power Workload

GreenSoftware How to build software for green datacenters? 1.Malleable energy demand – Idle nodes → Turn off/Sleep (S3) [COLP’01] – Reduce frequency (DVFS) → Lower quality 2.Move computation under renewables – Weather forecast → Green energy forecast – Delay computation or degrade quality – Leverage energy storage 4

Outline Motivation GreenSoftware – GreenSlot – GreenHadoop – GreenSwitch – GreenCassandra – … and others Conclusion 5

GreenSlot [SC’11] Batch jobs on SLURM (& Hadoop) Send idle nodes to S3 Predict solar availability Delay jobs within deadlines – Known jobs characteristics (length, deadline, size…) – Heuristic 6 Time Job 1 Power Deadline Job 2 Job 3 Job 4

GreenSlot [SC’11] Batch jobs on SLURM (& Hadoop) Send idle nodes to S3 Predict solar availability Delay jobs within deadlines – Known jobs characteristics (length, deadline, size…) – Heuristic 7 Time Job 1 Power Deadline Job 2 Job 3 Job 4

GreenHadoop [Eurosys’12] Batch jobs on Hadoop Send idle nodes to S3 Make required data available – Move data blocks Predict solar availability Delay jobs within deadlines – Predict global jobs energy consumption – Heuristic 8 Map Reduce 6 7 Shuffle

Covering subset GreenHadoop: Data management Deactivate servers to save energy – Some data might become unavailable Prior solution: covering subset [Leverich’09] – Set of servers always running has ALL data Our approach Only required data has to be available We usually require fewer active servers Server Block

GreenHadoop: Data management Server Active Decommission Down Server Server Required file Non-required file Server Server JobA 4 JobB 5 JobC 1 6 Running queue: 10

GreenHadoop: Data management Server Server Active Decommission Down GreenHadoop (computation) requires only 2 servers Server Server Server Required file Non-required file JobA 4 JobB 5 JobC 1 6 Running queue: 11

GreenHadoop: Data management Active Decommission Down Move required files to Active servers Server Server Server Server Server Replicate JobA 4 JobB 5 JobC 1 6 Running queue: 12

Server GreenHadoop: Data management Active Decommission Down Decommissioned server can be sent to Down Server Server Server Required file Non-required file 1 Server Server JobA 4 JobB 5 JobC 1 6 Running queue: 13

Server GreenHadoop: Data management Active Decommission Down Jobs to be executed change → Required files change Server Server Non-required file 1 Server Server JobA 4 JobB 5 JobC 1 6 JobD 8 Required file Running queue: 14

Server Server GreenHadoop: Data management Active Decommission Down Make missing data available Server Server Server Server Required file Non-required file JobB 5 JobC 1 JobD 8 Required file Running queue: 15

Server Server GreenHadoop: Data management Active Decommission Down Server Server Server GreenHadoop (computation) requires 3 servers Server Non-required file JobB 5 JobC 1 JobD 8 Required file Running queue: 16

GreenSwitch [ASPLOS’13] Batch jobs on Hadoop Similar to GreenHadoop Energy storage – Battery – Net metering Schedule workload and energy sources – Optimization Evaluation on Parasol (Presented on Monday by Thu) 17

GreenCassandra Distributed DB/storage on Cassandra Add an optional ring Degrade quality when no green DHT Ring A A A Double DHT Ring Optional A A Server Data

GreenSoftware summary TypeMalleable energyGreen adaptability GreenSlotBatch jobs Delay jobs Sleep servers Delay until green GreenHadoopBatch jobs Delay jobs Sleep servers Data management Delay until green GreenSwitchBatch/interactive jobs Delay jobs Sleep servers Delay until green Energy storage GreenCassandraDistributed storageOptional ringDegrade quality GreenSLAVMs Migrate VMs Sleep servers Route green energy to racks GreenParMPI jobs Change parallelism Sleep servers Greater parallelism on green GreenScaleNon-deferrable jobsCPU and mem DVFSFaster on green GreenNebulaGeo distributed VMsMigrate VMs“Follow the renewables” 19

Conclusions Green datacenters – Challenges & opportunities – Hardware/software solution GreenSoftware – Adapt software to green datacenters – Malleable energy demand – Match computation and renewables 20

GreenSoftware: Managing Datacenters Powered by Renewable Energy Íñigo Goiri, William Katsak, Md E Haque, Kien Le, Ryan Beauchea, Jordi Guitart, Jordi Torres, Thu D. Nguyen, Ricardo Bianchini Department of Computer Science

Other GreenSoftware GreenSLA [IGCC’13] – Bringing green energy to users – New hardware to route green energy GreenPar – MPI jobs with sub linear speedup – Use “Free” green energy GreenNebula – VMs in multiple geo distributed datacenters – Follow the sun GreenScale – Change frequency (DVFS) 22

GreenPar MPI jobs on VMs Add more computation capacity – Lower energy-efficiency – Use available “green” 23 Computation capacity Speedup Power Time

GreenSLA [IGCC’13] HPC jobs on VMs Users require % of green energy – Hardware to bring green energy to VMs Scheduling – Assign green energy to racks – Move VMs among racks – Heuristic 24

Default Green Datacenter Solar Inverter PDU Rack3 PDU Rack2 PDU Rack1 Mixed Bus Datacenter Power Distribution

GreenSLA: Proposed Power Distribution PDU Rack3 PDU Rack2 PDU Rack1 Battery Mixed Bus Control Module Charge Controller Green Bus SSS Solar Inverter Datacenter Power Distribution

Parasol without GreenSwitch Green use Green available Net metering Brown use IT load 27

GreenSwitch: deferrable workload Battery discharge Battery charge IT load Net metering 28 Green available Green use