1. Dana Butnariu Princeton University EDGE Lab June – September 2011 OPTIMAL SLEEPING IN DATACENTERS Joint work with Professor Mung Chiang, Ioannis Kamitsos, Sangtae Ha and Jasika Bawa

2. Background User/Client = Person using an internet service Request = Network service needed by the client Server/Core = Physical device which handles client requests Data center = Multiple servers grouped together in one location Delay/Latency = Time elapsed between issuing a request and receiving a reply Job = Form in which the client request is perceived at data center level Controller = Physical device which controls the behavior of servers inside a data center Thread = Processing unit available inside servers, responsible for handling client requests

3. Problems and solutions Problem 1 => Servers in data centers are either underutilized or idle for long periods of time. Effect => A lot of energy is wasted by these servers. Simple solution => Put servers to sleep when they are not used. Problem 2 => Putting servers to sleep too often can lead to high switching costs. Problem 3 => Sleeping and waking up servers takes time. Effect 1 => If an optimal algorithm is not used more energy may be consumed by switching server state. Effect 2 => User requests will be handled with high delay. Better solution => Use math and optimization theory to come up with an optimal server sleeping pattern.

4. Want optimality in terms of the tradeoff between energy consumption and request handling delays. Mathematical model designed by members of the EDGE Lab: P CPU : Power consumption due to active core. Q i : Queue occupancy at state i. E ch : Switching cost. W i : Optimal policy at previous state. Optimality

5. Theory and Practice Theoretical model offers good results in terms of the tradeoff between energy savings and delay. Problem 1 => Theory is just a prediction, not a fact. Will the theoretical model results be confirmed by experimental results? Problem 2 => Experiments in large data centers are hard to carry out, supervise and control. Simple solution => Build a small scale replica of a data centers inside the EDGE Lab. Use laptops to simulate servers and a central controller. Use power meters to measure laptop energy consumption. Use a multithreaded model for the central controller to simulate datacenter operations and interactions.

6. Experimental Setup

7. Controller: Generates jobs. Enqueues and dequeues jobs. Implements the optimal sleeping policy. Turns on and puts to sleep servers. Dispatches jobs to servers according to the sleeping policy. Servers: Handle job requests represented by video clips. Have a video encoding component and a video display. component for each job handled. Are turned on and go to sleep based on the central controller input.

8. Experimental Results – Energy savings vs load analysis When no idling is assumed, energy savings fall within 1-7% of theoretically predicted ones for all traffic loads. Energy savings naturally fall with idling as idling assumes that servers are not put to sleep, but instead left on when no job is running.

9. Experimental Results – Comparison with Base Policies Compared with baseline policies, the sleeping policy performs better in terms of energy savings. Baseline policy 1 => Number of on servers = Number of jobs in queue. Baseline policy 2 => Number of on servers = Number given by a certain job threshold.

10. Experimental Results – Multiple server performance Each server consumes the same power and processes jobs at the same speed. Moving from 1 to 8 servers, average delay decreases 4 times, but energy consumption increases only 1.6-1.7 times, giving a good energy consumption-delay tradeoff.

11. Experimental Results – Job splitting across servers Used c servers to simulate the performance of one single server which is c-times faster and consumes c-times more energy in order to handle a job. Diminishing returns from increasing number of servers => 4 servers sufficient to get most of the resource benefits.

12. Conclusions New solution : to lower the increasing energy usage in datacenters. which provides an optimal tradeoff between energy savings and delay in handling client requests. for which both theory and experimental results show that a balance between energy and delay is easily achievable. Experimental results: confirm theoretical predictions. show that energy savings are substantial for low loads (70-80%) and moderate for high loads (30-40%). prove that the optimal server sleeping policy behaves better in terms of energy savings than common baseline policies. show that even as delay increases by a moderate amount, energy consumption remains almost constant.

13. Acknowledgements Professor Mung Chiang, ELE Department. EDGE Lab members Ioannis Kamistos and Sangtae Ha. Undergraduate ELE major Jasika Bawa. All other members of the EDGE Lab. Questions?