1. Energy Conservation in Datacenters through Cluster Memory Management and Barely-Alive Memory Servers Vlasia Anagnostopoulou (vlasia@cs.ucsb.edu), Susmit Biswas, Alan Savage, Ricardo Bianchini*, Tao Yang, Frederic T. Chong Department of Computer Science, UC Santa Barbara *Department of Computer Science, Rutgers University

2. Dependence on Internet- services… More online services –new internet-services, email, informational sites, social networks… Example: –Growth of web-search 10,000 500,000 200,000,000 450,000,000

3. Environmental impact Internet-services live in datacenters Thousands of machines per datacenter Many datacenters across the globe –Energy consumption: ~1.2% total US [Ref: EPA] –Energy consumpt. growth:

4. Are Datacenters efficient? Strict performance standards for internet-services through SLAs Over-provisioning –Machines are under-utilized most of the time –Servers are inefficient at low or average utilization Ref: Barroso and Hölzle

5. Current techniques for efficiency Under low load, reconfigure cluster –Consolidate load into fewer machines –Turn rest off –Transition to low power idle state –Memory is not accessible in these states Operate at lower frequency (VS) Performance problems –For internet-services the working set typically doesn’t shrink with load! –Because of reboot, very slow to restart (~sec) –Have to warm-up memory

6. For web-search, memory is particularly critical Search dataset doesn’t change much with load! Searches have temporal locality -> Zipf’s distribution [Ref:Adamic] Intense database search –May search up to hundreds servers at a time But fairly light CPU task to process a search query Memory can and should be managed wisely, in order not to loose performance!

7. Our technique for efficiency + performance: Barely-Alive state: –CPU is turned off, memory is kept on –Much lower power consumption Distributed middleware: –Request distribution –Transition servers to BA state –Manage server memory content locally –Allocate optimal memory to services globally –Do not degrade performance (respect SLA) Hardware requirements?

8. How would it be implemented Memory is accessed through Memory Controller (MC) MC is on CPU (bummer!) Install small CPU on NIC Memory accessible like DMA via new CPU + MC Turn off main CPU Software requirements? OF F

9. Basic request distribution in a self- managed cluster algorithm Challenges of integrating BA servers into the request distribution scheme: Transition from Active to BA and vice versa Stale content of BA servers LARD: locality-aware request distribution [Ref: V.Pai+others] PRESS: its distributed version [Ref: Carrera+Bianchini] Main idea: Exploit locality in references by forwarding same requests to same machine But balance load evenly among machines I am less loaded than Server 3

10. Self-managed cluster with BA servers Transition Active to BA –Application decides on global level –Locally, if there are no procs or reqs –Make sure not to over-utilize active servers! Stale content of BA servers –Store installs new object (immutability) –Application may invalidate old objects at will –On activate, BA updates its Directory from active –Periodic activation or state swapping –Space of obsolete objects can be reclaimed Optimal memory allocation? Multiple services? Energy efficiency?

11. Middleware for efficient memory management Optimal memory allocation –Dynamically size memory to respect exactly the SLA requirement –Translate SLA requirement -> target hit-ratio –Use stack algorithm to predict optimal size from target hit-ratio Stack algorithm overview –Measures contribution of cache size to the hit-ratio –On a single pass, it calculates the cumulative hit-ratio with size How to adapt the stack algorithm for resizing the global cluster memory optimally? SizeHitsHit-ratio 16/966.7% 21/977.8% 30/977.8%

12. Optimal memory allocation Distributed stack algorithm –For each server: –Keep track of memory size + hit ratio information –On time window, broadcast size for desired hit-ratio. –Resize local stack with global average size Extension for BA servers, variable sized objects, multiple services…

13. Extension of distributed Stack Algo Include BA servers –Contribute fixed amount of memory (passive) Multiple-size objects –Separate stack for each object size –This leverages directory look-up Multiple services –Each service keeps its own stack in the memory –Memory partitioned across services Energy efficiency of BA state (without the efficiency yielded from the memory management)

14. Power savings potential

15. Cumulative power savings Synthetic search trace over 1 day (24h)

16. Future work Currently looking into more on-line and off- line apps (e.g. web-translation, sorting algorithm) Extend power consumption breakdown Sensitivity analysis of power savings to simulation’s parameters –(e.g. memory capacity, network assumptions, component access times, etc) Evaluation of distributed algorithm

17. Conclusions Datacenters have a growing impact on the environment Machines in datacenters are inefficient Memory is a critical component for performance for applications run on a cluster Exploit memory without degrading performance with Barely-Alive state + middleware Potential power savings up to 49%, without loss of performance

18. Questions? Thank you for your attention! vlasia@cs.ucsb.edu www.cs.ucsb.edu/~arch