1. Reducing Energy Consumption of Disk Storage Using Power Aware Cache Management Qingbo Zhu, Francis M. David, Christo F. Deveraj, Zhenmin Li, Yuanyuan Zhou Department of Computer Science University of Illinois at Urbana-Champaign Pei Cao* *Cisco Systems Inc. HPCA 04 02/17/2004

2. Data Centers: Service-based Computing router Web Servers Application Servers Database Servers switch … … … SAN … Storage Servers

3. Energy Problem Faced by Data Centers Data centers High electricity bills: up to 25% TCO $8M per year for a 30,000-square-foot data center [EERE news 2003] Increase as much as 25% annually [Energy User News 2002] Storage 27% of the total energy consumed [Maximum Inc. 2002]

4. Disk Power Model Disk power modes Active/idle/standby/sleep Spinup/down cost Breakeven time Metrics Energy consumption Average response time

5. Disk Power Management Schemes Oracle scheme (off-line) Practical scheme (on-line) access1 access2 IdleTime > BreakEvenTime Idle for BreakEvenTime Wait time

6. Current Research Status The idle periods in server workloads are too short to justify high spinup/down cost of server disks [ISCA 03][ISPASS 03] [ICS 03 ] IBM Ultrastar 36Z15 -- 135J/10.9s Multi-speed disk model [ISCA 03] RPMs: multiple intermediate power modes Smaller spinup/down costs Be able to save energy for server workloads Most previous work assume that all requests go directly to physical disks

7. Observation Many requests are filtered out by the storage cache EMC Symmetrix storage system Up to 128GB storage cache IBM ESS system Up to 64GB storage cache Cache replacement and write policies affect the access sequences to physical disks Block-based storage system

8. The Focus of Our Paper Power-aware off-line and on-line cache replacement algorithms and write policies reduce the disk energy consumption Clarification The underlying disk power management scheme is NOT changed The storage cache is always active

9. Outline Motivation Power aware cache management Belady s algorithm is NOT energy-optimal Off-line power-aware greedy algorithm On-line power-aware algorithm Four write policies Simulations Conclusion Limitations and future work

10. Breakeven-Time for Multiple Power Modes Energy Consumption Idle Period Length mode 0mode 1 mode 2 mode 3 Spinup cost Active mode t1 t3 t2 T E(T)

11. Is Belady s Algorithm Energy-Optimal? Belady s algorithm: performance-optimal Minimize the number of misses Evicting the block with the longest future reference distance Answer: NO! Only consider the access sequence Ignore requests arrival time Ignore multiple disk scenario

12. A Simple Example t A B BA C Disk 0 D An energy-optimal algorithm using dynamic programming Belady s algorithm power-aware algorithm C

13. Off-line Power-Aware Greedy Algorithm Idea: evicting the block with the smallest energy penalty Observation: take advantage of the knowledge about future s bound-to-happen misses Cold misses Capacity misses due to previous evictions D E F: bound-to-happen misses A B BC EFAD

14. How to Calculate Energy Penalty of Evicting a Block D E F: bound-to-happen misses A B BC EFD A E(DE)E(AE)E(DA)+ - Energy Penalty (A) = E(EF)E(BF)E(EB)+Energy Penalty (B) = -

15. Re-view Energy Consumption Idle Period Length mode 0mode 1 mode 2 mode 3 t1 t3 t2

16. On-line Power Aware Algorithm Idea: selectively keep blocks from inactive disks in the cache for a longer time Make inactive disks more inactive Idle Period Length Energy Saving mode 0 mode 1 mode 2 mode 3 Super Linear t1 t2 t4 t3 energy saving energy penalty <<

17. How to Measure Disk Activeness? Characteristics of inactive disks Small percentage of cold misses Large idle period lengths with high probability

18. How to Keep Track of Cold Misses? Bloom Filter: a space-efficient membership test method A vector v of m bits k independent hash functions ranging {1..m} Given an access for block a, check the bits at position If any of them is 0, a is cold miss and then set all bits 1 Otherwise, it is not a cold miss though we may be wrong 1.6M blocks with v = 2M bytes and k = 7 the accuracy is 99.18%

19. How to Keep Track of the Distribution of Idle Period Lengths? Histogram-based estimation Idle Period Length

20. Case Study: PA-LRU Applies to all cache replacement algorithms LRU, 2Q, MQ etc. PA-LRU Two LRU stacks LRU0: blocks from active disks LRU1: blocks from inactive disks Evict blocks from LRU0 first The evaluation of disk activeness is epoch-based Adapt to workload changes

21. Write Policy Write back Write through Write back with eager updates (WBEU) Eagerly write back all the dirty blocks when the target disk becomes active due to a read miss Write through with deferred updates (WTDU) Use a log disk which is always active Write the blocks to the log disk if the target disk is not active Flush back all the logged blocks when the target disk becomes active due to a read miss Retain persistent semantics

22. Evaluation Methodology Experiment setup DiskSim: IBM Ultrastar 36Z15 Enhanced by a multi-speed disk power model Enhanced by a CacheSim Real system traces: OLTP Cello96 Synthetic traces: Exponential distribution Pareto distribution

23. Energy (OLTP) OPG: energy saving 2% - 9% over Belady s algorithm PA-LRU: energy saving 16% over LRU

24. Average Response Time (OLTP) OPG: 4% better than belady s algorithm PA-LRU: 50% better than LRU (avoid expensive spinup)

25. Conclusion Power aware cache management plays an important role on disk energy consumption Belady s algorithm is NOT energy-optimal Evict the blocks with small energy penalty Make inactive disks more inactive

26. Future Work and Acknowledgements Limitations and future work Design online algorithms for a single disk as well Take prefetching into account Real system experiments Acknowledgements Anonymous reviewers Professor Lenny Pitt (UIUC) CMU Parallel Data Lab (for DiskSim) HP Lab (for Cello Trace) Questions?

27. Thanks!

28. Backup Slides

29. Write Policies (Exponential Distribution) Write back: up to 20% saving than write through WBEU: up to 60% saving than write through WTDU: up to 55% saving than write through

30. Energy-optimal problem

31. Offline Energy-optimal Algorithm Only two power state 1: active mode 0: standby mode Virtual time Only one disk Parameters: b: the number of disk blocks k: the number of cache blocks n: the input size m: threshold Cache State (C, t, i) The cache contains the blocks in set C after the first i+1 references and the last t consecutive reference were ache hit

32. Offline energy optimal algorithm Minimize energy: maximize the time the disk can spend in standby mode A(C,t,i): the maximum time that the disk spends in the standby mode until (C,t,i) is reached Dynamic programming: Extend to multiple disks:

33. Time Breakdown

34. Mean Inter-arrival Time

35. Simulation Results: Cello96 OPG: energy saving 5% - 7% over belady s algorithm PA-LRU: energy saving 2% - 3% Cello96: high cold miss ratio, larger than 65% for all disks

36. OPG is heuristic D E: bound-to-happen misses A B B C ED A

37. A Step Further … Consider both miss ratio and energy penalty Idea: don t differentiate among blocks whose energy penalty is smaller than a threshold T energy penalty smaller than T: round up to T T=0: pure greedy algorithm T is large enough: belady s algorithm

38. Data Centers: Service-based Computing Internet Web Servers Database Servers Local Storage Ethernet Storage Servers SAN