1. Power Management for Memory Systems Ming Chen Nov. 10 th, 2009 ECE 692 Topic Presentation 1

2. Why Power Control for Main Memory?  Memory capacities have been increasing significantly to accommodate CMPs. −CPU is no longer the only major power consumer.  Memory is highly under-utilized. −Requirement on amount −Requirement on bandwidth 85.4% 92.2% CPU & memoryCPU : memory 1.21 0.69 2

3. Limiting the Power Consumption of Main Memory Acknowledgments: The organization order and contents of some slides are based on Ricardo Bianchini’s slides. Bruno Diniz, Dorgival Guedes, Wagner Meira Jr. Federa University of Minas Gerais, Brazil Ricardo Bianchini Rutgers University, USA 3

4. Power Saving Vs. Power Control  A problem with two sides −Trade-off between power and performance  Power saving: −Guarantee performance first, then minimize power. −Performance is primary. −Save energy bill.  Power control: −Power capping: cooling, thermal, packaging, etc −Guarantee power budget first, then maximize performance. −Power budget is primary. −Avoid system failure and thermal violations. 4

5. What is This Paper About?  Bunches of work done to save power.  The first paper I have read on power control for memory.  Propose 4 policies for Power Limiting (PL) in memory. −Knapsack, LRU-Greedy, LRU-smooth, LRU-Ordered.  Combine Power Limiting with Energy Conserving (PL- EC).  Also provide performance guarantee. (PL-EC-Perf) An interesting paper that combines the two sides of the power problem together. 5

6. Power Actuators  RDRAM systems. −It is DRAM but not DDR SDRAM  Each chip can be transitioned independently.  Different power states 6

7. Power Limiting Memory Controller chip1 chip3 chip4chip2 A SN S 7

8. Power Limiting Memory Controller chip1 chip3 chip4chip2 access A SN S 8

9. Power Limiting Memory Controller chip1 chip3 chip4chip2 A SN S 9

10. Adjusting Power States Memory Controller chip1 chip3 chip4chip2 SN S N 10

11. Power Limiting Memory Controller chip1 chip3 chip4chip2 access S SA N Different approaches to adjust states. 11

12. Knapsack: Key Idea  Multi-Choice Knapsack Problem (MCKP) −Object: memory device. −Choices: multiple (power) states. −weight : the power consumption. −cost : the transition overhead to the active state.  Goal: Minimize the cost with the constraint of weight by putting each object in a state.  MCKP is NP-hard, which is solved off-line. 12

13. chip # power state Knapsack: An example  LRU queue is maintained for active devices.  The LRU device is the victim.  Switch the power states of the two chips. 13

14. LRU-Greedy  An LRU queue for all devices  If a device is to be accessed, move it to the tail and: −Active? Go on … −Not active? Put the LRU to the shallowest state. Still not? adjust the state for the next device. 14

15. LRU-Smooth  A LRU queue for all devices  If a device is to be accessed, move it to the tail and: −Active? Go on … −Not active? Put the LRU to the next lower-power state. Still not? adjust the state for the next device. 15

16. LRU-Ordered: Key Idea  An LRU queue for active devices  An ordered queue for devices in low-power mode (shallowest first)  If a device is to be accessed: −Active? Move it to the tail of LRU and Go on … −Not active? Move it from ordered queue to the tail of LRU queue. Put the LRU to the top of the ordered queue. Still not? adjust the state of the next device in the ordered queue to the next lower-power state. 16

17. LRU-Ordered: An example 17

18. Energy Conservation (PL-EC)  If idle time in the current state > break-even time, then lower power state.  Minimize delay*power 2 by Knapsack for different # of devices in the active state.  Whenever the # of active states is going to change: −An active device is transitioned to the next low-power state when threshold expires. −A low-power device is transitioned to the active state and it does not violate the budget  The memory controller looks up the table and adjusts the states.  If the activating device violates the budget, the basic scheme (PL) is used. 18

19. Performance Guarantee (PL-EC-Perf)  To what extent the energy to be saved?  Basic strategy is from Xiaodong Li’s ASPLOS04 paper. −5M-cycle epoch −User-defined slowdown (3%) compared with PL −Compute slack at runtime. −If slack < 0, disable EC until the end of epoch  Disabling EC means reverting back to the corresponding PL policy. 19

20. Evaluation Methodology  Single-core in-order CPU with integrated memory controller  Simics + memory subsystems  OS and physical mapping of virtual pages are both simulated.  Memory system is driven by traces generated by Simics.  Workloads: MediaBench, SPEC 2000, and client-server applications.  Memory size: 512 MB  Performance is measured by the execution time of a trace file. 20

21. Performance Vs. PL Polices  Knapsack and LRU-Ordered are best.  8 chips, 50% power budget. 21

22. Energy Vs. Policies  Compared with unrestricted execution.  Unrestricted < budget  bzip? (Critique)  8 chips, 50% power budget. 22

23. Performance Vs. Budget  8 chips under LRU-Ordered.  Performance degradation is very small. 23

24. Energy Vs. Power Budget  Saving decreases as budget decreases.  Uniform for all workload. 24  8 chips, 50% power budget.

25. Performance for PL-EC-Perf  8 chips, 25% power budget  LRU-Ordered  3% slowdown  PD: an explicit energy saving algorithm  PL/PL-EC-Perf almost works no worse than PD  Exception is bzip2 ? 25

26. Energy Saving for PL/PL-EC-Perf  8 chips, 25% power budget  LRU-Ordered  3% slowdown  PD: an explicit energy saving algorithm  PL/PL-EC-Perf has more energy saving than PD.  PD tends to send some chips to very deep states. 26

27. Conclusions  Four power limiting policies are proposed.(PL)  Performance degradation is surprisingly low.  Limiting power + energy conserving (PL-EC)  Limiting power + energy conserving + performance guarantee (PL-EC-Perf)  Limiting power consumption is as effective as doing energy conservation explicitly. 27

28. A Performance-Conserving Approach for Reducing Peak Power Consumption in Server Systems Acknowledgments: The organization order and contents of some slides are based on Wes Felter’s slides. Wes Felter, Karthick Rajamani, Tom Keller IBM Austin Research Lab Cosmin Rusu University of Pittsburgh 28

29. Motivations  System designers can no longer afford to accommodate peak power of all components (over- provisioning)  System failures due to power overload and thermal violation.  CPU is no long the only major power consumer.  CPU and the main memory share the same power/cooling facility. 29

30. Anti-Correlation of Processor and Memory Power  Processor and memory are not simultaneously highly utilized in workloads.  Intuitively, the processor can’t keep itself and the memory busy 30

31. Unconstrained System Power In theory, the system can use 83W. 31

32. What is This Paper About?  Power shifting between processor and memory.  The first paper that proposes the concept of “power shifting”.  Power estimation model based on the # of activities.  Propose 3 policies for power control in the server level. −PLI, sliding window, and on-demand 32

33. Processor Power Model  Power Vs. Dispatched Instr./cycle  100K-cycle interval  28 applications  Linear regression  33

34. Memory Power Model  Power Vs. Bandwidth  100K-cycle interval  28 applications  Linear regression   P mem =#ranks*#devices*V DRAM *((I active - I idle )*BW/ Peak BW + I idle )+ P others 34

35. Power Actuators  Power consumption correlates strongly with activity.  Activity regulation techniques: −Instruction decoding throttling −Clock throttling: effective duty cycles −DVFS  For processor: −Throttle at the instruction dispatch unit of the pipeline.  For memory: −Limit the total # of memory requests 35

36. Processor Core System Power Controller Dispatched Insns. Counter Memory Fetch Throttling Memory Controller Request Counter Request Throttling Goes into powerdown mode when idle Extensive clock gating System Architecture 36

37.  The # of activities is the same in the next interval.  Estimate the power based on history.  Allocate power based on estimates.  Power allocation is enforced by thresholds of the # of activities  Activity-dependent power and standby power Key Ideas 37

38.  Power estimation −CPU power = C1*DPC0+C2 −Memory power = M1*BW0+M2 − Power to be allocated: P dynamic = P budget – C2 – M2 −DPC1 and BW1 for the next interval −Estimated active power: P est = DPC0*C1+BW0*M1  Power allocation −DPC1= DPC0*P dynamic /P est −BW1=BW0*P dynamic /P est  Threshold −D th = DPC1*Period −M th =BW1*Period Proportional-Last-Interval Policy 38

39.  Sliding window −Shorter interval is better for estimation accuracy. −Larger interval is better for reducing noise. −A larger window includes 20 intervals.  On demand −No violations, no throttling −Interval should be small enough.  Run-To-Exhaustion (RTE) −Power is monitored cycle-by-cycle. −Throttle when power is violated. −Impractical but provides a comparison.  Static: proportional to the peak power Other Related Policies 39

40. Simulation Environment  Traces from hardware (SPEC) and Mambo (e.g. JBB)  Integrated simulation environment −Turandot+PowerTimer (Zhigang Hu et al., IBM TJ Watson) Timing and power core model 2GHz 970-like core w/ aggressive clock gating 512KB L2 cache (power not simulated) −+ MEMSIM Timing and power DRAM model 4GB, 4 ranks of 128-bit 400MHz DDR (PC3200) −Both simulators synchronized every cycle 40

41.  Budget: 40 W  Much better than static budgeting PLI Vs. Static (1) 41

42.  Budget: 50 W  Average unconstrained power consumption < budget PLI Vs. Static (2) 42

43.  100K-cycle interval, 40W budget  On-demand is generally the best. −At the cost of at least one-interval budget violation Policy Comparison 43

44.  On-demand and the sliding window are generally the best.  On-demand is even better than RTE for art. −Not proactively throttle activities and at cost of short violations. Normalized to RTE 44

45. Interval Size (PLI)  ammp: highly variable even at small interval and steady power.  Small interval has better fit for variations.  Generally, highly application-dependent. 45

46. Critiques  Paper 1 −Lack explanations for different workload (e.g. bzip). −Examples used for policies are not typical. −Do not explain why the performance is surprisingly slightly degraded.  Paper 2 −Open-loop estimation based scheme −Model verification −Power is not shifting but throttled −Very large performance degradation even budget is larger than the average. 46

47. Comparison of the Two Papers Limiting PowerPower Shifting TargetPeak power of DRAM systemPeak power in the server level GoalPeak power capping with energy conserving Peak power capping MethodologyKnapsack optimization + heuristic Open-loop estimation SolutionsA bunch of policies and comparison ExperimentsSimulation Power budgetLarger than the average 47

48. Thank you ! 48