1. 1 MemScale: Active Low-Power Modes for Main Memory Qingyuan Deng, David Meisner*, Luiz Ramos, Thomas F. Wenisch*, and Ricardo Bianchini Rutgers University *University of Michigan

2. 2 Server memory power challenges Power consumption of a Google server [Barroso & Hoelzle’07] DRAM power varies little with load Memory power represents 30-40% of total power for typical loads Fraction is larger since memory controller power is not included Compute Load (%) Power (% of peak)

3. 3 Improving memory energy efficiency Observation: Memory bandwidth is rarely fully utilized [Meisner’11]; we can save energy during periods of light and moderate load Previous approaches Leveraging DRAM idle low-power state [Lebeck’00][Delaluz’01][Li’04][Diniz’07]… Rank sub-setting and DRAM reorganization [ Ahn’09][Udipi’10][Zheng’10]… Memory controller power is typically not considered Need active low-power modes to save energy when underutilized Frequency has greater impact on bandwidth than latency

4. 4 MemScale: Active low-power modes for memory Goal: Dynamically scale memory frequency to conserve energy Hardware mechanism: Frequency scaling (DFS) of the channels, DIMMs, DRAM devices Voltage & frequency scaling (DVFS) of the memory controller Key challenge: Conserving significant energy while meeting performance constraints Approach: Online profiling to estimate performance and bandwidth demand Epoch-based modeling and control to meet performance constraints Main result: System energy savings of 18% with average performance loss of 4%

5. 5 Outline Motivation and overview Background on memory systems MemScale: DVFS for the memory system Results Conclusions

6. 6 Impact of frequency scaling on memory latency ACT CL Burst PRE ACTCLPREBurst Time ACTCLPREBurst MC 800 MHz 400 MHz For DDR3 DRAM, scaling frequency from 800MHz to 400MHz: bandwidth down by 50%, latency up by only 10% Req Reply

7. 7 Opportunity for MemScale Background: clock tree, I/O driver, register, PLL, DLL, refresh, others Effects of lower frequency on power: Lowers background power linearly (~f) Lowers MC power by cubic factor (~f^3) Dynamic: read, write, termination MC: memory controller

8. 8 Outline Motivation and overview Background on memory systems MemScale: DVFS for the memory system Results Conclusions

9. 99 MemScale design Goal: Minimize energy under user-specified slowdown bound Approach: OS-managed, epoch-based memory frequency tuning Each epoch (e.g., an OS quantum): 1.Profile performance & bandwidth demand New performance counters track mem latency, queue occupancies 2.Estimate performance & energy at each frequency Models estimate queuing delays & system energy 3.Re-lock to best frequency; continue tracking performance Slack: delta between estimated & observed performance 4.Carry slack forward to performance target for next epoch

10. 10 Frequency and slack management Time Epoch 1Epoch 2Epoch 3Epoch 4 High Freq. Low Freq. MC, Bus + DRAM CPU Pos. Slack Neg. Slack Pos. Slack Profiling Target Actual Calculate slack vs. target Estimate performance/energy via models

11. 11 Modeling of performance and energy New performance counters enable estimate of Level of contention (bank and bus) Energy consumption CPI of each application Avg memory latency Performance slack Estimate full system energy

12. 12 MemScale adjusts frequency dynamically Timeline of workload mix MID3

13. 13 Outline Motivation and overview Background on memory systems MemScale: DVFS for the memory system Results Conclusions

14. 14 Methodology Detailed simulation 16 cores, 16MB LLC, 4 DDR3 channels, 8 DIMMs Multi-programmed workloads from SPEC suites Power modes 10 frequencies between 200 and 800 MHz Power consumption Micron’s DRAM power model Memory system power = 40% of total server power

15. 15 Results – energy savings and performance Memory energy savings of 44% System energy savings of 18% always within performance bound Average energy savingsPerformance overhead

16. 16 Alternative approaches Fast power-down Transition ranks into fast power-down mode when idle Decoupled-DIMM [Zheng’09] Low frequency DRAM + high frequency DIMMs & channels Static Pre-selected active low-power mode w/o dynamic scaling Unrealistic: needs a priori knowledge of workload behavior

17. 17 Results – comparison to alternative approaches Performance overhead (MID)Full system energy savings (MID) Energy Savings (%)CPI increase (%) Fast-PD Decoupled-DIMM Static MemScale MemScale+Fast-PD Fast-PD Decoupled-DIMM Static MemScale MemScale+Fast-PD

18. 18 Conclusions MemScale contributions: Active low-power modes for the memory subsystem New perf. counters to capture energy and contention OS policy to choose best power mode dynamically Avg 18% system energy savings, avg 4% performance loss In the paper Performance and energy models Sensitivity analyses (including lower performance bounds) Energy break-down comparison

19. 19 THANKS! SPONSORS: