1. Hot-and-Cold: Using Criticality in the Design of Energy-Efficient Caches Rajeev Balasubramonian, University of Utah Viji Srinivasan, IBM T.J. Watson Sandhya Dwarkadas, University of Rochester Alper Buyuktosunoglu, IBM T.J. Watson

2. All Instructions are not Created Equal Critical instructions – lie on the program critical path Non-critical instructions – can be slowed without increasing execution time Potential to improve cache performance (?) [Srinivasan ’01] [Fisk ’99] Prioritization policies [Fields ’01] [Tune ’01] Energy-efficient ALUs [Seng ’01]

3. Energy-Delay Trade-Offs Example energy-delay trade-off techniques:  Voltage scaling, transistor sizing, way prediction, serial-access  Gated-ground cells, high V t VtVt Normalized Leakage Normalized Delay Low8.50.88 Nominal11 High0.231.34 Transistor sizingVariable threshold voltage

4. Exploiting Criticality Design two static banks –  hot bank: fast and high power  cold bank: slow and low power Instructions have to be classified as critical or not and Data has to be placed in one of two banks Energy-efficient ALUs are easier to handle as there is no associated storage

5. Criticality Metric Oldest-N: The N oldest instructions in the queue are critical  Younger instructions are likely to be on mispredicted paths or can tolerate latencies  N can be varied based on program needs  Minimal hardware overhead  Behavior comparable to more complex metrics

6. Instruction Classification

7. Data Classification Exclusively non-critical Exclusively critical

8. Hot-and-Cold Microarchitecture Dispatch Issue Queue Bank Predictor Placement Predictor L2 Cold bank Hot bank Criticality Counters

9. Performance Results

10. Energy Results

11. Results Summary Bank mispredict rate of 9.5% Criticality mismatch rate of 26% Performance loss = 2.7% (data reorganization) + (0.8 x slowdown) L1 cache energy savings of 37%

12. Related Work Recent split-cache organization by Abella and Gonzalez [ICCD’03] Base Fast Slow Data allocation based on criticality of accessing instruction

13. Conclusions Data and instruction classification is reasonably accurate Overhead from contention is non-trivial Results are worthwhile in limited settings The use of criticality for data cache reorganization yields little benefit