1 of 20 Phase-based Cache Reconfiguration for a Highly-Configurable Two-Level Cache Hierarchy This work was supported by the U.S. National Science Foundation and the Semiconductor Research Corporation + Also Affiliated with NSF Center for High- Performance Reconfigurable Computing Ann Gordon-Ross + University of Florida Department of Electrical and Computer Engineering Jeremy Lau* Google Inc. Brad Calder* Microsoft Corporation *This work was done while the author was affiliated with the University of California, San Diego

2 of 14 2 Cache Power Consumption Memory access: 50% of embedded processor’s system power –Caches are power hungry ARM920T (Segars 01) M*CORE (Lee/Moyer/Arends 99) Thus, caches are a good candidate for optimizations Different applications have vastly different cache requirements –Total size, line size, associativity 4KB 16 byte, 2-way 2KB 32 byte direct-mapped 8KB 64 byte, 4-way

3 of 14 3 Configurable Caches Even hard processors contain configurable caches –Specialized software instructions can change cache parameters –Specialized hardware enables the cache to be configured at startup or in system during runtime Motorola M*CORE – Malik ISLPED’00, Albonesi MICRO’00, Zhang ISCA’03 2KB 8 KB, 4-way base cache 2KB 8 KB, 2-way 2KB 8 KB, direct- mapped Way concatenation 2KB 4 KB, 2-way 2KB 2 KB, direct- mapped Way shutdown Configurable Line size 16 byte physical line size Tunable cache Tuning hw

4 of 14 4 Cache Tuning Cache tuning is the process of determining the appropriate cache parameters for an application –Requires a tunable cache Cache parameter values can be varied during runtime –Requires tuning hardware Orchestrates cache tuning Energy Executing in base configuration Tunable cache Tuning hw TC Cache Tuning TC Download application Microprocessor Cache energy savings of 62% on average!

5 of 14 5 Phase-Based Cache Tuning However, applications show varying operating requirements throughout execution Greater energy savings potential if the cache can be tuned for each one of these phases Time varying behavior for IPC, level one data cache hits, branch predictor hits, and power consumption for gcc (using the integrate input set) Base cache energy Application-tuned Time Energy Consumption Phase-tuned Change cache Need a method to detect phase changes during runtime

6 of 14 6 Phase Classification Break application into fixed sized intervals –Intervals measured in dynamic instructions executed Group intervals with similar characteristics as the same phase Optimizations applied to one interval of a phase will work equally well with every other interval of the same phase

7 of 14 7 Phase Prediction Predict when a phase transition will occur and which phase will be entered Uses two predictors: –Set of phases leading up to the next phase –Duration of time spent in phases Benefit for cache tuning –Can determine best configuration for each phase, save that configuration, and then change directly to it when the phase is predicted

8 of 14 8 Experimental Results Examined a large selection of SPEC2000 Integer and Floating point benchmarks Phase classified entire benchmark Determined best cache configuration for each phase Modified SimpleScalar with configurable cache Executed benchmarks in their entirety with SimpleScalar to gather cache hit and miss statistics

9 of 14 9 Phase-Based Tuning Methodology Phase Classification

10 of Results - Energy Consumption Note: Avg modified averages only the benchmarks were phase-based tuning is favorable

11 of Results - Performance Note: Avg modified averages only the benchmarks were phase-based tuning is favorable

12 of Results - Energy Savings Energy savings compared to previous phase-based tuning techniques

13 of Design Space Exploration Speedup

14 of Conclusions Phase-based cache tuning for a highly configurable cache –1800x greater configurability compared to previous methods Comparable energy savings to application-based tuning –8% greater savings on average 8x speedup in design space exploration time 17% additional energy savings compared to previous methods