1. Power Management in Multicores Minshu Zhao

2. Outline Introduction Review of Power management technique Power management in Multicore ◦ Identify Multicores Characteristics ◦ Apply power management technique Future of multicore

3. Review on low power technique Clock gating ◦ + Gating can be done on fine grained ◦ + Save dynamic power ◦ - Not affect static power Power Gating ◦ + save both dynamic and static power ◦ - need microseconds to power up again ◦ - lost data or need some form of state retention FF EN CK FF Vdd EN

4. Review on low power technique Voltage (Frequency) Scaling ◦ Scale down frequency and/or voltage, sacrifice performance for power  I ∝ (Vdd-Vt) ~ Vdd  f ∝ Vdd  P ∝ CV 2 f ∝ V 3 Variable device threshold ◦ Use high vt transistor to reduce leakage ◦ + reduce leakage ◦ - vt is generally fixed for one transistor

5. Outline Introduction Review of Power management technique Power management in Multicore ◦ Identify Multicores Characteristics ◦ Apply power management technique Future of multicore

6. Identify Multicore Characteristics Half of the chip is cores ◦ Large dynamic power ◦ Unbalanced power consumption among cores Another Half of the chip is Cache ◦ Large Leakage Power

7. Outline Introduction Review of Power management technique Power management in Multicore ◦ Identify Multicores Characteristics ◦ Apply power management technique  To Cores  To Caches Future of multicore

8. Traditional DVFS Motivation ◦ Large Computation/Memory Gap Problems to apply to multi-core ◦ Slow  Microsecond timescales ◦ Coarse-grained adjustment  In operating system ◦ All cores arrive at a single chip-wide VF setting  Lose potential power saving Power supply Off-chip regulator Core0 Core1 Core2 Core3

9. Per-core DVFS & on-chip regulator On-chip vs. off-chip regulator ◦ Tens of nanoseconds vs. microseconds Per-Core vs. Chip- Wide DVFS ◦ Benefit heterogeneous workload Power supply Off-chip regulator Core0 Core1 Core2 Core3 On-chip Regulator Wonyoung Kim; Gupta, M.S.; Gu-Yeon Wei; Brooks, D.;, "System level analysis of fast, per-core DVFS using on-chip switching regulators," High Performance Computer Architecture, 2008. HPCA 2008.

10. Per-core DVFS & on-chip regulator Application ◦ Multi-Core Global Power Management  Monitor power & performance  Apply policies by per-core DVFS Problem ◦ Overhead is large

11. Thread Motion Cores have different Voltage-Frequency setting Migrate thread between cores Apply DVFS benefits to program variability by observe micro architectural events Fast movement create effective voltage level Krishna K. Rangan, Gu-Yeon Wei, and David Brooks. 2009. Thread motion: fine- grained power management for multi-core systems. In Proceedings of the 36th annual international symposium on Computer architecture (ISCA '09). High-VF Low-VF

12. Thread Motion Application ◦ Thread Motion Framework  Evaluation driven by micro architectural events  Time-driven  Miss-driven  Predict IPC for the next interval  Move thread if needed Problem ◦ Potential Cache penalty  Clustered multicore with shared L1 cache within cluster ◦ Register file transfer penalty  Store them in the shared cache

13. Heterogeneous Cores Motivation ◦ Different applications have different resource requirements  Large ILP -> VLIW ◦ Different Power conditions  full battery vs. low battery Combine existing processor architecture and do core-selection to minimize energy Rakesh Kumar, Dean M. Tullsen, Parthasarathy Ranganathan, Norman P. Jouppi, and Keith I. Farkas. 2004. Single-ISA Heterogeneous Multi-Core Architectures for Multithreaded Workload Performance. In Proceedings of the 31st annual international symposium on Computer architecture (ISCA '04).

14. Outline Introduction Review of Power management technique Power management in Multicore ◦ Identify Multicores Characteristics ◦ Apply power management technique  To Cores  To Caches Future of multicore

15. Gated-Vdd cache Use high- Vt transistor to turn off power supply + reduce power when turn off - data stored in low power mode are lost Michael Powell, Se-Hyun Yang, Babak Falsafi, Kaushik Roy, and T. N. Vijaykumar. 2000. Gated-Vdd: a circuit technique to reduce leakage in deep-submicron cache memories. In Proceedings of the 2000 international symposium on Low power electronics and design (ISLPED '00). ACM, New York, NY, USA, 90-95. SRAM CELL Vdd Gnd Gated-vdd control

16. Gated-Vdd cache Application ◦ Dynamically resizable i-cache  Evaluate miss rate at every time interval and upsize/downsize the cache using gated-vdd Problem ◦ Data remapping on the fly Yang, S.; Powell, M.D.; Falsafi, B.; Roy, K.; Vijaykumar, T.N.;, "An integrated circuit/architecture approach to reducing leakage in deep-submicron high- performance I-caches," High-Performance Computer Architecture, 2001. HPCA.

17. Gated-Vdd cache Application ◦ Cache Decay  Turn a cache line off if some cycles elapsed since last access  The decay interval can be adaptive to the program Problem ◦ Data lost in sleep cache line, suffer cache miss Kaxiras, S.; Zhigang Hu; Martonosi, M.;, "Cache decay: exploiting generational behavior to reduce cache leakage power," Computer Architecture, 2001. Proceedings. 28th Annual International Symposium on, vol., no., pp.240-251, 2001

18. ABB-Multi-threshold CMOS Increase Vsb in the sleep mode Effectively increase vth to reduce leakage + State Preserved in sleep mode - Need long time to switch from sleep K. Nii, et. al. A low power SRAM using auto-backgate-controlled MT-CMOS. Proc. of Int. Symp. Low Power Electronics and Design, 1998, pp. 293-298. 1.0V 0V 1.0V / 3.3V 0V / 1.0V

19. Drowsy Caches Apply DVFS to Cache + Waking up cost is small + State preserve - Save not as much leakage power Krisztián Flautner, Nam Sung Kim, Steve Martin, David Blaauw, and Trevor Mudge. 2002. Drowsy caches: simple techniques for reducing leakage power. In Proceedings of the 29th annual international symposium on Computer architecture (ISCA '02). IEEE Computer Society, Washington, DC, USA, 148-157. Vdd 1V 0.3V drowsy SRAM CELL

20. Drowsy Caches Application ◦ Simple policy  Put all lines into sleep periodically and wake up afterwards ◦ No-access policy  Put the lines which is not access in the window in sleep ◦ 90% of the lines can be drowsy mode Problem Normalized total energy Normalized leakage energy Run time increase Avg0.460.290.41% Drowsy cacheGated-Vdd Leakage power6.24nW0.02nW

21. Outline Introduction Review of Power management technique Power management in Multicore ◦ Identify Multicores Characteristics ◦ Apply power management technique Future of multicore

22. Future multicore Dark silicon (transistor under-utilization) ◦ Power constraints  Power down the transistor to reduce power ◦ Memory wall  Waiting for the memory to continue computation ◦ Lack of parallelism  Do not have enough work for transistor Hadi Esmaeilzadeh, Emily Blem, Renee St. Amant, Karthikeyan Sankaralingam, and Doug Burger. 2011. Dark silicon and the end of multicore scaling. In Proceeding of the 38th annual international symposium on Computer architecture (ISCA '11).

23. Future multicore Power constraints ◦ New Device– FinFET Memory wall ◦ New Technology – 3D IC Lack of parallelism ◦ Auto parallization

24. Thank you !