1. ECE 692 20091 Power Control for Chip Multiprocessors Xue Li Oct 27, 2009

2. ECE 692 20092 Outline Two ways to control power of chip multiprocessors –MPC control with online model estimation –Simple closed loop control with risk evaluation

3. ECE 692 20093 Temperature-Constrained Power Control for Chip Multiprocessors with Online Model Estimation Yefu Wang, Kai Ma, Xiaorui Wang

4. ECE 692 20094 Introduction Power and thermal are the major constraints for further throughput improvement of CMP –Peak power consumption of a CMP should be controlled to enable higher computing densities. –The temperature of a CMP should be kept lower than a threshold in case of thermal failures. –Performance delivered per watt needs to be maximized.

5. ECE 692 20095 State of the Art Power control for CMP –Open-loop search or optimization [Isci’06], [Teodorescu’08], etc. Highly dependent on the accuracy of the system model –Heuristics [Isci’06], [Meng’08], etc. No theoretical guarantee of control accuracy/stability –Chip-wide DVFS (Dynamic Voltage and Frequency Scaling) [McGowen’06], [Floyd’07], etc. Suboptimal in performance Dynamic thermal management –Heuristics or feedback control theory [Brooks’01], [Skadron’03], etc. Power and temperature are controlled separately

6. ECE 692 20096 Challenges and Solutions Multiple cores may need to be manipulated simultaneously to control both power and temperature. Multi-Input-Multi-Output (MIMO) control Optimal control algorithms need to be designed for power shifting among different cores. Model predictive control (MPC) theory Different cores may be coupled together. Specific design constraints Workload is unpredictable at design time. Online parameter estimation Control accuracy and system stability is critical Theoretically guaranteed control performance and stability

7. ECE 692 20097 Temperature-Constrained Power Control MIMO control loop invoked periodically –Power monitor sends the chip-level power consumption to the controller –Controller reads temperature and performance metrics of each core –Controller computes new DVFS levels based on MPC control theory –New per-core DVFS levels are sent to the cores –Online model estimator updates the power model

8. ECE 692 20098 Steps of Model Predictive Control System modeling –Power model Controller design –MPC controller design –Constrains: Frequency range Power budget Temperature Other design requirements System stability analysis

9. ECE 692 20099 Steps of Model Predictive Control System modelingSystem modeling –Power model Controller design –MPC controller design –Constrains: Frequency range Power budget Temperature Other design requirements System stability analysis

10. ECE 692 200910 System Modeling: Power Model (1) Power consumption of one core A Estimated system parameters Initial value can be defined by system identification May change for different workloads Can be updated by online estimation

11. ECE 692 200911 System Modeling: Power Model (2) Total power consumption of CMP Power model validation Total power consumption of the chip

12. ECE 692 200912 Steps of Model Predictive Control System modeling –Power model Controller designController design –MPC controller design –Constrains: Frequency rangeFrequency range Power budgetPower budget TemperatureTemperature Other design requirementsOther design requirements System stability analysis

13. ECE 692 200913 Controller Design: MPC Controller Control objective: minimize the cost function Control accuracyPerformance optimization Model prediction Measured from power meter: feedback

14. ECE 692 200914 Controller Design: Constraints (1) Physical frequency range Power budget for each core Other design requirements

15. ECE 692 200915 Controller Design: Constraints (2) Model between temperature and frequency –Temperature & power –Power & frequency Temperature constraint

16. ECE 692 200916 Steps of Model Predictive Control System modeling –Power model Controller design –MPC controller design –Constrains: frequency range power budget temperature other design requirements System stability analysisSystem stability analysis

17. ECE 692 200917 Controller Design: Stability Analysis Stability: –Converge to desired bounds from any initial condition Unknown system gain: –Actual system parameter, estimating system parameter –The bigger range, the better system adaptability. The system is proved to be stable in a wide range –Uniform workload 0< g ≤ 8.83 –Different workload 0 < g 1 ≤ 15.7 0 < g 2 ≤ 17.6 The model can work as long as the real parameter of a system is less than 8.83 times of the value used to design the system.

18. ECE 692 200918 Online Model Estimation Recursive Least Square (RLS) estimator to update the model periodically –RLS estimator recordsand –The estimator calculates and –The estimator updates with in the system model

19. ECE 692 200919 System Implementation Power lines (Current signal) Current probe (1mv/A) USB interface Physical TestbedSimulation CPUIntel Xeon X5365Alpha 21264 like Cores44, 8, 16 Power MonitorDigital MultiMeterWattch Temp SensorCoretemp driver ControllerSoftware WorkloadSPEC CPU 2006

20. ECE 692 200920 Experimentation Baselines Empirical results –Control accuracy –Application performance –Temperature constraints –Online model estimator Simulation results –Control accuracy –Application performance

21. ECE 692 200921 Experimentation Baselines Empirical results –Control accuracy –Application performance –Temperature constraints –Online model estimator Simulation results –Control accuracy –Application performance

22. ECE 692 200922 Baselines Priority –Per-core DVFS –Heuristic based Power > budget DVFS decreases by 1 Power < budget DVFS increases by 1 Improved priority –Priority with safety margin MaxBIPS –Per-core DVFS –Predictive based: uses a typical workload to build a static table offline –Exhaustive search from combination of DVFS levels for all cores Workload sensitive

23. ECE 692 200923 Baselines: MaxBIPS Define two N*M matrices: Power and BIPS –N: number of cores –M: number of power modes Fill in the matrices with actual and predictive values –Power: cubic scaling –BIPS: linear scaling Find out the power and core combination to achieve best BIPS under power budget Core1Core2 Mode12016 Mode21714 Core1Core2 Mode18060 Mode26952 Power Matrix BIPS Matrix Actual value ModePower Savings Performance Degradation Mode1None Mode215%5% BIPSPower 80+60=14020+16=36 80+52=13220+14=34 69+60=12917+16=33 69+52=12117+14=31 If power budget is 32, last one will be selected

24. ECE 692 200924 Experimentation Baselines Empirical results –Control accuracy –Application performance –Temperature constraints –Online model estimator Simulation results –Control accuracy –Application performance

25. ECE 692 200925 Empirical Results: Control Accuracy (1) Comparison of steady state errors –Steady state error: violation of power budget at different power level. –MPC follows the set point well.

26. ECE 692 200926 Empirical Results: Control Accuracy (2) MPC V.S. MaxBIPS / Priority / Improved Priority Much lower than the set point Fits well Oscillates around the set point Exceeds the budget at times

27. ECE 692 200927 Empirical Results: Application Performance SPEC performance between MPC, MaxBIPS and improved priority under different power budgets. –MPC achieves better performance because MPC can precisely achieve the set-point power. –Average improvement of MPC is 9.69% over MaxBIPS and 8.95% over Improved Priority.

28. ECE 692 200928 Empirical Results: Temperature Constraints Emulate a thermal emergency by lowering the temperature constraint –Figure (a) shows that the temperature of cores are quickly constrained to the lower bound. –Figure (b) shows that the temperature constraints works effectively to reduce power consumption.

29. ECE 692 200929 Empirical Results: Online Model Estimator MPC V.S. MPC with estimator –Workload may change significantly at run time. –Estimator can correct system parameters dynamically. –MPC without estimator suffers large oscillations.

30. ECE 692 200930 Experimentation Baselines Empirical results –Control accuracy –Application performance –Temperature constraints –Online model estimator Simulation results –Control accuracy –Application performance

31. ECE 692 200931 Simulation Results: Control Accuracy Simulation with more cores (4, 8, 16) –Average power and standard deviation of different control method. MPC precisely converges to the budget. MaxBIPS’ absence of 16 due to exponentially increase of static prediction table

32. ECE 692 200932 Simulation Results: Application Performance SPEC benchmark performance comparison under different number of cores (Set point = 95%, 85%)

33. ECE 692 200933 Conclusion A temperature-constrained chip-level power controller –Designed based on MPC control theory –Accurately controls power consumption –Temperatures of the cores are limited to stay below the constraint. –An online model estimator periodically updates the system model Compared with state-of-the-art work –More accurate power control –Better application performance

34. ECE 692 200934 Multi-Optimization Power Management for Chip Multiprocessors Ke Meng, Russ Joseph, Robert P. Dick Northwestern University Li Shang University of Colorado

35. ECE 692 200935 Introduction Power is still a first-class design constraint in CMP era. –Higher transistor density –Higher leakage power Power is still a precious computing resource –When power is limited, maximizing the chip- wide performance requires global and local coordination. High power density Thermal Issues

36. ECE 692 200936 System Framework Select power optimizations and allowable power modes Collect data from sensors and counters; calculate power /performance. Analyze, search and tune Soft-limit budget

37. ECE 692 200937 Optimization Pool (1) DVFS –Simple models Frequency: linear with voltage Power: changes cubically with voltage Performance: roughly linear with frequency –High efficiency Cubical relationship between frequency and power

38. ECE 692 200938 Optimization Pool (2) Cache resizing –Large leakage: big savings –Workload variety: unused private capacity

39. ECE 692 200939 Models and Experimentations Models –Dynamic voltage / frequency scaling (DVFS) –Cache resizing –Unified analytic models –Risk evaluation –Search algorithms Experimentation –Configuration –Model validation –Model evaluation –Power violation

40. ECE 692 200940 Models and Experimentations ModelsModels –Dynamic voltage / frequency scaling (DVFS) –Cache resizing –Unified analytic models –Risk evaluation –Search algorithms Experimentation –Configuration –Model validation –Model evaluation –Power violation

41. ECE 692 200941 Analytic Models: DVFS DVFS modeling –CPI stack counters: counts computing stalls and L2 miss stalls Computing stalls: changes with frequency L2 miss stalls: constant in spite of frequency –Performance model Power: Cubic with frequency

42. ECE 692 200942 Analytic Models: Cache Resizing Cache resizing modeling –Non-stall cycles –Stall cycles due to cache misses Power: Average leakage power of a cache way times number of active ways

43. ECE 692 200943 Analytic Models: Unification Unified analytic models with DVFS and cache resizing –Performance Weak interaction among multiple optimization allow independent speed-ups –Power DVFS has a strong influence Additive contribution of cache resizing

44. ECE 692 200944 Analytic Models: Risk Evaluation Why to do risk evaluation? –Some optimizations are more prone to phase adjustment. –Severe performance loss and power violation. How to do risk evaluation? –DVFS: assume zero risk. –Cache resizing: cache activities variation threshold.

45. ECE 692 200945 Brute-force search –Traverse all possible power modes –Always find the best combination –Slow when search space are large Greedy search –Take currently best step available Current best step: power mode with the maximal delta power/performance ratio. –Fast –Can get stuck in local minima Results show it happens rarely Analytic Models: Search Algorithms(1)

46. ECE 692 200946 Models and Experimentations Models –Dynamic voltage / frequency scaling (DVFS) –Cache resizing –Unified analytic models –Risk evaluation –Search algorithms ExperimentationExperimentation –Configuration –Model validation –Model evaluation –Power violation

47. ECE 692 200947 Experiment: Configuration Processor Setup Cores4 Alpha21264-like cores L1 I/D Cache64KB 2-way private 64B blocks L2 Cache2MB 8-way private 128B blocks Tech node65 nm DVFS Range85%, 90%, 95%, 100% of 3GHz Group No.WorkloadsStability Group Aequake, swim, sixtrack, gccModerate Group Bapplu, gap, facerec, vortexModerate Group Cmesa, eon, lucas, wupwiseStable Group Dart, mcf, parser, vprUn-stable

48. ECE 692 200948 Experiment: Model Validation Cache CPI model validation

49. ECE 692 200949 Experiment: Model Evaluation (1) Modeling-greedy vs. modeling- global / trial-and-error –Trial-and-error (DVFS + cache resizing): Starting trial-stage when entering a stable phase Only works with workloads possessing stable phases (Group C). –Analytical modeling (DVFS + cache resizing): 8% perf loss vs. 35% power saving Greedy search works extremely well

50. ECE 692 200950 Experiment: Model Evaluation (2) Modeling with risk management vs. MaxBIPS –Simple (DVFS + cache resizing): Analytical modeling without risk evaluation. –With risk evaluation: Results either better or almost unchanged. –MaxBIPS (only DVFS): Not always the worst. Difficult to manage multiple optimizations Even with risk evaluation, errors can be made before risk being identified.

51. ECE 692 200951 Conclusion Power problem is critical in CMP. CMP power management must coordinate global and local power usage. Analytical modeling are more favorable than trial-and-error. Risk evaluation is necessary to avoid frequent prediction errors.

52. ECE 692 200952 Comparison 1 st paper2 nd Paper ControllingPredictive basedHeuristic based Power budgetHard limitSoft limit Temperature management YesNo L2 cache involvement NoYes Hardware implementation YesNo

53. ECE 692 200953 Critiques First paper –Temperature constraint seems much higher than normal working condition. –Explanation of in temperature constraints is not very clear. Second paper –Modeling accuracy is low. –No absolute guarantee of power consumption. –Too many arbitrary assumptions.

54. ECE 692 200954 Thank you