1. Energy and Power
Lecture notes S. Yalamanchili and S. Mukhopadhyay

2. Some Useful Reading
ation
itching_and_leakage
ore-i5-2500t-2390t-i3-2100t-pentium- g620t.html

3. Historical Scaling

4. Technology Scaling
GATE
GATE
SOURCE
BODY
DRAIN
SOURCE
DRAIN
tox L
30% scaling down in dimensions  doubles transistor density
Power per transistor
Vdd scaling  lower power
Transistor delay = Cgate Vdd/ISAT
Cgate, Vdd scaling  lower delay

5. Fundamental Trends
High Volume Manufacturing
2004
2006
2008
2010
2012
2014
2016
2018
Technology Node (nm) 90 65 45 32 22 16 11 8
Integration Capacity (BT) 2 4 64
128
256
Delay = CV/I scaling
0.7
~0.7
>0.7
Delay scaling will slow down
Energy/Logic Op scaling
>0.35
>0.5
Energy scaling will slow down
Bulk Planar CMOS
High Probability Low Probability
Alternate, 3G etc
Low Probability High Probability
Variability
Medium High Very High
ILD (K) ~3
<3
Reduce slowly towards 2-2.5
RC Delay 1
Metal Layers
6-7
7-8
8-9
0.5 to 1 layer per generation
Source: Shekhar Borkar, Intel Corp.

6. ITRS Roadmap for Logic Devices
From: “ExaScale Computing Study: Technology Challenges in Achieving Exascale Systems,” P. Kogge, et.al, 2008

7. Where Does the Power Go in CMOS?
Dynamic Power Consumption
Charging and discharging capacitance
Short Circuit Power
Short circuit path between supply rails during switching
Nominally 10%-20% of dynamic power and can be ignored for a first order analysis
Leakage
Leaky transistors

8. Dynamic Power
Dynamic power is used in charging and discharging the capacitances in the CMOS circuit.
Time
VDD
Voltage T
Output Capacitor Charging
Output Capacitor Discharging
Input to CMOS inverter
iDD CL
PDYNAMIC = CL x VDD x VDD x Frequency

9. Static Power
Technology scaling has caused transistors to become smaller and smaller. As a result, static power has become a substantial portion of the total power.
Gate Leakage
Junction Leakage
Sub-threshold Leakage
Input = 0
Output = VDD
PSTATIC = VDD x ISTATIC

10. Energy-Delay Interaction
EDP
Energy or delay
VDD
VDD
Delay decreases with supply voltage but energy/power increases

11. Static Energy-Delay Interaction
leakage or delay
Vth
leakage
delay
tox
SOURCE
DRAIN L
GATE
Static energy increases exponentially with decrease in threshold voltage
Delay increases with threshold voltage

12. Same Energy = area under the curve
Power Vs. Energy P2
Power(watts) P1 P0
Same Energy = area under the curve
Time
Power(watts) P0
Time
Energy is a rate of expenditure of energy
One joule/sec = one watt
Both profiles use the same amount of energy at different rates or power

13. Optimizing Power vs. Energy
Maximize battery life  minimize energy
Thermal envelopes  minimize peak power

14. The Problem
Historically performance scaling was accompanied by power scaling
This is no longer true  power densities are increasing

15. The End of Dennard Scaling
GATE
SOURCE
DRAIN
tox L
Voltage is no longer scaling at the same rate
Slower scaling in power per transistor  increasing power densities
From R. Dennard, et al., “Design of ion-implanted MOSFETs with very small physical dimensions,” IEEE Journal of Solid State Circuits, vol. SC-9, no. 5, pp , Oct

16. Chip Power Densities
From: “ExaScale Computing Study: Technology Challenges in Achieving Exascale Systems,” P. Kogge, et.al, 2008

17. What is the Problem? Based on scaling using Pentium-class cores
Mukhopadhyay and Yalamanchili (2009)
Dark silicon
Based on scaling using Pentium-class cores
While Moore’s Law continues, scaling phenomena have changed
Power densities are increasing with each generation

18. The Power Wall
Power per transistor scales with frequency but also scales with Vdd
Lower Vdd can be compensated for with increased pipelining to keep throughput constant
Power per transistor is not same as power per area  power density is the problem!
Multiple units can be run at lower frequencies to keep throughput constant, while saving power

19. The Advent of Dark Silicon?
In-order core
Out of-order core
Cannot afford to turn on all devices at once
How do we manage the power and thermals?
64-core asymmetric chip multiprocessor layout and failure probability distribution

20. What are my Options? Better technology
Manufacturing
New Devices  non-CMOS?
Be more efficient – activity management
Clock gating
Power gating
Power management
Improved architecture
Simpler pipelines
Parallelism

21. Activity Management Clock Gating Power Gating
Vdd
Combinational Logic
clk
cond
input
Power gate transistor
Core 0
Core 1
Turn off clock to a block of logic
Eliminate unnecessary transitions/activity
Clock distribution power
Turn off power to a block of logic, e.g., core
No leakage

22. Power Management Software controlled power management
Optimize power and/or energy
Orchestrated by the operating system or application libraries
Industry standard interfaces for power management
Advanced Configuration and Power Interface (ACPI)
Hardware power management
Optimized power/energy
Failsafe operation, e.g., protect against thermal emergencies

23. Processor Power States
Performance States – P-states
Operate at different voltage/frequencies
Recall delay-voltage relationship
Lower voltage  lower leakage
Lower frequency  lower power (not the same as energy!)
Lower frequency  longer execution time
Idle States - C-states
Sleep states
Differ is how much state is saved
SW or HW managed transitions between states!

24. Multiple Voltage Frequency Domains
Intel Sandy Bridge Processor
Cores and ring in one DVFS domain
Graphics unit in another DVFS domain
Cores and portion of cache can be gated off
From E. Rotem et. Al. HotChips 2011

25. Power States
From:

26. Intel Sandy Bridge Processor
Power Gating
Turn off components that are not being used
Lose all state information
Costs of powering down
Costs of powering up
Smart shutdown
Models to guide decisions
Intel Sandy Bridge Processor

27. Simplify Core Design
AMD Bulldozer Core
Support for out of order execution, schedulers, branch prediction, etc. consumes more energy per instruction
Can fit many more simpler cores on a dies
ARM A7 Core (arm.com)

28. Parallelism and Power How much of the chip area is devoted to compute?
IBM Power5
AMD Trinity
Source: forwardthinking.pcmag.com
Source: IBM
How much of the chip area is devoted to compute?
Run many cores slower. Why does this reduce power?

29. Parallelism Concurrency + lower frequency  greater energy efficiency
Example
Core
Cache
Core
Cache
Core
Cache
4X #cores
0.75x voltage
0.5x Frequency
1X power
2X in performance
Core
Cache
Core
Cache

30. Microarchitectural Level Models
How can we study power consumption without building circuits?
Models
Models can are available at multiple levels of abstraction.
We are interested in microarchitectural models

31. Processor Microarchitecture
Fetch
Decode
Execute/Writeback
Register Files
ALU
MUL
Instruction Cache
Instruction Decoder
Fetch
Queue
Instruction Queue
FPU LD
Branch
Prediction
Instruction TLB ST
Data
TLB
L1 Data Cache
Network
Memory
On-Chip
Network
L2 Data Cache
NoC Router

32. Energy/Power Calculation
How do we calculate energy or power dissipation for a given microarchitecture?
Energy/Power varies between:
Different ISA; ARM vs Intel x86
Different microarchitecture; in-order vs out-of-order
Different applications; memory vs compute-bound
Different technologies; 90nm vs 22nm technology
Different operation conditions; frequency, temperature

33. Architecture Activity (1)
icache.read++;
fbuffer.write++;
Register Files
ALU
Activity 1: Instruction Fetch
MUL
Instruction Cache
Instruction Decoder
Fetch
Queue
Instruction Queue
FPU LD
Branch
Prediction
Instruction TLB ST
Collect activity counts of each architecture component (through simulation or measurement).
List of components differs between microarchitectures.
Activity counts at each component differs between applications.
Data
TLB
L1 Data Cache
On-Chip
Network
L2 Data Cache
NoC Router

34. Architecture Activity (2)
fbuffer.read++;
idecoder.logic++;
Register Files
ALU
Activity 2: Instruction Decode
MUL
Instruction Cache
Instruction Decoder
Fetch
Queue
Instruction Queue
FPU LD
Branch
Prediction
Instruction TLB ST
Read/write accesses to caches, buffers, etc.
Logical accesses to logic blocks such as decoder, ALUs, etc.
Tradeoff of differentiating more access types (accuracy) vs simulation speed (complexity).
Data
TLB
L1 Data Cache
On-Chip
Network
L2 Data Cache
NoC Router

35. Power and Architecture Activity
For example, At nth clock cycle, collected counters are:
Data cache:
read = 20, write = 12;
per-read energy = 0.5nJ; per-write energy = 0.6nJ;
Read energy = read*per-read energy = 10nJ
Write energy = write*per-write energy = 7.2nJ
Total activity energy = read+write energies = 17.2nJ
If n = 50th clock cycle and clock frequency = 2GHz, Total activity power = energy*clock_freq/n = 688mW
*Note: n/clock_freq = n clock periods in sec power = time average of energy

36. Things to consider (1) Circuit-level Estimation Tool
How do we calculate per-read/write energies?
Per-access energies can be estimated from circuit-level designs and analyses.
There are various open-source tools for this.
Architecture Specification
Technology Parameters
Circuit-level
Estimation Tool
Estimation Results:
Area, Energy, Timing, etc.

37. Things to consider (2) Is per-access energy always the same?
Per-access energy in fact depends on:
how many bits are switching
how they are switching (0→1 or 1→0)
It is reasonable to assume constant per-access energy in long-term observation (e.g., n = 1M clock cycles); the number of switching bits are averaged (e.g., 50% of bits are switching).
Most architecture simulators do not capture bit- level details due to simulation complexity.

38. Things to consider (3)
If a register file didn’t have read/write accesses but held data, what is the energy dissipation?
Energy (or power) is largely comprised of dynamic and static dissipations.
Dynamic (or switching) energy refers to energy dissipation due to switching activities.
Static (or leakage) energy is dissipation to keep the electronic system turned on.
In this case, the register file has no dynamic energy dissipation but consumes static energy.

39. Thermal Issues Heat can cause damage to the chip
Need failsafe operation
Thermal fields change the physical characteristics
Leakage current and therefore power increases
Delay increases
Device degradation becomes worse
Cooling solution determines the permitted power dissipation

40. Thermal Design Power (TDP)
This is the maximum power at which the part is designed to operate
Dictates the design of the cooling system
Max temperature  Tjmax
Typically fixed by worst case workload
Parts are typically operating below the TDP
Opportunities for turbo mode?
AMD Trinity APU

41. Trinity TDP
Source:

42. Exploiting the Physics
Most of time the part is operating well below its thermal limit
Leaving performance on the table
Can temporarily boost frequency (and therefore power dissipation) for short periods of time, e.g., seconds
Temperature changes slowly

43. Low power – build up thermal credits
Boosting
Intel Sandy Bridge
Exploit package physics
Temperature changes on the order of milliseconds
Use the thermal headroom
Turbo boost region
Max Power
TDP Power
10s of seconds
Low power – build up thermal credits

44. Conclusions
Power/energy is the leading driver of modern architecture design
Power and energy management is key to scalability
Need integrated power/energy, performance, thermal management in fielded systems
What about energy/power efficient algorithms?

45. Study Guide
Explain the difference between energy dissipation and power dissipation
Distinguish between static power dissipation and dynamic power dissipation
Be able to apply the simplified McPAT power model to a simple datapath and instruction sequence
Explain dynamic voltage frequency scaling
What are power states?
Why is this an advantage?
What is the impact of DVFS on i) energy, ii) execution time, and iii) power

46. Study Guide (cont.) How is thermal design power (TDP) calculated?
When using boost algorithms, what determines the duration of the high frequency operation?
How does a power virus work?
Describe how throttling works
Know the power dissipation in some modern processor-memory systems drawn from the embedded, server, and high performance computing segments