1. COSC6385 Advanced Computer Architecture
Lecture 4. Performance
Instructor: Weidong Shi (Larry), PhD
Computer Science Department
University of Houston

2. Topics
Performance

3. CPU Performance Execution Time = Seconds / Program Microarchitecture
System architecture
Microarchitecture, pipeline depth
Circuit design
Technology
Programmer
Algorithms
ISA
Compilers

4. Architecture Comparison
Many architecture research just make the following assumptions
Instructions / program is fixed
Same binary ()
Same compiler ()
Same benchmark
Seconds per cycle is constant ()
Same frequency
Same pipeline depth
Typically a bad assumption today
Focus on IPC or CPI
It is more complicated for today’s architects !

5. Example: Calculating CPI
Run benchmark and collect workload characterization (simulate, machine counters, or sampling)
Base Machine (Reg / Reg)
Op Freq Cycles CPI(i) (% Time)
ALU 50% (33%)
Load 20% (27%)
Store 10% (13%)
Branch 20% (27%)
1.5
Typical Mix of
instruction types
in program
Design guideline: Make the common case fast
MIPS 1% rule: only consider adding an instruction of it is shown to add 1% performance improvement on reasonable benchmarks.

6. Performance Comparison
For some program running on machine X, PerformanceX = 1 / Execution timeX
"X is n times faster than Y" PerformanceX / PerformanceY = n = speedup of X over Y
Problem:
machine A runs a program in 20 seconds
machine B runs the same program in 25 seconds

7. Performance Evaluation: Benchmark
(Real) Programs
In the form of collection of programs
E.g., SPEC, Winstone, SYSMARK, 3D Winbench, EEMBC
Kernels:
Small key pieces of real programs
E.g., Livermore Fortran Loops Kernels (LFK), Linpack
Modified (or scripted)
To focus on some particular aspects (e.g. remove I/O, focus on CPU)
(Toy) Benchmarks
Produce expected results
Synthetic Benchmarks:
Representative instruction mix
Important for
Architectural and microarchitectural design trade-off
Competitive analysis of real products

8. Performance Summary Measurement
Average of total execution time
This is Arithmetic Mean (Weighted Arithmetic Mean)

9. Performance Summary Measurement
Ratei is a function of 1/Timei
Used to represent the average “rate” such as instruction per cycle (IPC)

10. Why Harmonic Mean? 30 mph for the first 10 miles
90 mph for the next 10 miles
Average speed? (30+90)/2 = 60 mph??
Wrong!
Average speed = total distance / total time
(10+10)/(10/ /90) = 45 mph

11. New Breed of Metrics Performance / Watt Performance / Joule (Energy)
Performance achievable at the same cooling capacity
Performance / Joule (Energy)
Achievable performance at the lifetime of the same energy source (i.e., battery = energy)

13. Amdahl’s Law (Law of Diminishing Returns)
Make the common case faster
Speedup
= Perfnew / Perfold = Told / Tnew =
Performance improvement from using faster mode is limited by the fraction the faster mode can be applied. f
(1 - f)
Told
(1 - f)
Tnew
f / P

14. Amdahl’s Law Accelerate HD encoding using GPU
Assume 50% time spent on ME
Overall performance speedup
ME offloaded to GPU, F = 50%
The rest on CPU

15. Amdahl’s Law Analogy – A Driving Metaphor
Driving from Houston to South Padre Island
60 miles/hr from Houston to Kingsville
120 miles/hr from Kingsville to South Padre Island
How much time you can save compared against driving all the way at 60 miles/hr from Houston to Padre Island?
about 5hr 10min vs. 6hr 10min
Key is to speed up the biggie portion,
i.e. speed up frequently executed blocks

16. Parallelism vs. Speedup1 10
100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Speed-up
Code portion in Faster mode (f)
Amdahl's Law speed-up as a function of parallelism
P=1
P=2
P=4
P=8
P=16
P=32
P=64
1.97x
1.11x
1.33x

17. Gustafson’s Law Seq Parallel Seq P * Parallel Time 
Amdahl’s Law killed massive parallel processing (MPP)
Gustafson came to rescue
Seq
Tnew
Parallel
Told
Seq
P * Parallel Time
Assume: Seq + Parallel = (Tnew) 
Speedup = Seq + p * (1 – Seq) where p=parallel factor
If Seq diminishes with increased problem size, Speedup  p

19. Gustafson’s Law 50mph 100 miles 100mph 2nd 100 miles 66.67mph 50mph
3rd 100 miles
75mph
50mph
100 miles
100mph
2nd 100 miles
100mph
3rd 100 miles
100mph
4th 100 miles
80mph
Suppose a car has already been travelling for some time at speed of less than 50km/h, and when given enough time and distance to travel, the car’s average speed can reach 100km/h as long as it drives faster than 100 km/h for some time. And also the average speed can reach 120km/h and even 150km/h as long as it drives fast enough in the following part

20. Amdahl versus Gustafson
Who is right?

21. Amdahl versus Gustafson
Amdahl’s presumption of fixed data size
Both laws are in fact different perspective over the same truth – one sees data size as fixed and the other sees the relation as a function of data size

22. Big Data NASA Climate Simulation The Large Hadron Collider Wall mart
32 petabytes
The Large Hadron Collider
25 petabytes annually
Wall mart
2.5 petabytes per hour

23. The Principle of Locality
Knuth made the original observation about program locality in 1971.
… less than 4 percent of a program generally accounts for more than half of its running time.
90/10 rule: a program spends 90% of its execution time in only 10% of the code
Two types of locality
Temporal locality (locality in time)
Spatial locality (locality in space)
Memory subsystem design heavily leverages the locality concept for better performance

24. Example of Performance Evaluation (I)
Operation
Frequency
Clock cycle count
ALU Ops (reg-reg)
43% 1
Loads
21% 2
Stores
12%
Branches
24%
Assume 25% of the ALU ops directly use a loaded operand
that is not used again. We propose adding ALU instructions
that have one src operand in memory. These new reg-mem
instructions spend 2 clock cycles. Also assume that the
extended instruction set increase branch’s clock by 1,
but no impact to cycle time.
Would this change improve performance ?

25. Example of Performance Evaluation (I)
Operation
Frequency
Clock cycle count
ALU Ops (reg-reg)
43% 1
Loads
21% 2
Stores
12%
Branches
24%
Assume 25% of the ALU ops directly use a loaded operand
that is not used again. We propose adding ALU instructions
that have one src operand in memory. These new reg-mem
instructions spend 2 clock cycles. Also assume that the
extended instruction set increase branch’s clock by 1,
but no impact to cycle time.
Would this change improve performance ?

26. Example of Performance Evaluation (II)
FP instructions = 25%
Average CPI of FP instructions = 4.0
Average CPI of other instructions = 1.33
FPSQRT = 2% of all instructions, CPI of FPSQRT = 20
Design Option 1: decrease the CPI of FQSQRT to 2
Design Option 2: decease the average CPI of all FP instructions to 2.5

27. Example of Performance Evaluation (II)
FP instructions = 25%
Average CPI of FP instructions = 4.0
Average CPI of other instructions = 1.33
FPSQRT = 2% of all instructions, CPI of FPSQRT = 20
Design Option 1: decrease the CPI of FPSQRT to 2
Design Option 2: decease the average CPI of all FP instructions to 2.5
Original CPI = 0.25* *(1-0.25) = 2.0
Option 1 CPI = 2.0 – 2%*(20-2) = 1.64
Option 2 CPI = 0.25* *(1-0.25) = 1.625
Speedup of Option 1 = 2/ =
Speedup of Option 2 = 2/1.625 =

28. Example of Performance Evaluation (III)
Clock freq = 1.4 GHz
FP insturctionss = 25%
Average CPI of FP instructions = 4.0
Average CPI of other instructions = 1.33
FPSQRT = 2%, CPI of FPSQRT = 20
Design Option 1: decrease the CPI of FPSQRT to 2, clock freq = 1.2GHz
Design Option 2: decease the average CPI of all FP instructions to 2.5, clock freq = 1.1 GHz

29. Example of Performance Evaluation (III)
Clock freq = 1.4 GHz
FP insturctionss = 25%
Average CPI of FP instructions = 4.0
Average CPI of other instructions = 1.33
FPSQRT = 2%, CPI of FPSQRT = 20
Design Option 1: decrease the CPI of FPSQRT to 2, clock freq = 1.2GHz
Design Option 2: decease the average CPI of all FP instructions to 2.5, clock freq = 1.1 GHz
Original CPI = 2.0, IPC = 1/2, Inst/Sec = ½*1.4G = 0.7G inst/s
Option 1 CPI = 1.64, IPC = 1/1.64, Inst/Sec = 1/1.64*1.2G = 0.73G inst/s
Option 2 CPI = 1.625, IPC = 1/1.625, Inst/Sec = 1/1.625*1.1G = 0.68G inst/s