1. CSCE 212 Chapter 4: Assessing and Understanding Performance
Instructor: Jason D. Bakos

2. Measuring Performance
Basic problem: how can we tell if one processor is faster than another?
Execution time is analogous to performance
Execution time (per processor core) =

3. Example
A program runs in 10 seconds on computer A, which has a 4 GHz clock. We are designing computer B that must run the program in 6 seconds. We can increase clock rate but the computer will require 1.2 times as many clock cycles. What clock rate should be target for computer B?

4. Performance Factors Execution time = Problems: Performance factors:
instructions per program * cycles per instruction * seconds per cycle
Problems:
Cycles per instruction depends on many factors
i.e. instruction mix, I/O, memory
Performance factors:
Algorithm  instruction count, CPI
Programming language  instruction count, CPI
Compiler  instruction count, CPI
ISA  instruction count, clock rate, CPI

5. Performance Measures CPI = cycles per instruction
CPI provides one way of comparing two different implementations of the same instruction set architecture
CPI is tricky!
Instructions from the same ISA require different number of cycles
Depends on program
Memory behavior affects CPI
CPI is usually defined for a particular program on a particular CPU

6. CPI instruction class A B C CPI 1 2 3 Code sequence Instruction counts4

7. Example
Suppose we have two implementations of the same ISA. Computer A has a cycle time of 250 ps and a CPI of 2.0 for some program, and computer B has a clock cycle time of 500 ps and a CPI of 1.2 for the same program. Which computer is faster and by how much?

8. Performance Measures Program execution time vs CPU execution time
Must account for other programs and I/O
User CPU time
CPU time spent executing a program
System CPU time
Amount of CPU time the operating system spends on behalf of a program
Ex. system calls, such as I/O, synchronization, etc.

9. Relative Performance Solutions: Total execution time Arithmetic mean
Computer A
Computer B
Program 1 (seconds) 1 10
Program 2 (seconds)
1000
100
Solutions:
Total execution time
Arithmetic mean
Weight arithmetic mean (based on workload for each program)

10. MIPS Instructions per second MIPS is problematic
Does not take into account the capabilities of the instructions
MIPS varies between programs on the same computer!
Depends on average CPI of the instruction mix
MIPS can vary inversely with performance

11. Benchmarks
Set of programs that are used to evaluate a machine’s performance
SPEC CPU 2000 suite
12 integer and 14 floating-point
Measured as speedup relative to Sun Ultra MHz
CINT2000 and CFP2000 is geometric mean of ratios

12. Amdahl’s Law Basic idea: improve the common case Example:
Program runs in 100 s, multiplies accord for 80 s
I want a 5X overall speedup