1. Lecture 2: Computer Performance
Professor Mike Schulte
Computer Architecture
ECE 201

2. Reminders
Office hours from 12:00 to 1:00 on Tuesday and Thursday in PL324
Course web page at
Download copies of Course Syllabus and Course Schedule
Finish reading Chapters 1 and 2 by next class period.
Fill out the Course Survey (Assignment 0)

3. Performance and Cost Purchasing perspective
given a collection of machines, which has the
best performance ?
least cost ?
best performance / cost ?
Design perspective
faced with design options, which has the
best performance improvement ?
Both require
basis for comparison
metric for evaluation
Our goal is to understand cost & performance implications of architectural choices

4. Two notions of “performance”
Plane
DC to Paris
6.5 hours
3 hours
Speed
610 mph
1350 mph
Passengers
470
132
Throughput (pmph)
286,700
178,200
Boeing 747
BAD/Sud Concodre
Which has higher performance?
° Time to do the task (Execution Time)
– execution time, response time, latency
° Tasks per day, hour, week, sec, ns. .. (Performance)
– performance, throughput, bandwidth
Response time and throughput often are in opposition - why?

5. Performance Definitions
Performance is in units of things-per-second
bigger is better
If we are primarily concerned with response time
performance(x) = execution_time(x)
" X is n times faster than Y" means
performance(X) execution_time(Y)
n = =
performance(Y) execution_time(X)
When is throughput more important than execution time?
When is execution time more important than throughput?

6. Peformance Examples Time of Concorde vs. Boeing 747?
Concord is 1350 mph / 610 mph = 2.2 times faster
= 6.5 hours / 3 hours
Throughput of Concorde vs. Boeing 747 ?
Concord is 178,200 pmph / 286,700 pmph = 0.62 “times faster”
Boeing is 286,700 pmph / 178,200 pmph = “times faster”
Boeing is 1.6 times (“60%”) faster in terms of throughput
Concord is 2.2 times (“120%”) faster in terms of flying time
When discussing processor performance, we will focus primarily on execution time for a single job - why?

7. Understanding Performance
How are the following likely to effect response time and throughput?
Increasing the clock speed of a given processor.
Increasing the number of jobs in a system (e.g., having a single computer service multiple users).
Increasing the number of processors in a sytem that uses multiple processors (e.g., a network of ATM machines).
If an Pentium III runs a program in 8 seconds and a PowerPC runs the same program in 10 seconds, how many times faster is the Pentium Pro?
n = 10 / 8 = 1.25 times faster (or 25% faster)
Why might someone chose to buy the PowerPC in this case?

8. Definitions of Time
Time can be defined in different ways, depending on what we are measuring:
Response time : Total time to complete a task, including time spent executing on the CPU, accessing disk and memory, waiting for I/O and other processes, and operating sytem overhead.
CPU execution time : Total time a CPU spends computing on a given task (excludes time for I/O or running other programs). This is also referred to as simply CPU time.
User CPU time : Total time CPU spends in the program
System CPU execution time : Total time operating sytems spends executing tasks for the program.
For example, a program may have a system CPU time of 22 sec., a user CPU time of 90 sec., a CPU execution time of 112 sec., and a response time of 162 sec..

9. The clock rate is the inverse of the clock cycle time.
Computer Clocks
A computer clock runs at a constant rate and determines when events take placed in hardware.
Clk
clock period
The clock cycle time is the amount of time for one clock period to elapse (e.g. 5 ns).
The clock rate is the inverse of the clock cycle time.
For example, if a computer has a clock cycle time of 5 ns, the clock rate is: 1
= 200 MHz
5 x 10 sec -9

10. The time to execute a given program can be computed as
Computing CPU time
The time to execute a given program can be computed as
CPU time = CPU clock cycles x clock cycle time
Since clock cycle time and clock rate are reciprocals
CPU time = CPU clock cycles / clock rate
The number of CPU clock cycles can be determined by
CPU clock cycles = (instructions/program) x (clock cycles/instruction)
= Instruction count x CPI
which gives
CPU time = Instruction count x CPI x clock cycle time
CPU time = Instruction count x CPI / clock rate
The units for this are
instructions cIock cycles seconds
seconds = x x
program instruction clock cycle

11. Example of Computing CPU time
If a computer has a clock rate of 50 HHz, how long does it take to execute a program with 1,000 instructions, if the CPI for the program is 3.5?
Using the equation
CPU time = Instruction count x CPI / clock rate
gives
CPU time = 1000 x 3.5 / (50 x 10 )
If a computer’s clock rate increases from 200 MHz to 250 MHz and the other factors remain the same, how many times faster will the computer be?
CPU time old clock rate new MHz
= = = 1.25
CPU time new clock rate old MHZ
What simplifying assumptions did we make? 6

12. Factors affecting CPU Performance
Which factors are affected by each of the following?
instr. count CPI clock rate
Program
Compiler
Instr. Set Arch.
Organization
Technology
CPU time = Seconds = Instructions x Cycles x Seconds
Program Program Instruction Cycle

13. The CPI is the average number of cycles per instruction.
Computing CPI
The CPI is the average number of cycles per instruction.
If for each instruction type, we know its frequency and number of cycles need to execute it, we can compute the overall CPI as follows:
CPI = Σ CPI x F
For example n i i
i = 1
Op F CPI CPI x F % Time
ALU 50% %
Load 20% %
Store 10% %
Branch 20% %
Total 100% % i i i i

14. The two main measure of performance are
Performance Summary
The two main measure of performance are
execution time : time to do the task
throughput : number of tasks completed per unit time
Performance and execution time are reciprocals. Increasing performance, decreases execution time.
The time to execute a given program can be computed as:
CPU time = Instruction count x CPI x clock cycle time
CPU time = Instruction count x CPI / clock rate
These factors are affected by compiler technology, the instruction set architecture, the machine organization, and the underlying technology.
When trying to improve performance, look at what occurs frequently => make the common case fast.

15. Computer Benchmarks
A benchmark is a program or set of programs used to evaluate computer performance.
Benchmarks allow us to make performance comparisons based on execution times
Benchmarks should
Be representative of the type of applications run on the computer
Not be overly dependent on one or two features of a computer
Benchmarks can vary greatly in terms of their complexity and their usefulness.

16. Types of Benchmarks Cons Pros very specific non-portable
difficult to run, or
measure
hard to identify cause
representative
Actual Target Workload
portable
widely used
improvements useful in reality
less representative
Full Application Benchmarks
(e.g., SPEC benchmarks)
does not measure memory system
easy to run, early in design cycle
Small “Kernel”
Benchmarks
“peak” may be a long way from application performance
identify peak capability and potential bottlenecks
Microbenchmarks

17. SPEC: System Performance Evaluation Cooperative
The SPEC Bencmarks are the most widely used benchamrks for reporting workstation and PC performance.
First Round SPEC CPU89
10 programs yielding a single number
Second Round SPEC CPU92
SPEC CINT92 (6 integer programs) and SPEC CFP92 (14 floating point programs)
Compiler flags can be set differently for different programs
Third Round SPEC CPU95
New set of programs: SPEC CINT95 (8 integer programs) and SPEC CFP95 (10 floating point)
Single compiler flag setting for all programs
Fourth Round SPEC CPU2000
New set of programs: SPEC CINT2000 (12 integer programs) and SPEC CFP2000 (14 floating point)
Value reported is the SPEC ratio
CPU time of reference machine / CPU time of measured machine

18. Other SPEC Benchmarks JVM98: SFS97: Web99: HPC96: APC, MEDIA, OPC
Measures performance of Java Virtual Machines
SFS97:
Measures performance of network file server (NFS) protocols
Web99:
Measures performance of World Wide Web applications
HPC96:
Measures performance of large, industrial applications
APC, MEDIA, OPC
Meausre performance of graphics applications
For more information about the SPEC benchmarks see:

19. Examples of SPEC95 Benchmarks
SPEC ratios are shown for the Pentium and the Pentium Pro (Pentium+) processors
What can we learn from this information?

20. Poor Performance Metrics
Marketing metrics for computer performance included MIPS and MFLOPS
MIPS : millions of instructions per second
MIPS = instruction count / (execution time x 10^6)
For example, a program that executes 3 million instructions in 2 seconds has a MIPS rating of 1.5
Advantage : Easy to understand and measure
Disadvantages : May not reflect actual performance, since simple instructions do better.
MFLOPS : millions of floating point operations per second
MFLOPS = floating point operations / (execution time x 10^6)
For example, s program that executes 4 million instructions in 5 seconds has a MFLOPS rating of 0.8
Disadvantages : Same as MIPS,only measures floating point

21. Speedup due to an enhancement is defined as:
Amdahl's Law
Speedup due to an enhancement is defined as:
ExTime old Performance new
Speedup = =
ExTime new Performance old
Suppose that an enhancement accelerates a fraction
Fractionenhanced of the task by a factor Speedupenhanced,
ExTimenew = ExTimeold x (1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced 1
ExTimeold
ExTimenew
Speedup = =
(1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced

22. Example of Amdahl’s Law
Floating point instructions are improved to run twice as fast, but only 10% of the time was spent on these instructions originally. How much faster is the new machine? 1
ExTimeold
ExTimenew
Speedup = =
(1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced 1
Speedup =
= 1.053
( ) /2
The new machine is times as fast, or 5.3% faster.
How much faster would the new machine be if floating point instructions become 100 times faster? 1
Speedup =
= 1.109
( ) /100

23. Estimating Perfomance Improvements
Assume a processor currently requires 10 seconds to execute a program and processor performance improves by 50 percent per year.
By what factor does processor performance improve in 5 years?
( )^5 = 7.59
How long will it take a processor to execute the program after 5 years?
ExTimenew = 10/7.59 = 1.32 seconds
What assumptions are made in the above problem?

24. M1 has a clock rate of 50 MHz and M2 has a clock rate of 75 MHz.
Performance Example
Computers M1 and M2 are two implementations of the same instruction set.
M1 has a clock rate of 50 MHz and M2 has a clock rate of 75 MHz.
M1 has a CPI of 2.8 and M2 has a CPI of 3.2 for a given program.
How many times faster is M2 than M1 for this program?
What would the clock rate of M1 have to be for them to have the same execution time?
ExTimeM ICM1 x CPIM1 / Clock RateM1
2.8/50 = =
= 1.31
ExTimeM ICM2 x CPIM2 / Clock RateM2
3.2/75

25. Summary of Performance Evaluation
Good benchmarks, such as the SPEC benchmarks, can provide an accurate method for evaluating and comparing computer performance.
MIPS and MFLOPS are easy to use, but inaccurate indicators of performance.
Amdahl’s law provides an efficient method for determining speedup due to an enhancement.
Make the common case fast!