1. 1 Recap

2. 2 Measuring Performance  A computer user: response time (execution time).  A computer center manager - throughput - the total amount of work done in a period of time.  CPU time : a very good and fair measure of performance.  CPU time can also be divided into user CPU time (program) and system CPU time (OS).

3. 3 Aspects of CPU Execution Time CPU Time = Instruction count x CPI x Clock cycle Instruction Count I ClockCycle C CPI Depends on: CPU Organization Technology Depends on: Program Used Compiler ISA CPU Organization Depends on: Program Used Compiler ISA

4. 4 Factors Affecting CPU Performance CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle CPI C Clock Cycle C Instruction I Count I Program Compiler Organization Technology Instruction Set Architecture (ISA) X X X X X X X X X

5. 5 Example: tradeoff between C and CPI Assume stores can execute in 1 cycle by slowing clock 15% Should this be implemented? OpFrequencyCycle Count ALU ops43%1 Loads21%1 Stores12%2 Branches24%2

6. 6 Simple Example Old CPI = 0.43 x 1 + 0.21 x 1 + 0.12 x 2 + 0.24 x 2 = 1.36 New CPI = 0.43 x 1 + 0.21 x 1 + 0.12 x 1 + 0.24 x 2 = 1.24 Speedup = old time/new time –= {I x old CPI x C}/{I x new CPI x 1.15 C} –= 1.36 / (1.24 x 1.15) = 0.95 Answer: Don’t make the change

7. 7 Some Caveats Inter-dependence of I, CPI, and C: Improvement In One May Impact Another –increasing pipeline depth tends to increase clock speed but may increase CPI –Change in ISA to reduce instruction count may require a design with slower clock => May Not Improve Performance –CPI depends on instruction mix => Smaller Instruction Count May Not Improve Performance

8. 8 Code Size & Performance

9. 9 Benchmarks and Benchmarking In lack of a universal task pick some programs that represent common tasks Use representative programs to compare performance of systems: CAUTIONS: –Comparisons are as good as the benchmarks are in representing your real workload. –Many parameters affect measured performance

10. 10 Example: We must use the same compiler Compiler “enhancements” and performance  1998 Morgan Kaufmann Publishers

11. 11 Benchmark Suites A Suite Is a Collection of Representative Benchmarks From Different Application Domains Weakness of Any One Benchmark Likely to Be Compensated By Another Standard Performance Evaluation Corporation (SPEC) –Most Popular Benchmark Suite –Suite Consists of Kernels, Small Fragments, Large Applications –SPEC2006: CINT2006, CFP2006 –http://www.spec.org/ Benchmark suites for servers –SPECSFS: measures performance of File servers –SPECWeb: measurers performance of Web servers

12. 12 SPEC CPU2006 Programs Benchmark Language Descriptions 400.PerlbenchC Programming Language 401.bzip2 C Compression 403.GccCC Compiler 429.mcf C Combinatorial Optimization 445.gobmk C Artificial Intelligence: Go 456.HmmerC Search Gene Sequence 458.sjeng C Artificial Intelligence: chess 462.libquantum CPhysics / Quantum Computing 464.h264refCVideo Compression 471.omnetpp C++Discrete Event Simulation 473.astar C++Path-finding Algorithms 483.xalancbmk C++XML Processing CINT2006 (Integer) Source: http://www.spec.org/osg/cpu2006/CINT2006/

13. 13 SPEC CPU2006 Programs Benchmark Language Descriptions 410.BwavesFortran Fluid Dynamics 416.GamessFortran Quantum Chemistry 433.MilcC Physics / Quantum Chromodynamics 434.ZeusmpFortran Physics / CFD 435.GromacsC, FortranBiochemistry / Molecular Dynamics 436.cactusADMC, FortranPhysics / General 437.leslie3dFortranFluid Dynamics 444.NamdC++Biology / Molecular Dynamics 447.dealIIC++ Finite Element Analysis 450.SoplexC++ Linear Programming, Optimization 453.PovrayC++ Image Ray-tracing 454.CalculixC, FortranStructural Mechanics 459.GemsFDTDFortran Computational Electromagnetics 465.TontoFortranQuantum Chemistry 470.LbmCFluid Dynamics 481.WrfC, FortranWeather 482.sphinx3CSpeech CFP2006 (Floating Point) Source: http://www.spec.org/osg/cpu2006/CFP2006/

14. 14 Top 20 SPEC CPU2006 Results (As of August 2007) # MHz Processor int peak int baseMHz Processor fp peak fp base 13000Core 2 Duo E685022.620.24700 POWER622.417.8 24700POWER6 21.617.83000 Core 2 Duo E685019.318.7 33000Xeon 5160 21.017.91600 Dual-Core Itanium 2 18.117.3 43000Xeon X5365 20.818.91600 Dual-Core Itanium 2 17.817.0 52666Core 2 Duo E6750 20.518.32666 Core 2 Duo E675017.717.1 62667Core 2 Duo E6700 20.017.93000 Xeon 516017.717.1 72667Core 2 Quad Q670019.717.63000 Opteron 222217.416.0 82666Xeon X535519.117.32667 Core 2 Duo E670016.916.3 92666Xeon 515019.117.32800 Opteron 222016.713.3 102666Xeon X535518.917.23000 Xeon 516016.616.1 112667Xeon X535518.616.82667 Xeon X535516.616.1 122933Core 2 18.517.82667 Core 2 Quad Q670016.616.1 132400Core 2 Quad Q660018.516.52666 Xeon X535516.616.1 142600Core 2 Duo X780018.316.42933 Core 2 Extreme X680016.216.0 152667Xeon 515017.616.62400 Core 2 Quad Q660016.015.4 162400Core 2 Duo T770017.616.61400 Dual-Core Itanium 2 15.915.2 172333Xeon E534517.515.92667 Xeon 515015.915.5 182333Xeon 514817.415.92333 Xeon E534515.414.9 192333Xeon 514017.415.72600 Opteron 221815.412.5 202660Xeon X535517.415.72400 Xeon X322015.315.1 Source: http://www.spec.org/cpu2006/results/cint2006.html Top 20 SPECfp2006 Top 20 SPECint2006

15. 15 Performance Evaluation Using Benchmarks “For better or worse, benchmarks shape a field” Good products created when we have: –Good benchmarks –Good ways to summarize performance Given sales depend in big part on performance relative to competition, there is big investment in improving products as reported by performance summary If benchmarks inadequate, then choose between improving product for real programs vs. improving product to get more sales; Sales almost always wins!

16. 16 How to Summarize Performance