1. SPEC HPG Benchmarks for HPC Systems Kumaran Kalyanasundaram for SPEC High-Performance Group Kumaran Kalyanasundaram, PhD Chair, SPEC HPG Manager, SGI Performace Engineering

2. SPEC HPGs Purpose zThe High Performance Group focuses on the development of application benchmarks for high performance computers.

3. SPEC HPG zFounded in 1994 (Perfect Benchmarks initiative became HPG). zMembers from industry and academia. zTwo active benchmarks - SPEC OMP & SPEC HPC2002. zNew MPI2006 benchmark currently under development.

4. OMPL2001 Founding of SPEC HPG HPC96 OMP2001 HPC2002 MPI2006 Jan 1994Oct 1995June 2001June 2002Jan 2003 2006 SPEC HPG Benchmark Suites

5. SPEC OMP zBenchmark suite developed by SPEC HPG (High Performance Group) zBenchmark suite for performance testing of shared memory processor systems zUses OpenMP versions of SPEC CPU2000 benchmarks and candidates

6. Why Did SPEC Choose OpenMP? zBenchmark suite is focused on SMP systems zOpenMP is a standard, and is applicable to Fortran, C, and C++. zDirective based OpenMP allows serial version to remain largely intact. zQuickest path to parallel code conversion.

7. OMP/CPU2000 Similarities zSame tools used to run the benchmarks zSimilar run and reporting rules zUses geometric mean to calculate overall performance relative to a baseline system zSimilar output format

8. SPEC OMP Benchmark Principles zSource code based zLimited code and directive modifications zFocused on SMP performance zRequires a base run xwith no source modifications xsingle set of compiler flags for all benchmarks zSPEC supplied tools required to run benchmark

9. OMPM2001 Benchmarks

10. OMP vs CPU2000

11. Program Memory Footprints

12. SPEC HPC2002 Benchmark zFull Application benchmarks (including I/O) targeted at HPC platforms zSerial and parallel (OpenMP and/or MPI) zCurrently three applications : ySPECenv: weather forecast ySPECseis: seismic processing, used in the search for oil and gas ySPECchem: comp. chemistry, used in chemical and pharmaceutical industries (gamess) zAll codes include several data sizes

13. zAn application benchmark suite that measures CPU, memory bw, interconnect, compiler, MPI performance. zSearch program is open till March 31st, 06 zCandidate codes in the areas of Comp. Chemistry, weather forecasting, HE Physics, Oceanography, CFD, etc. SPEC MPI2006

14. zVery large data sets for MPI2006. zFollow onto SPEC OMPM(L)2001. zUpdate SPEC HPC2002 suite. Future Goals