1. On the Integration and Use of OpenMP Performance Tools in the SPEC OMP2001 Benchmarks Bernd Mohr 1, Allen D. Malony 2, Rudi Eigenmann 3 1 Forschungszentrum Jülich, John von Neumann - Institut für Computing, Zentralinstitut für Angewandte Mathematik, b.mohr@fz-juelich.de 2 Department of Computer and Information Science, University of Oregon, malony@cs.uoregon.edu 3 Department of Electrical and Computer Engineering, Purdue University, eigenman@ecn.purdue.edu

2. WOMPAT 2002 2 August 5, 2002 Outline  SPEC OMP2001 benchmark suite  Detailed performance characterization study  Integrated performance tools in benchmarking suites  Motivation  Approach for OMP2001  POMP OpenMP performance monitoring interface  Opari automatic instrumentation  Profiling and trace measurement  EXPERT and TAU performance analysis  Experiments  Concluding remarks

3. WOMPAT 2002 3 August 5, 2002 SPEC OMP2001 Benchmark Suite  11 application programs used in scientific computing  CFD: APPLU, APSI, GALGEL, MGRID, SWIM  Molecular dynamics: AMMP  Crash simulation: FMA3D  Neural network: ART  Genetic algorithm: GAFORT  Earthquake modeling: EQUAKE  Quantum chromodynamics: WUPWISE  Fortran and C source code with OpenMP parallelization  Medium and large data sets  Goals of portability and relative ease of use

4. WOMPAT 2002 4 August 5, 2002 OMP2001 Performance Measurement Studies  OMP2001 measures and reports total execution time only  Scalability results for different processor numbers  “Performance Characteristics of the SPEC OMP2001 Benchmarks,” Aslot and Eigenman, EWOMP 2001  Study performance characteristics in detail  Timing profiles (scalability) across parallel sections  Memory system and cache (hardware counter) profiles  Use of high-resolution timers and hardware counters  Quantitative and qualitative explanations  Custom instrumentation and measurement libraries  Required hand-instrumentation of OpenMP constructs

5. WOMPAT 2002 5 August 5, 2002 Performance Tools and Benchmark Suites  Detailed performance measurement and analysis reveal interesting runtime characteristics in application codes  Important for performance diagnosis and tuning  Help to understand effects of new parallel API (OpenMP)  Benchmark suites typically do not have integrated tools  Portability of performance tools is poor  Hard to configure tools for benchmarking methodology  Tools often require manual application and operation  Automatic and portable performance tools could allow more in-depth, cross-platform performance analysis  Goal: integrated performance tools for OMP2001

6. WOMPAT 2002 6 August 5, 2002 Approach for OMP2001  Leverage state-of-the-art performance instrumentation, measurement, and analysis technology  POMP OpenMP performance monitoring interface  Opari automatic OpenMP source instrumentation  Performance profile and trace measurement libraries  EXPERT automatic event trace analyzer  TAU performance analysis system  Configure performance tools as integrated and automated components in OMP2001 benchmarking methodology  Conduct performance experiments on OMP2001codes  Evaluate with respect to portability, ease of use, results

7. WOMPAT 2002 7 August 5, 2002 Issues  Level of measurement detail  What is necessary and appropriate?  Could use base level and allow user-configured levels  Full program execution vs. portion of program execution  Distribution complexity  Tool packages should be added to benchmark distribution  Packages need to be easily obtained and configured  Must be public domain or licensed through SPEC  Publishing of detailed performance results  Part of official SPEC benchmark report?  …

8. WOMPAT 2002 8 August 5, 2002 TAU Performance System Framework  Tuning and Analysis Utilities  Performance system framework for scalable parallel and distributed high-performance computing  Targets a general complex system computation model  nodes / contexts / threads  Multi-level: system / software / parallelism  Measurement and analysis abstraction  Integrated toolkit for performance instrumentation, measurement, analysis, and visualization  Portable performance profiling/tracing facility  Open software approach

9. WOMPAT 2002 9 August 5, 2002 TAU Performance System Architecture EPILOG Paraver