November 13, 2006 Performance Engineering Research Institute 1 Scientific Discovery through Advanced Computation Performance Engineering Research Institute David H Bailey Lawrence Berkeley National Laboratory

November 13, 2006 Performance Engineering Research Institute 2 Argonne National Laboratory Lawrence Berkeley National Laboratory Lawrence Livermore National Laboratory Oak Ridge National Laboratory Rice University University of California at San Diego University of Maryland University of North Carolina University of Southern California University of Tennessee Participating Institutions

November 13, 2006 Performance Engineering Research Institute 3 SciDAC Scientific Discovery through Advanced Computation DOE Office of Science’s path to Petascale computational science Maximizing performance is getting harder: Systems are more complicated O(100K) multi-core CPUs SIMD extensions Codes are more complicated Multi-disciplinary Multi-scale IBM BlueGene at LLNL Cray Xt3 at ORNL BeamBeam3D accelerator modeling POP model of El Nino

November 13, 2006 Performance Engineering Research Institute 4 Example of the Challenge Figure courtesy Roger Logan, LLNL

November 13, 2006 Performance Engineering Research Institute 5 SciDAC-1 PERC Performance Evaluation Research Center (PERC) Initial goal was to develop performance related tools Benchmarks Analysis Modeling Optimization Second phase refocused on SciDAC applications incl. Community Climate System Model Plasma Microturbulence Project Omega3P accelerator model

November 13, 2006 Performance Engineering Research Institute 6 Some Lessons Learned Performance portability is critical Codes outlive machines Scientists can’t publish that they migrated code Computational scientists were not interested in tools They wanted experts to work with them Such experts are not scalable

November 13, 2006 Performance Engineering Research Institute 7 SciDAC-2 PERI Performance Engineering Research Institute Performance modeling of applications How fast do we expect to go? Automatic tuning Long term research goal Remove burden from scientific programmers Application engagement Near-term impact on SciDAC applications

November 13, 2006 Performance Engineering Research Institute 8 Performance Modeling Modeling is critical for automation of tuning Need to know where to focus effort Where are the bottlenecks? Need to know when we’re done How fast can we hope to go? Obvious improvements: Greater accuracy Reduced cost Modeling efforts contribute to procurements and other activities beyond PERI automatic tuning

November 13, 2006 Performance Engineering Research Institute 9 Performance Tuning Humans have been doing this for 50 years Compilers have been doing it statically for 40 years Recent self-tuning libraries: PHIPAC, ATLAS, FFTW, SPIRAL, SPOOLES Performance Engineering Research Institute goal: Automatic performance tuning of applications

November 13, 2006 Performance Engineering Research Institute 10 Automatic Performance Tuning of Scientific Code Long-term goals for PERI  Automate the process of tuning software to maximize its performance  Reduce the performance portability challenge facing computational scientists.  Address the problem that performance experts are in short supply  Build upon forty years of human experience and recent success with linear algebra libraries PERI automatic tuning framework

November 13, 2006 Performance Engineering Research Institute 11 Automatic Tuning Steps Triage: where to focus effort Semantic analysis: traditional compiler analysis Transformation: code restructuring Code generation: domain specific code Off-line search: empirical experiments Assembly: choose the best components Training runs: performance data for feedback On-line search: optimize long-running jobs

November 13, 2006 Performance Engineering Research Institute 12 Early Results Empirical optimization of dense matrix-matrix multiplication (Hall, USC) Empirical optimization of Madness kernel (Moore, UTK)

November 13, 2006 Performance Engineering Research Institute 13 PERI Portal Automatic tuning is common goal of half-a-dozen research projects No hope of actually integrating them into one system e.g., Open64, SUIF, and ROSE compilers Instead PERI will bring up a Web portal which will be our interface to application developers There will often be “a man behind the curtain” Goal is research demonstration of capability

November 13, 2006 Performance Engineering Research Institute 14 Application Engagement  Application Engagement  Work directly with DOE computational scientists  Ensure successful performance porting of scientific software  Focus PERI research on real problems  Application Liaisons  Build long-term personal relationships with PERI researchers and scientific code teams  Tiger Teams  Focus on DOE’s highest priorities  SciDAC-2  INCITE Maximizing scientific throughput Optimizing arithmetic kernels

November 13, 2006 Performance Engineering Research Institute 15 Summary SciDAC-2 Performance Engineering Research Institute Performance modeling of scientific applications so we understand what performance is possible Automatic performance tuning to alleviate computational scientists from this recurring problem Near-term impact via direct engagement with SciDAC application teams