1. Application Performance Prediction Javier Delgado Feb. 9, 2009 X

2. Motivation (General) Optimal usage of grid resources through “smarter” meta-scheduling Many users overestimate job requirements Reduced idle time for compute resources Save utility and energy costs Optimal resource selection for most expedient job return time

3. WRF Prediction Possibilities Over 50% of Classified Disasters Hurricanes Flash Floods Droughts Source: adrc.asia

4. Without Performance Prediction Users need additional knowledge How long will the job take? Where to send? etc. Unfair preemption of resources

5. Example Scenario 3 Resources Marenostrum (10K+ core supercomputer) Mind (32 core hyperthreading cluster) GCB (8 core hyperthreading cluster) 2 jobs 1 continental US WRF simulation (urgent) 1 simulation of a 75 x 75 portion of Florida (for benchmarking)

6. Example Scenario User has no knowledge of how long either simulation will last Intuitively, Marenostrum will be faster However, the user has have exclusive access to Mind (i.e. no queue time) How should the jobs be allocated?

7. Example Scenario CONUS Job Benchmark Job Marenostrum (32 nodes)Mind (all nodes)GCB (all nodes) 45 minutes180 minutes500 minutes Marenostrum (32 nodes)Mind (all nodes)GCB (all nodes) 3 minutes20 minutes50 minutes

8. Example Scenario Execution Prediction (aprof) can estimate execution time on each system Other tools can be used for queue time prediction With the above two, and using information from metascheduler, automatic allocation is feasible

9. Motivation (Storm Mitigation) Humane Thousands of lives can be saved Economical Millions of dollars needed to fix damages If given more time, we can minimize this

10. 10-km WRF 4-km WRF Dashed magenta indicates approximate area of rainfall Produced by convective parameterization Parameterized convection (on the 10 km grid) cannot differentiate different mode of convection Why So Many Processors? Source: NCAR ( www.ncep.noaa.gov/nwp50/Presentations/Thu_06_17_04/Session_9/Kuo_50th_NWP/Kuo_50th_NWP.ppt)

11. Process

12. Completed Work Prediction Experiments 3 Different Platforms 1 domain

13. To Do Testing with different domains Testing on new platforms Cross-cluster testing Model Refinement, as necessary (GPU Programming)

14. Typical Tasks Code Inspection C++ programming (for the model) Python and BASH scripting for testing Analysis of model and/or results using statistics techniques

15. Thank You! Any Questions?