1. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Slide 1 Exascale? No problem! Paul Henning Los Alamos National Laboratory phenning@lanl.gov LAUR 10-05371

2. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Slide 2 Exascale? No problem! Paul Henning Los Alamos National Laboratory phenning@lanl.gov (But most apps are screwed)

3. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D According to Merriam-Webster… “screw” 4.a (1): to mistreat or exploit through extortion, trickery, or unfair actions; especially: to deprive of or cheat out of something due or expected Slide 3 The majority of developers, not yet working on petascale projects, are going to be blindsided by pervasive changes coming to computing environments. Can we (read, “you”) help them?

4. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Exascale applications have been hammered on in other settings… Combustion Nuclear energy Biology and biofuels Fusion Materials Climate modeling High-energy and nuclear physics National security Slide 4

5. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D The “Path to Zero” provides a suite of exascale drivers Cradle-to-grave assessments to support complex-wide process and resource optimization System-scale simulation support for national policy decisions on stockpile changes or reductions System-scale physics of understanding of nuclear weapons, including complex interacting microscale processes Slide 5 Source: Scientific Grand Challenges for National Security: The Role of Computing at the Extreme Scales Workshop Report

6. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Large scale simulation capability is cheap compared to alternatives “Icecap” UGT, suspended on U.S. entrance to Underground Nuclear Testing Moratorium, 10/3/92 157’ tower over 94”×1675’ hole 350k-500k lbs of gear, miles of cable This would have been the 929 th test at NTS Slide 6 Information from DOE/NV-1212, May 2007 (NTS “Icecap” Fact Sheet)

7. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Exascale hardware will be achieved by depriving* software of expected execution environments Lots of threads and vector units Total memory doesn’t scale with threads/flops Jaguar ~0.3PB Exascale ~64PB 213x != 1000x Less cache/thread and/or local store Smaller (than node) coherency domains Will require many types of parallelism, simultaneously Radical changes in interconnect possible Slide 7 * screwing

8. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D “Blue collar” supercomputing may be more impacted than high-end High-end is already planning for it Exascale community will not, by definition, fail. All hardware is going to be affected, thanks to power/mobility drivers Exascale needs processor specialization to meet power requirements Desktops are still selling well, but aren’t coming near the growth of mobile computing Mobile processors look to specialization to reduce power and size When does the “traditional” CPU, and its programming model, die? Are the scientific and commercial applications expecting traditional performance increases without modifying their codes? Slide 8

9. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Q: Numerical libraries? “Library” has two meanings: A collection of related functionality (presumably) assembled by domain experts A collection of reusable object code to link/load into applications Object libraries have two benefits Easy access to highly tuned platform-specific variants Convenience for apps developers (faster compiles, pre-installed) Note: libraries are products, apps are not! Dynamically-loaded libraries are mostly bunk for scientific apps No space savings for single executables, unnecessary relocation costs Band-aid for bad language support: dlopen, cross-language linking Object libraries are opaque to global parallelization, optimization and resource allocation/scheduling. Slide 9

10. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D A: Not –lmega_parallel_foo, thanks. (-lm is fine) At O(1B) threads, concurrency is far more important than optimizing serial instruction streams. We can’t assume that a single kernel/algorithm can fully utilize all of the hardware on a node On node resource scheduling will migrate into the application We need to consider MPMD Libraries don’t need to be object files Example: C++ template libraries “Compiler” gets to see everything for resource management Can still do file-at-a-time conversion to IRs, just don’t go to object code Slide 10

11. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D Just some closing notes… The TOP500 stagnates without investment and effort The first sustained exaflop/sec calculation will run in 2018 Self-fulfilling prophecy Where? Somewhere east of here (on a sphere) What? HPL, of course! And then a particle/MC method Autotuning could look at higher-level resource allocation issues How much of each processor type should be allocated to kernels that can run simultaneously? What are “optimal” memory layouts for multiple processor and communication types on a node? — Vector vs. scalar — DMAs and local store vs. PGAS vs. MPI Slide 11

12. Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA U N C L A S S I F I E D DOE Scientific Grand Challenges Workshop Series Slide 12 http://www.er.doe.gov/ascr/Misc/GrandChallenges.html