High Performance Computing for Real Time M & S Randall Shumaker Director, Institute for Simulation and Training

There is an important issue here that cant be ignored: the rate of change is actually accelerating This is very important for military planning, it cannot be ignored away

Growth in important IT technologies Optical Fiber (bits/second) Silicon Computer Chips (number of transistors) Data Storage (bits/square inch) Number of years Performance per Dollar Doubling Time (months) Source: Scientific American Jan 2001 The Triumph of the Light p81-86

High End Computing Power

The Top 10 Machines November 2007 Rmax is in TeraFLOPS = One Trillion (10 12 ) Floating Point Operations per second

Projected Top 500 computing power

Storage has more than kept pace Circuit City external hard drive advertisement Jan Gbyte USB 2.0 drive $ Gigabytes/Dollar!

Why cant it go on forever? Source: Economist 10 May 2003 The heat dissipation issue: power density of Intel processors (watts/ sq Cm)

IBM demonstration - 80 cores on a die Shifting strategies - multiple processing cores on each die

What does all this mean for less high end computing?

Comparable Trends in Nature Time (days) Growth of a Weed in the Negev Desert Height ( inches ) Harvester Ant Colony Size By these standards computers are probably late midlife in growth

When might this reasonably be expected to taper off? (Prediction is hard, particularly about the future) multicore

Where is the PC world now? Apple has announced the Mac Pro dual quad Typical Configuration: Two 3.0 GHz Quad core Intel Xeon (8 - cores) 16 G-bytes RAM 2 T-bytes Disc Dual link DVI video 256 M bytes graphics memory inch flat panel graphics displays Cost: $8,422 plus tax, free shipping The good news: This is a highly capable system, theoretical peak performance is > 600 GigaFLOPS. Five years ago this would have been a supercomputer. In 5 years this will cost under $2000. The bad news: Pretty much none of your current software can take effective advantage of this capability.

What are some implications of these numbers?

Some perspective: Computing Power and Capabilities The Hans Moravec vision

Computing Power and Capabilities The Hans Moravec vision Intel Pentium 4 EE (2005) GHz AMD Athlon FX-60 dual core (2006) GHz Blue gene L (2007) 478 Tflops

Well Hans!, computing power isnt really the problem Visual metaphor for the state of the art in software development

Where might we like to apply more computing power other than building a monkey brain?

Interactive Simulation Needs –Real time capability using fast processors and high-speed interconnects –High fidelity –Low latency/High bandwidth interconnects –Real time I/O –Connection to real world assets –Fixed frame rates (some apps) Strategies –Message Passing Interface (MPI) or Scalable Link Interface (SLI) –Ltd shared memory processing (SMP) or distributed processing Interfaces with sensory processors (e.g., interactive visualization, haptics, …) Scalability in terms of HPC architecture and simulation entities

Areas for Investigation Extents of single image environments –Terrain/Environment –Interacting entities Live, virtual, constructive experimentation –Scalable simulations –Multi-scale simulations –Control of propagating granularity HPC architectures for interaction –Map HPC types to applications Techniques for porting interactive applications to HPC platforms Tools for interaction

Areas for Investigation (cont) Lets remember the human factor –How will a user interact with an HPC? –How will multiple users interact with an HPC & maintain coherence of I/O? –How will interim results be gathered? –How can timely and relevant HF experiments be developed to influence the design? Get developers involved…

UCF/IST high performance computing initiative

The Purchase Competitive Procurement (7 Bids) IBM X-Series –24 Node (192 Cores) ~1.8 TeraFLOPS –16 GB Memory/Node(768 Gigabytes RAM) –20 TB Spinning Storage (20,000 GigaBytes) –Red Hat Linux1 –3 Years Support Expect Delivery in mid Feb 2008

How this fits within the big HPC picture 1st increment 2nd increment

HPC Infrastructure Support Sys Admin and Parallelization Expertise Training from IBM –15 Days On-Site –Tuned to Our Skill Set Support from IBM –3 Year Warranty/Software Licenses Services –Storage/Network Config –Resource Mgmt Setup –…

Some slides we stole from Roger Smith, CTO PEOSTRI, to validate our views

HPC Applications Batch Jobs Computational Fluid Dynamics Computational Chemistry Protein Folding Cryptanalysis Interactive JFCOM Urban Challenge 2004 Joint SAF

Future Simulation Hardware HPC in its various forms may be an important part of the future of simulation … we intend to find out We need the support of our Team Orlando partners HPC offers the power to Create larger scenarios with higher fidelity Drive innovation in simulation software architecture Globally distribute training from a Simulation IT Center Coordinate different technologies for different problems: Distributed Processing, Clusters, Shared Memory, Multicore, GPGPU, Cell

HPC in the Sim Center Shared Memory: Big Scenarios Tight connection between Sims distributed across multiple CPU and memory. E.g. Very large single instance of WARSIM on 128 processors Cluster: Multi-chunked World Many Sim instances on individual CPUs with local memory. E.g. Multiple geographically divided OneSAF instances. GPU: Vector Operations Off-loaded vector operations like rendering and LOS. E.g. Graphic heads for HPC or LOS for sims. Shared Cluster GPU Simulation Server Multi-core: Multi-threaded Software Every CPU in the machine can support multiple threads. E.g. Movement, AI, Sensors parsed off to a separate core.

Training to Every Unit C4I IT E-BOX Sim GUI, Web I/F, Google Earth [Analogy: GIAC, Tapestry, Phosphor] Sim Center 21 Shared Cluster GPU Closed, Portable Sim Center Sim IT CTRL NET Regional Ctr Cluster GPU Network, Service Oriented Arch

HPC Competition Suffolk - JFCOM UR04, Wright-Patt, Maui, Joint SAF Federation 1,000,000 Simple Entities Active Getting their own 256 node HPC (generic cluster) Huntsville – MDA, SMDC Advanced Research Center Hypervelocity Missile Center Redstone Technical Test Center Orlando – PEO-STRI, RDECOM OneSAF Chemical Plumes (SAIC) OneSAF LOS on GPU (RDECOM STTC, UNC Chapel Hill, SAIC) WARSIM on Multiple Cores (PM CONSIM, Lockheed, Northrop)

Team Orlando PEO-STRI, STTC, & UCF IST HPC $1M Congressional Earmark PM CONSIM Support Supercomputing 2007 Conference Panel … Bring Your Corporate Expertise Here

Competitive Summary Is HPC or specialized computing an important part of the future of Interactive Training Simulation? If so … Who are the competitors? How is Team Orlando positioned to compete? What can your organization contribute? Where will HPC-enabled simulation be based? Suffolk, Huntsville, Orlando, DC-area, Leavenworth

Questions?