Presentation is loading. Please wait.

Presentation is loading. Please wait.

HPCMP Benchmarking and Performance Analysis Mark Cowan USACE ERDC ITL in support of DoD HPCMP Tuesday, April 17, 2012.

Similar presentations

Presentation on theme: "HPCMP Benchmarking and Performance Analysis Mark Cowan USACE ERDC ITL in support of DoD HPCMP Tuesday, April 17, 2012."— Presentation transcript:

1 HPCMP Benchmarking and Performance Analysis Mark Cowan USACE ERDC ITL in support of DoD HPCMP Tuesday, April 17, 2012

2 What is the HPCMP? Initiated in 1992 Congressional mandate to modernize DoD’s HPC capabilities Assembled from collection of HPC departments across Army, Air Force, and Navy labs and test centers

3 What is the HPCMP? FOCUS Solve military and security problems using HPC hardware and software Assess technical and management risks Performance Time Available resources Cost Schedule Supports DoD objectives through research, development, test and evaluation

4 Where we benchmark

5 Migrate to a 2-year acquisition cycle Why the radical change? Entice more vendors into the competition Vendor feedback  remove or alleviate disincentives Review the entirety of the TI acquisition process Line-by-line justification of benchmarking rules document Address both HPC community and vendor concerns Comprehensive reevaluation of how we benchmark Analyze the codes Justify the test cases

6 Migrate to a 2-year acquisition cycle Dangers? Time the milestones poorly on the calendar and miss out on release of cutting-edge technology Difficult problem How to schedule activities to maximize likelihood of hitting publicly-available products months in advance, while being blind to intricacies of chip fabrication schedules and unforeseen recalls?


8 TI-11/12 benchmarking applications ADCIRC – Coastal Circulation and Storm Surge model –100% Fortran, MPI –Uses METIS library (C) –205K LOC ALEGRA – Hydrodynamic and solid dynamics plus magnetic field and thermal transport –96% C, 4% Fortran, MPI –978K LOC AVUS (Cobalt-60) – Turbulent flow CFD code –Fortran, MPI, 29K LOC CTH – Shock physics code –~58% Fortran/~42% C, MPI, 900K LOC GAMESS – Quantum chemistry code –Fortran, MPI, 330K LOC HYCOM – Ocean circulation modeling code –Fortran, MPI, 31K LOC ICEPIC – Particle-in-cell magnetohydrodynamics code –C, MPI, 350K LOC LAMMPS – Molecular dynamics code –C++, MPI, 45K LOC █ Predicted █ Benchmarked

9 Components of testing packages Applications tested on representative input sets CODECASE Distinguished Core Count Time (sec) on DIAMOND Core Counts ADCIRCbaroclinic1024 8959 512, 768, 1024, 1280, 1536, 1792, 2048 ADCIRChurricane1280 2082 512, 768, 1024, 1280, 1536, 1792, 2048 ALEGRAobliqueImp1536 1640 1024, 1280, 1536, 1792, 2048 ALEGRAexplWire256 944 256, 384, 512, 768, 1024 AVUSwaverider1024 941 384, 512, 768, 1024, 1536 AVUSturret-td1280 1332 768, 1024, 1280, 1536, 2048 CTHfixed-grid1280 3399 768, 1024, 1280, 1536, 2048 CTHamr1280 2535 768, 1024, 1280, 1536, 2048 GAMESSDFT-grad256 4701 128, 192, 256, 384, 512 GAMESSMP2-grad512 2536 128, 256, 512, 768, 1024 GAMESSCC-energy1024 3658 512, 768, 1024, 1536, 2048 HYCOMlrg1353 3020 1001, 1353, 1516, 1770, 2045 ICEPICmagnetron384 2559 256, 384, 512, 768, 1024 ICEPICgyrotron2048 3639 1536, 1792, 2048, 2304, 2560 LAMMPSAu1024 3182 128, 256, 384, 512, 1024, 1280, 1536, 2048

10 Some components of HPC procurement cycle

11 Acquire new versions of codes Port codes to various machines Acquire test cases Develop or acquire accuracy checks Test codes, get times to compare Assemble package for vendors

12 Some components of HPC procurement cycle Run codes with test cases on installed DSRC machines Optimize! How fast can we go?

13 Some components of HPC procurement cycle We review vendor submittal Anything suspicious? How do vendor times compare to ours? How did vendors optimize? How risky is vendor’s proposal? Present our results

14 Components of testing packages continued Timers measure the elapsed running times Accuracy checks ensure validity of output files Often requires determination of acceptable error bounds

15 How the test packages are used Run all test cases on 5 different DSRC machines to acquire times Debug test packages Quantify variation across/within machines Compare times to proposed systems

16 Machine attributes Architectures Used in Study DSRCNameMakeModelChip Set Processor Speed (GHz) Interconnect Number of Cores Cores per Node Operating System ERDCDiamondSGIAltix ICE Intel Xeon QC2.8 DDR4 InfiniBand153608SUSE Linux MHPCCManaDell PowerEdge M610 Intel Xeon QC2.8 DDR InfiniBand92168Linux NAVYDaVinciIBMPower6 IBM P6 DC4.7 DDR Infiniband480032AIX NAVYEinsteinCrayXT5 Cray Opteron QC2.3SeaStar2+127368CNL ERDCGarnetCrayXE6 AMD Opteron 64-bit2.4Cray Gemini2022416CLE

17 RESULTS! Graphs of runtimes

18 Risk Assessment: Major Areas Assessed Compliance assessment –Ability to follow benchmark rules –Number of test case results provided –Results within accuracy criteria Assessment of risk in meeting proposed times in acceptance tests –Differences between benchmarked and proposed system Processor, interconnect, and I/O system differences –Quality of estimation procedure Quality of explanation and soundness of estimation procedure –Aggressiveness of final estimate Comparison with measured benchmark system times Comparison with predicted times Assessment of likelihood of users and/or developers using proposed code modifications –Acceptability of proposed code modifications

19 Benchmarking website URL:

20 Benchmarking website continued Narrative of website purpose, codes tested Heatmap of systems best suited for applications

21 Benchmarking website continued Brief description of application Brief description of test cases

22 Benchmarking website continued An example of how we made the heatmap for allocation choices

23 Benchmarking website continued Got a question? Want to suggest an improvement? Contact us.

24 Performance Team Members Mark Cowan – ERDC – Chair Larry Davis – HPCMPO Lloyd Slonaker – AFRL Tim Sell – AFRL Laura Brown – ERDC Mahbubur Rashid – ERDC Christine Cuicchi – NAVO Matt Grismer – AFRL Jerry Boatz – AFRL

25 Performance Team Advisors William Ward – HPCMPO Steve Finn – DTRA Carrie Leach – ERDC Paul Bennett – ERDC Tom Oppe – ERDC Henry Newman – Instrumental Michael Laurenzano – SDSC Bronis de Supinski – LLNL Joseph Swartz – LM Allan Snavely – SDSC Laura Carrington – SDSC Robert Pennington – NSF Nick Wright – NERSC James Ianni – ARL

26 Questions?

27 Contact me… Mark Cowan USACE ERDC ITL Computational Analysis Branch 3909 Halls Ferry Road Building 8000, Room 1255 Vicksburg, MS 39180 (601) 634-2665


29 AVUS: Code description CFD code, formerly COBALT_60 Simulates 3-D turbulent viscous flow over irregular geometries Grid-based, reads a large grid file AVUS: 29K lines of Fortran 90 code Uses ParMETIS: 12K lines of C code Parallelism via MPI, no OpenMP Runs on Cray XT, IBM Power, SGI Altix, Linux clusters

30 CTH: Code description CTA: CSM (Computational Structural Mechanics) Shock Physics Two-step, 2 nd order accurate Eulerian algorithm is used to solve the mass, momentum, and energy conservation equations An explicit approach that does not require solving a linear system Has both static and adaptive mesh capabilities Parallelism via MPI 900K LOC, 58% FORTRAN and 42% C Uses NetCDF, supplied with distribution

31 GAMESS: Code description CTA: CCM (Computational Chemistry, Biology, and Materials Science) Ab Initio Quantum chemistry Computes many energy integrals with molecular data in form of atom positions and electron orbitals Communication depends on platform LAPI, Sockets, SHMEM, MPI Code composition: 99% FORTRAN, 1% C

32 HYCOM: Code description CTA: Climate/Weather/Ocean Modeling and Simulation (CWO) A primitive equation ocean general circulation model Communication is MPI (MPI-2 is available) 100% FORTRAN Version 2.2.27

33 HYCOM: MPI-2 details HYCOM may be run with MPI or MPI-2 MPI-2 is MPI with additional features such as parallel I/O, dynamic process management and remote memory operations HYCOM utilizes parallel I/O feature Parallel I/O times required starting with TI-10

34 ICEPIC: Code description CTA: Computational Electromagnetics and Acoustics (CEA) Particle-in-cell plasma physics code Ions and electrons move under influence of electromagnetic fields Particles are updated in a grid-free manner; grouped in cells which are periodically adjusted to preserve load balance Fields calculated on a structured, static grid and dual grid according to Maxwell's Equations Can simulate plasmas contained in complex geometries Used in electromagnetic device design ~350K lines of code, 100% C++, C

35 LAMMPS: Code description CTA: CCM (Comp Chemistry, Biology, & Material Science) Classical molecular dynamics code that models particles in a liquid, solid, or gaseous state Calculates atomic velocities, positions, system energy, and temperature After equilibration: surface tension, radial pressure, and phase change Post-processing: pair-correlation function and diffusion coefficients All actions occur within box (usually orthogonal) Distributed-memory message-passing parallelism (MPI) Highly-portable C++ Libraries needed: MPI and single-processor FFT

36 ADCIRC: Code description ADCIRC Coastal Circulation and Storm Surge Model Solves time dependent, free surface circulation and transport problems in 2 and 3 dimensions. Use the finite element method in space, which permits highly flexible, unstructured grids. Typical ADCIRC applications have included: Modeling tides and wind driven circulation, Analysis of hurricane storm surge and flooding, Dredging feasibility and material disposal studies, Larval transport studies, and Near shore marine operations

37 “BASE” ALEGRA: Code description ALE code -- Arbitrary Lagrangian-Eulerian -- provides flexibility, accuracy and reduced numerical dissipation over pure Eulerian code; modern remeshing technology allows for robust mesh smoothing and control. Hydrodynamic and solid dynamics Models large distortions and strong shock propagation in multiple-materials Finite element code; descendent of PRONTO and uses some CTH Eulerian technology Energy deposition and explosive burn models Geometry -- 2D/3D Cartesian, 2D cylindrical Material Models in ALEGRA: Equations of State Elastic-Plastic Models Fracture Models Pressure and temperature during formation of jet from shaped charge

38 “BASE” ALEGRA: Code description

39 ALEGRA_MHD: Code description All hydrodynamics/solid dynamic modules of "base" ALEGRA PLUS magnetic field and thermal transport effects Lorentz forces, Joule heating, thermal transport and simple models for radiating excess energy 2D and 3D versions 2D modeling with the magnetic flux density vector components in or out of the plane with the corresponding current density out of or in the plane, respectively. 3D uses a magnetic diffusion solution based on edge and face elements which maintains the discrete flux divergence-free property during magnetic solve and constrained transport remap stage Lumped element coupled circuit equations Magnetic and thermal conduction Advanced models for thermal and electrical conductivity Emission model radiates excess energy when medium is optically thin while accounting for reabsorption

Download ppt "HPCMP Benchmarking and Performance Analysis Mark Cowan USACE ERDC ITL in support of DoD HPCMP Tuesday, April 17, 2012."

Similar presentations

Ads by Google