1. NDIA CREATE-D.Post 2/1/2011 Page-1 Distribution Statement A: Approved for public release; distribution is unlimited. The DoD Computational Research and Engineering Acquisition Tools and Environments (CREATE) Program Dr. Douglass Post Chief Scientist and CREATE Program Manager DoD High Performance Computing Modernization Program NDIA Workshop, 1 Feb 2011, Arlington, VA

2. NDIA CREATE-D.Post 2/1/2011 Page-2 Distribution Statement A: Approved for public release; distribution is unlimited. DoD HPC Modernization Program

3. NDIA CREATE-D.Post 2/1/2011 Page-3 Distribution Statement A: Approved for public release; distribution is unlimited. DoD HPC Modernization Program In 2011, DoD HPCMP will have 5 centers, with 2x10 15 FLOPs & 200,000 processors High Bandwidth, Low Latency Network Linking 284 sites Gather Service and Agency Requirements and Allocate Resources Software Development Supports 5 Institutes, CREATE and Education

4. NDIA CREATE-D.Post 2/1/2011 Page-4 Distribution Statement A: Approved for public release; distribution is unlimited. What Do We Want To Accomplish Today? Describe the DoD CREATE program and paradigm Describe how CREATE and the paradigm of multiphysics-based computational engineering can help the US defense industry design, deliver and sustain DoD weapons systems in less time with lower risks Start engaging with US defense industries to ensure that CREATE will help enable US defense industries to directly support DoD acquisition Describe the current and future technical capabilities of CREATE Begin to use NDIA as a vehicle to connect CREATE and the HPCMP computer centers to the US Defense Industry

5. NDIA CREATE-D.Post 2/1/2011 Page-5 Distribution Statement A: Approved for public release; distribution is unlimited. Acquisition Takes Too Long! Major root causes: Rapidly changing requirements Continuing pressure to deliver complex systems faster A build and test methodology that results in problems being discovered late in the acquisition process Root causes lead to: Extensive testing to identify flaws and integration problems Extensive rework to fix design flaws and integration problems Impact: Delays to fix problems Large technical, schedule and budget risks

6. NDIA CREATE-D.Post 2/1/2011 Page-6 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE Programmatic Objectives and Concept Objective: Sustain the US military competitive advantage by developing and deploying scalable multiphysics-based computational engineering tools to design and analyze DoD weapons system performance –Produce optimized designs with fewer defects and design flaws –Reduce testing and rework to enable faster system development and deployment How? –Supplement physical testing with multiphysics-based computational performance predictions specifically targeted to complex DoD systems Ancillary benefit –Higher productivity for the shrinking number of US defense community engineers and scientists

7. NDIA CREATE-D.Post 2/1/2011 Page-7 Distribution Statement A: Approved for public release; distribution is unlimited. Testing Conundrum Weapons systems are increasing in complexity –Need more complex testing in face of pressure to reduce testing facilities and test time for all phases of acquisition Multiphysics-based performance predictions can improve testing effectiveness and efficiency –Supplement and partially replace real prototypes with virtual prototypes –Frees testing community to identify and understand the phenomena that determine system behavior Example 1 : For wind tunnels, physics-based computational engineering models can REDUCE: 1.“Overall testing workload through identifying the most important data to take rather than “One Factor at a Time” approaches and using condensing calculated performance and accessing it with non-linear systems identification (SID) methods 2.Down stream effects of late defect discovery on total development cycle time 3.Amount of testing required to determine support interference or wind tunnel wall effects or to better understand scaling issues from wind tunnel to flight conditions” 1 Kraft, E., “Integrating Computational Science and Engineering with Testing to Re-engineer the Aeronautical Development Process.” AIAA 2010-139. CREATE AV Board of Directors chair

8. NDIA CREATE-D.Post 2/1/2011 Page-8 Distribution Statement A: Approved for public release; distribution is unlimited. Present Product Development Process Iterated Design  Build  Test Cycles Long time to market –Requires many lengthy and expensive design/build/test iteration loops Process converges slowly –Process is rigid, not responsive to new requirements –Design flaws discovered late in process leading to rework –Systems Integration happens late in process Build Physical Product RequirementsDesign Test Physical Product Manufacture, Sustain, and Modify (Many) Design iterations F-22 Flight Test

9. NDIA CREATE-D.Post 2/1/2011 Page-9 Distribution Statement A: Approved for public release; distribution is unlimited. Reduced design and development time –Highly scalable computational performance analysis of virtual prototypes reduces the need to test real prototypes Process converges much faster –Process is flexible, very responsive to new requirements –Design flaws early in process reducing rework –Systems Integration happens at every step of the process Build Mesh RequirementsDesign on Computer Analyze Performance Manufacture, Sustain, and Modify (Many) Design iterations MultiPhysics-Based Performance Analysis Increases Productivity for Complex Systems Ground- based and Flight Tests

10. NDIA CREATE-D.Post 2/1/2011 Page-10 Distribution Statement A: Approved for public release; distribution is unlimited. Analyze and Test Virtual Product Requirements Build and Test Physical Product Market Design iterations Reduced time to market from 3 years to less than 1 year In 2009, Goodyear introduced 62 new products using this process This Paradigm Has Been Proven Successful in Industry Design and Mesh Virtual Product Miller, L.K., “Simulation-Based Engineering for Industrial Competitive Advantage.” Computing in Science and Engineering, 12, pp. 14–21, 2010. Flight Radial Fortera TripleTred

11. NDIA CREATE-D.Post 2/1/2011 Page-11 Distribution Statement A: Approved for public release; distribution is unlimited. Performance Analysis of Virtual Prototypes Is the Key Concept Development Engineering DevelopmentPost Development Material Solution Analysis Technology Development Engineering and Manufacturing Development Production and Deployment Operation and Support BAC IOCFOC AOA Mission Need Determination Experimental Sub-System Prototypes Experimental System Prototypes Inject physics into design early and all through the process! Virtual Integrated Prototyping Environment

12. NDIA CREATE-D.Post 2/1/2011 Page-12 Distribution Statement A: Approved for public release; distribution is unlimited. Enabling Technology The 10 15 increase in computer power since 1945 can enable us to develop and deploy codes during the next decade that are much more powerful than past tools: –Utilize accurate solution methods –Include all the effects we know to be important –Model a complete system –Complete parameter surveys in hours rather than days to weeks to months In ~ 10 years, workstations will be as powerful as today’s supercomputers Greatest opportunities for 2020 (and 2010) include large-scale codes that integrate many multi-scale effects to model a complete system Moore’s “Law” 10 -6 0.0001 0.01 1 100 10 4 10 6 10 8 194019501960197019801990200020102020 Computing Power For The World's Fastest Computer Floating Point Operations/sec Performance (GFLOPs/sec) Year Cores Supercomputers Workstation Performance

13. NDIA CREATE-D.Post 2/1/2011 Page-13 Distribution Statement A: Approved for public release; distribution is unlimited. What is CREATE? CREATE is a DoD program to develop and deploy multiphysics-based software for engineering design and analysis of: Air Vehicles (AV) –Aerodynamics, structural mechanics, propulsion, control, … Ships –Shock vulnerability, hydrodynamics, concept design Radio Frequency (RF) Antennas –RF Antenna electromagnetics and integration with platforms Mesh and Geometry (MG) Generation –Rapid generation of mesh and geometry representations F-35 CREATE tools support all stages of acquisition from rapid early stage design to full life-cycle sustainment

14. NDIA CREATE-D.Post 2/1/2011 Page-14 Distribution Statement A: Approved for public release; distribution is unlimited. DaVinci targets early-phase acquisition engineering (conceptual design) Firebolt is a module for propulsion systems in fixed and rotary-wing air vehicles. Module includes engineering models of propulsion systems suitable for conceptual studies and also high-fidelity, multi-physics simulation. A CREATE-AV Software Product Fire bolt Helios is a high-fidelity, full-vehicle, multi-physics analysis tool for arbitrary rotary-wing aircraft Helios Kestrel is a high-fidelity, full-vehicle, multi-physics analysis tool for arbitrary fixed-wing aircraft CREATE-AV Product Development and Deployment

15. NDIA CREATE-D.Post 2/1/2011 Page-15 Distribution Statement A: Approved for public release; distribution is unlimited. Rapid Design and Integration –Rapid development, assessment, and integration of candidate ship designs to avoid cost versus capability mismatches –Partnership with ONR & NAVSEA Ship Shock & Damage –Provide analysis of shock and damage effects and reduce need for tests to assess ship shock and damage effects Ship Hydrodynamics –Accelerate and improve all stages of ship hydrodynamic design (seaway loads, sea keeping, resistance…) CREATE – Ships Product Components

16. NDIA CREATE-D.Post 2/1/2011 Page-16 Distribution Statement A: Approved for public release; distribution is unlimited. SENTRI enables antenna design integrated with platforms –Able to calculate multiple antenna arrays for small- and large-scale systems (small UAVs  ships) ●Staged capability –Rapid design – coarse resolution with fast, first-order codes for small antenna systems –High fidelity detailed design, high-resolution complex antenna systems integrated with large platforms –Enabled by improved computational algorithms (n 3  n log n scaling improvement) CREATE – Radio Frequency (RF) Product Component Small Less Complex Larger Highly Complex

17. NDIA CREATE-D.Post 2/1/2011 Page-17 Distribution Statement A: Approved for public release; distribution is unlimited. CAPSTONE provides geometry and meshing needs for all phases of acquisition engineering (conceptual-, preliminary-, detailed-design and operational-support) Enable parametric, associative geometry and meshes in AV:DaVinci, Ships:RDI; geometry-based mesh adaptivity Produce analyzable representations for complex and detailed analysis CAPSTONE: Frontend CAPSTONE: Core CAPSTONE – CREATE-MG Product Overview

18. NDIA CREATE-D.Post 2/1/2011 Page-18 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE Is a Multi-Service, Multi-institutional Program HPCMP Executing Agent HPCMP Director CREATE Program Integrated Hydro Design Environment NSWC-Carderock Kestrel Eglin AFB Shadow-Ops NAVAIR, Patuxent River Helios Army, Ames DaVinci ASC & AFRL Firebolt AEDC SENTRI Official HPCMP Advisory Panel Ships Project NSWC- Carderock Air Vehicles Project NAVAIR, Patuxent River RF Antennas Project AFRL, WPAFB Mesh & Geometry Project NRL Capstone Rapid Design NAVSEA 05D NAVYRANS NSWC- Carderock, Navy Enhanced Sierra Mechanics NSWC-Carderock

19. NDIA CREATE-D.Post 2/1/2011 Page-19 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE is a Distributed Set of Teams at DoD Engineering Centers SNL AFRL SPAWAR AMES U of Wy 46 th Test Wing ETI ASC AFRL U of Iowa AEDC Pax River MGAVRFShips SNL U of Mich HPCMP Carderock NAVSEA Indian Head ONR NRL HPCMP CERDEC Penn State

20. NDIA CREATE-D.Post 2/1/2011 Page-20 Distribution Statement A: Approved for public release; distribution is unlimited. The CREATE Approach Software is being built by government-led teams Each product is released following a roadmap –Each year there is a release of a usable application –Each release builds on the previous release and adds the increased capability called for in the roadmap –Each release is beta-tested by targeted user communities before a broader release –Each release goes through a rigorous V&V process Releases are progressively more scalable for massively parallel computers and responsive to user requirements CREATE Program is guided by DoD service acquisition engineering organizations and their senior leadership and US defense industry (where possible)

21. NDIA CREATE-D.Post 2/1/2011 Page-21 Distribution Statement A: Approved for public release; distribution is unlimited. Software Engineering Focused on Producing High Quality Engineering Software Tools Goal is maintainable, extensible, portable and reliable software products developed with the right balance of agility and flexibility and short and long term planning based Use Software Engineering Processes Specifically Designed for Technical Software Documents: ICD +…. –Manuals: Technical, Developer, User (+ training, tutorials, sample input and output,…) V&V is a key priority –Verification at every level –Validation as much as feasible (large potential role for defense industry) –Beta testing a key opportunity for defense industry input User support and training

22. NDIA CREATE-D.Post 2/1/2011 Page-22 Distribution Statement A: Approved for public release; distribution is unlimited. HPCMP Executing Agent Cray Henry (Director) CREATE Program Douglass Post, IPA (PM) Luanne Handley-Blair (Dep PM) CREATE is Guided by Senior Science and Technology Leaders From the Services Official HPCMP Advisory Panel Ships Myles Hurwitz, IPA (PM) Robert Keane (con.) Air Vehicles Robert Meakin, IPA (PM) Chris Atwood (Dep) RF Antennas John D’Angelo (PM) Board of Directors *RADM Thomas Eccles, NAVSEA Chief Engineer Dr. Walter Jones, SES, Executive Director, ONR Mr. C.F. Snyder, SES, 33Technical Director, NSWC, Carderock Division Mr. Glen Sturtevant, Director, S&T, PEO, SHIPS Mr. Cray Henry, Director, HPCMP Board of Directors *Dr. Ed Kraft, ST, Chief Technologist, AEDC/CZ Samuel T Crews, U.S. Army Aviation and Missile Research, Development, and Engineering Center Mr. Richard Gilpin, Director, AIR-4.3 NAVAIR Mr. Tom Rudowsky, Director, NAVAIR), (IBST) Dr. Michael Scully, ST, Aeroflightdynamics Directorate (AMRDEC), U.S. Army RDECOM Mr. Cray Henry, Director, HPCMP Board of Directors *Dr. Frederick Tokarz, SES, Senior Leader, AF ASC Engineering Directorate Mr. Dennis DeCarlo, Chief Antenna Systems Engineer, NAVAIR Dr. Betsy Delong, ONR RF Antenna contact PEO (SHIPS) Dr. Janet Fender, SES, Chief Scientist, AF Air Combat Command Dr. Mahbub Hoque, Chief Scientist (STCD), CERDEC Dr. Robert Peterkin, ST, Chief Scientist, AFRL Directed Energy Directorate Mr. Cray Henry, Director, HPCMP Board of Directors *Dr. Robert Meakin, PM CREATE Air Vehicles Mr. Myles Hurwitz, CREATE Ships Project Manager Dr. Sudip Sodanjh, Computational Engineering Leader, SNL Dr. John D’Angelo, CREATE RF Antenna Project Manager Mr. Cray Henry, Director, HPCMP *Board of Director Chairs MG Saikat Dey (PM)

23. NDIA CREATE-D.Post 2/1/2011 Page-23 Distribution Statement A: Approved for public release; distribution is unlimited. 36 Months After Program Start, CREATE Has Delivered Usable Software 5 beta releases in FY09/10 with V&V and ~150 beta testers: –Helios 1.0—Accurate calculation of rotorcraft vortex shedding –Kestrel 1.0—Rigid body CFD fixed wing AV with preliminary aeroelastics –SENTRI 1.0 and 1.5—Initial RF antenna design and analysis with V&V –NESM 0.1—Initial ship shock vulnerability analysis for underwater explosions –IHDE 1.0—Initial user interface for ship hydrodynamics 10 releases planned for FY10/11 and each succeeding year Helios 2.0, Kestrel 2.0, SENTRI 2.0, NESM 1.0 and IHDE 2.0 plus: –Rapid design: RDI 1.0 for Ships, DaVinci 1.0 for Aircraft plus SENTRI 2.0 –Components: CAPSTONE 1.0 for geometry and mesh, Firebolt 1.0 for gas turbines –Detailed ship hydrodynamics NavyFoam 1.0 (seakeeping, drag, resistance,…) Developing approaches to improve scalability Antenna Radiation Near Fields Free Space More accurate vortex shedding Kestrel vs. F-16 Flight Data M=0.95, Alt.=10,000 ft

24. NDIA CREATE-D.Post 2/1/2011 Page-24 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE-AV Developed An Architecture to Facilitate Multi-physics Integration Aerodynamics Solvers Structural Dynamics Solvers “Light-Weight” Infrastructure – Highly Scalable and Modular (available as executable) Well defined APIs available for use by industry for proprietary modules OR for DoD to leverage innovations from research community Event-Based Infrastructure Unstruc Strand Cartesian FEM Comprehensive Fluid Structure Interaction Propulsion Control Laws Curvilinear Autopilot Aircraft Trim 6-DOF On-the-Fly Data Harvesting … … Research Tools Industry Tools Stubs represent Application Program Interfaces (APIs)s Force/Moment Calculator

25. NDIA CREATE-D.Post 2/1/2011 Page-25 Distribution Statement A: Approved for public release; distribution is unlimited. POC: Ms. Ryan Fitzgerald, FQ Engineer NAVAIR 4.3.2.5 New Forward Component Early Success Rapid Deployment of EP-3E Shadow-Ops: CREATE staff use computational tools to support acquisition programs  provide experience and establish connections and value Performed CFD analysis of impact of electronic countermeasure pod for EP-3E flight clearance–Not sufficient time for conventional process (flight tests) –Eliminated construction cost of wind tunnel model and tests and need for contractor flying quality report. –Provided aircraft flying qualities characteristics within required time frame. –Provided data required to issue flight clearance in time for direct deployment. Reduced overall program cost and time Only 1 flying qualities flight test required – Saving between 3–4 flight tests System was deployed in the forward theatre in less than four months instead of twelve

26. NDIA CREATE-D.Post 2/1/2011 Page-26 Distribution Statement A: Approved for public release; distribution is unlimited. HPCMP Computer Centers CREATE is part of the DoD High Performance Computing Modernization Program (HPCMP) The HPCMP has five large computer centers with 200,000 cores and 2 Peta-FLOPS, and upgrades this capability annually CREATE is being linked to these centers HPCMP has a vision to open up their computer centers to the US defense acquisition community (both government and industry)

27. NDIA CREATE-D.Post 2/1/2011 Page-27 Distribution Statement A: Approved for public release; distribution is unlimited. What the HPCMP Offers Approximately a dozen large HPC systems from a variety of vendors at five DoD Shared Resource Centers Connectivity to these centers via a capable wide-area network – the Defense Research and Engineering Network Expert help in all aspects of setting up accounts and using HPC systems through the Consolidated Customer Assistance Center and the User Productivity Enhancement, Technology Transfer and Training program Access to many commercial applications software packages for analysis HPCMP is seeking descriptions of industry use cases for high performance computer usage –Problem domains, engineering workflows, user interfaces, software tools, entry and access processes, software application requirements, etc.  questionnaire in back of room All members of the DoD RDT&E community are eligible to access and use these capabilities. All members of the DoD RDT&E community are eligible to access and use these capabilities.

28. NDIA CREATE-D.Post 2/1/2011 Page-28 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE Is For Government and US Defense Industry Engineers CREATE can reduce the acquisition time and risks for complex DoD air vehicles, ships and RF antennas CREATE adoption by DoD engineering community is important for US defense competitiveness CREATE, through the NDIA, is seeking your guidance and input to develop tools that will be useful to you and will be adopted and used by you

29. NDIA CREATE-D.Post 2/1/2011 Page-29 Distribution Statement A: Approved for public release; distribution is unlimited. But…. Constraints exist –Need to maintain US competitive advantage  US government control of distribution (e.g. export control), and retention of unlimited use rights –Must provide “Equal and fair access” for all the US defense companies Support needs can overwhelm the small CREATE development teams (~ 5 to 7 developers per team) –10 teams for a total program staff ~ 75 professionals (government, industry and academia) –Software is maturing rapidly but is generally not yet ready for heavy usage –CREATE will need to build up the capability to support their user community –Support effort needs to be balanced with development, and we likely need more resources Tools are not yet mature, but are maturing –Early CY2011, ~ 5 products are releasing version 2 –Late CY2011, other 5 will release either version 1 or 2 –CY2012, repeat with version n+1

30. NDIA CREATE-D.Post 2/1/2011 Page-30 Distribution Statement A: Approved for public release; distribution is unlimited. CREATE Needs Your Help to Ensure That You Can Use The CREATE Tools CREATE needs beta testers –Get feedback from your engineers to ensure that we meet your needs and don’t spend years producing tools that are not useful CREATE needs information on your workflows and design processes –Need to know what our targeted user community needs CREATE needs product models and validation data –Codes are only a model of nature and must be validated –Need good initial conditions and validation data –CREATE team has security clearances and access to secure computers (export controlled  classified computing at all levels) CREATE needs your recommendations for the technical capability required for your problems –NDIA Participation in technical reviews and discussions of the CREATE tools –Tell us “Have we got it right?” and “How we can make it better.”

31. NDIA CREATE-D.Post 2/1/2011 Page-31 Distribution Statement A: Approved for public release; distribution is unlimited. Additional References

32. NDIA CREATE-D.Post 2/1/2011 Page-32 Distribution Statement A: Approved for public release; distribution is unlimited. Physics-based Engineering Software Helped The US Win Cold War. Nuclear weapons are complex, expensive, and hard to test ~ 5 to 10 tests per system DOE NNSA uses computational tools for: Design development, optimization, & analysis. DOE NNSA labs own the biggest supercomputers 1945 Atomic Bombs Heavy Hydrogen Bombs ICBM (Lighter,smaller) MIRV ( even lighter, smaller ) Improved yield to weight SLBM ( even lighter, smaller ) Improved safety Improved robustness 2010 Computer Power 0.000000001 GigaFlops/s 1,000,000 GigaFlops/s TestingWeapon CapabilityComputational Design Increasing Computational Design Capability Improvements over time: Solution methods Spatial resolution Temporal resolution Geometric fidelity 1-D to 2-D to 3-D Physics models ……. Underground Test ban Air Tests NIF

33. NDIA CREATE-D.Post 2/1/2011 Page-33 Distribution Statement A: Approved for public release; distribution is unlimited. It Takes a Village! Sponsors  Provide mission, resources and support − Designers—End-users  Engineers to use the tools to design products − Codes  Takes a good team ~ 10 years and ~ $100M to develop a complex code − V&V  Dedicated experiments and tests − Computers  Capability to develop codes and run the problems quickly and conveniently

34. NDIA CREATE-D.Post 2/1/2011 Page-34 Distribution Statement A: Approved for public release; distribution is unlimited. Developing These Codes Takes A Large Team A Long Time 2003 ~20 From: D. Post, R. Kendall and E. Whitney, “Case Study of the Falcon Code Project”, Proceedings of the Workshop on Software Engineering for High Performance Computing, International Conference on Software Engineering, May 15, 2005, St. Louis, Missouri.

35. NDIA CREATE-D.Post 2/1/2011 Page-35 Distribution Statement A: Approved for public release; distribution is unlimited. Development of Science-Based Software is Complex!

36. NDIA CREATE-D.Post 2/1/2011 Page-36 Distribution Statement A: Approved for public release; distribution is unlimited. Development of Science-Based Software is Risky!

37. NDIA CREATE-D.Post 2/1/2011 Page-37 Distribution Statement A: Approved for public release; distribution is unlimited. Incremental Development and Deployment in Annual Releases Rel 1 Rel 2 Rel 3 Fielding Capability,… Stages 10-12 years 1-5 years 6-9 years Develop Initial Requirements and Plans Rel 6 1 12 4 8 Rel 4 Rel 5 Rel 8 Rel 9 Rel 10 Rel 11 Rel 7 Year Improve scalability,… Physics integration tests,… PetaFlop (10 15 FLOPs) computers ExaFLOP (10 18 FLOPs) computers

38. NDIA CREATE-D.Post 2/1/2011 Page-38 Distribution Statement A: Approved for public release; distribution is unlimited. 38 Kestrel SE Docs

39. NDIA CREATE-D.Post 2/1/2011 Page-39 Distribution Statement A: Approved for public release; distribution is unlimited. 39 Helios SE Docs

40. NDIA CREATE-D.Post 2/1/2011 Page-40 Distribution Statement A: Approved for public release; distribution is unlimited. 40 Firebolt SE Docs

41. NDIA CREATE-D.Post 2/1/2011 Page-41 Distribution Statement A: Approved for public release; distribution is unlimited. 41 DaVinci SE Docs