1. Development of Verification and Validation Procedures for Computer Simulation use in Roadside Safety Applications NCHRP 22-24 DEFINITIONS AND PROCEDURES Worcester Polytechnic Institute Battelle Memorial Institute Politecnico di Milano

2. Roadside Safety Hardware Guardrails, bridge railings, crash cushions, breakaway light poles and signs, etc. Decisions traditionally based on full-scale crash testing. Formal crash testing specifications since the 1960’s. Today: – NCHRP Report 350 in the US – EN1317 in Europe Acceptance from FHWA primarily based on crash testing.

3. FEA IN ROADSIDE HARDWARE DESIGN Specialty codes prior to 1990 (i.e., NARD, Guard, BarrierVII, etc.) Shift to DYNA/LSDYNA in mid-1990’s Today – Almost exclusively LSDYNA – Used in nearly all new product developments – Requests for approval now coming based partly or entirely on LSDYNA results. – Decision-makers need a way to judge good from bad results. Who do you trust? How do you make an acceptance decision based on simulations?

4. MOTIVATION We have formal standards on how to perform and evaluate full-scale crash tests. Decision makers want a formal standard on how to perform and evaluate FEA simulations used in the approval process. NCHRP 22-24 was initiated to develop these procedures. Develop a procedure and format for validation and verification report for simulations that can be used like a crash test report.

5. OBJECTIVE … to develop guidelines for verification and validation of detailed finite element analysis for crash simulations of roadside safety features. The focus of these guidelines will be on establishing accuracy, credibility, and confidence in the results of crash test simulations intended (1) to support policy decisions and (2) to be used for approval of design modifications to roadside safety devices that were originally approved with full-scale crash testing.

6. TASKS 1.Literature review 2.Survey of practice 3.Verification metrics and procedures 4.Validation metrics and procedures 5.Draft guidelines 6.Interim report 7.Interim report review 8.Execute the plan 9.Test cases 10.Review guidelines 11.Recommendations 12.Final report

7. Meeting Agenda 9:00-9:30Introductions/Instructions (Niessner, Focke) 9:30-10:30Definitions and V&V Procedures (Ray) 10:30-11:30ROBUST Project Summary (Anghileri) 11:30-NoonSurvey Results Noon-1:00Lunch 1:00- 2:30V&V Metrics (Ray) 2:30-4:00Future Work for Task 8 (Ray)

8. These are domain-specific guides with more step-by-step procedures and defined metrics. These are general purpose, broad- based guides that out-line general procedures and provide terminology definitions. They are not step-by-step guides. Existing V&V Procedures NASA DoD AIAA Sandia Los Almos LLNL ASME FHWA/NARD FAA FRA Several organizations have developed V&V procedures in recent years. ASME Guide draws on all the above guides in developing its recommendations. Lockhead EU (rail) Chrysler

9. ASME V&V 10-2006 History 1999 An ad hoc verification & validation specialty committee was formed under the auspices of the United States Association for Computational Mechanics (USACM). 2001 ASME approved the committee’s charter: To develop standards for assessing the correctness and credibility of modeling and simulation in computational solid mechanics. Committee was assigned the title and designation of the ASME Committee for Verification & Validation in Computational Solid Mechanics (PTC 60). 2006 ASME published the “Guide for verification and validation in computational solid mechanics.” ASME V&V 10-2006. 2007 Developing a series of “best practices guides”

10. 10 ASME V&V 10-2006 Committee Members M. C. Anderson, Los Alamos National Laboratory J. A. Cafeo, General Motors Corporation R. L. Crane, The American Society of Mechanical Engineers S. W. Doebling, Los Alamos National Laboratory J. H. Fortna, ANSYS M. E. Giltrud, Defense Threat Deduction Agency J. K. Gran, SRI International T. K. Hasselman, Acta Inc. H. M. Kim, Boeing R. W. Logan, Lawrence Livermore National Laboratory H. U. Mair, Institute for Defense Analyses A. K. Noor, Old Dominion University W. L. Oberkampf, Sandia National Laboratories J. T. Oden, University of Texas D. K. Pace, Consultant T. Paez, Sandia National Laboratories A. B. Pifko, Consultant L. Proctor, MSC Software J. N. Reddy, Texas A & M University P. J. Roache, Consultant L. E. Schwer, Schwer Engineering P. E. Senseny, Consultant M. S. Shephard, Rensselaer Polytechnic Institute D. A. Simons, Northrop Grumman B. H. Thacker, Southwest Research Institute T. G. Trucano, Sandia National Laboratories R. J. Yang, Ford Motor Company Y. Zhao, St. Jude Medical The committee derives its authority from the diversity of its membership and consensus of opinion. The committee derives its authority from the diversity of its membership and consensus of opinion.

11. 11 ASME V&V 10-2006 Outline 1.Introduction – the general concepts of verification and validation are introduced and the important role of a V&V Plan is described. 2.Model Development – from conceptual model, to mathematical model, and finally the computational model are the keys stages of model development. 3.Verification – is subdivided into two major components: code verification - seeking to remove programming and logic errors in the computer program, and calculation verification – to estimate the numerical errors due to discretization approximations. 4.Validation – experiments performed expressly for the purpose of model validation are the key to validation, but comparison of these results with model results depends on uncertainty quantification and accuracy assessment of the results.

12. ASME V&V 10-2006 The Guide does provide a: Framework and process for V&V activities. Standard definitions for V&V terms. The Guide does not provide: A step-by-step procedure for V&V. Specific recommendation for metrics.

13. ASME V&V 10-2006 Some Definitions Validation -- The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. – Model results are compared to physical experiments. Verification -- The process of determining that a computational model accurately represents the underlying mathematical model and its solution. – Model results are compared to known mathematical solution. Calibration -- The process of adjusting physical modeling parameters in the computational model to improve agreement with experimental data. – Physical experiments used to estimate model parameters.

14. ASME V&V 10-2006 Validation The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model.

15. ASME V&V 10-2006 Verification The process of determining that a computational model accurately represents the underlying mathematical model and its solution.

16. ASME V&V 10-2006 Calibration The process of adjusting physical modeling parameters in the computational model to improve agreement with experimental data.

17. The airplane is manufactured by Karel Klenor - KLN, Choceň, the firm is one of the biggest producers of composites in the Czech Republic www.kln.cz “We need to know the wing tip deflection of the ABC experimental aircraft under a distributed load of X Newtons/meter,” in this case the reality of interest is the aircraft wing. Before we begin to develop a model, a reality of interest is identified (i.e., what is the physical system to be modeled). ASME V&V 10-2006 Model Development

18. Conceptual Model – “the collection of assumptions and descriptions of physical processes representing the solid mechanics behavior of the reality of interest from which the mathematical model and validation experiments can be constructed.” ASME V&V 10-2006 Model Development

19. Mathematical Model – “The mathematical equations, boundary values, initial conditions, and modeling data needed to describe the conceptual model.” ASME V&V 10-2006 Model Development

20. Computational Model – “The numerical implementation of the mathematical model, usually in the form of numerical discretization, solution algorithm, and convergence criteria.” Commercial Software ASME V&V 10-2006 Model Development

21. ASME V&V 10-2006 V&V Process

22. Verification ASME V&V 10-2006 V&V Process

23. Validation Verification ASME V&V 10-2006 V&V Process

24. 24 If the model passes the comparison tests in the V&V Process Plan, then it can be used to make the desired predictions with confidence. When it is said that the model is validated for the intended use, it is not the just the Computational model, which likely will have to change for the predictions of interest, but the Mathematical and Conceptual models upon which the Computation model was built that have been validated. It is through the Validation of the Conceptual model that confidence is gained that the correct physics (mechanics) were included in the model development. ASME V&V 10-2006 What is a validated model?

25. 25 The goal of the validation process is to assess the predictive capability of the model by comparing the predictive results of the model with validation experiments. Three key elements of Validation: 1.Precision Testing 2.Uncertainty Quantification 3.Comparative Metrics ASME V&V 10-2006 Validation Process

26. ASME V&V 10-2006 Comments on the V&V Process The V&V process diagram is valid not only for whole models but for components, assemblies, parts, etc. While most roadside safety work uses LSDYNA, this process and definitions are applicable to any numerical simulation software (e.g., MADYMO, BVII, HVOSM, HVE, etc.). We can not usually do code verification – we do not generally have access to the code. – Counter example: Yvonne Murray’s soil and timber models for LSDYNA. We can do calculation verification – this is another word for benchmarking. – Example: do different versions of LSDYNA produce the same result? Do different computational platforms produce the same result? Notice the comparison is quantitative. – Qualitative validation is not really validation because it is subjective.

27. Recommendation The project team recommends that we adopt the ASME V&V 10-2006 Guide as a basis for the basic V&V process and definition of terms. ASME V&V 10-2006 Validation -- The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. Project 22-24 Panel Validation is the process of assessing and improving the confidence in the usefulness of the computational model for real- world applications. … Validation is making sure the model truly represents reality, within a acceptable range of tolerance. Comparison is between a model solution and a physical experiment.

28. Recommendation The project team recommends that we adopt the ASME V&V 10-2006 Guide as a basis for the basic V&V process and definition of terms. ASME V&V 10-2006 Verification -- The process of determining that a computational model accurately represents the underlying mathematical model and its solution. Project 22-24 Panel Verification ensures that the computer model behaves properly according to basic physical laws and properties as allowed by the capabilities of the code from which it is built. … In more simple terms, verification is making sure the model works right as far as the computer code (or application) is concerned. Comparison is between a model solution and a known mathematical solution.

29. Recommendation The project team recommends that we adopt the ASME V&V 10-2006 Guide as a basis for the basic V&V process and definition of terms because … The 22-24 panel definitions are more or less consistent with the ASME definitions and … The ASME definitions have been adopted by a broad cross-section of the computational solid mechanics community and... The ASME process includes all the essential pieces needed in computation roadside safety. Discussion?

30. Hierarchical Modelling The ultimate goal is to use a validated model to extrapolate results to an untested situation. We need to have confidence in the model before we can use it to predict untested situations.

31. Hierarchical Modelling Vehicle a ssembly Barrier a ssembly Whole model level Top Rail assembly Middle Rail assembly Rubrail part Postpart Assembly Level Guardrail part Spacer part Blockout part Stiffnerparts Main-rail part Posts parts Part Level

32. Hierarchical Modelling