1. Slide serial no 1 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch 3 rd European-American Workshop on Reliability of NDE and Demining, September 10-13, 2002, Berlin Germany Computer Modelling and Simulation for Prediction of POD and Inspection Reliability Martin Wall and Steve Burch UK National NDT Centre NNDTC Paper 24 Thursday 12 September 2002

2. Slide serial no 2 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Trial – Conventional Approach Why not model POD, PFI? - A Complementary approach POD trial on airframes Sandia National Labs USA

3. Slide serial no 3 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Scope of Talk Overview of POD modelling with Demonstration Background Basis of POD models Demonstration Where Models are useful Human reliability Conclusions

4. Slide serial no 4 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Modelling of POD is not new! NDE Models have been used for many years as part of the validation and qualification of NDE methods. ã v a approach [Berens] has been used for many years for evaluating POD in the US aerospace industry. Over the last 10 years increasingly sophisticated models have been developed for predicting POD notably at the UK National NDT Centre NNDTC and Iowa State University in the USA. NNDTC and BAM have separately developed signal/noise methods to predict POD. Simulation Models developed at CEA (Paper 25) could be suitable for adaptation. Current models run on standard PC's, produce simulated data and can be linked into CAD packages and probabilistic models.

5. Slide serial no 5 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Models developed at NNDTC The UK National NDT Centre started looking in early 1980s at modelling POD. Models developed include: ESA/ESTEC Models Project 12228/96 Wall/Burch1997-2000 Ultrasonic C-Scan Radiography Technique-Independent Model (image data) Ultrasonic TOFD Silk 1996 Signal-Noise Models Wedgwood/Wall 1997 Ultrasonic Pulse-echo Ogilvy 1989-92 Radiography Windsor/ Wall 1990-95 Eddy Current Holt 1995

6. Slide serial no 6 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Basis of a POD Model Calculation of single-point POD & PFI: No background signal

7. Slide serial no 7 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Basis - Single-point POD & PFI with background signal

8. Slide serial no 8 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Features of the POD Models Modular approach Model and simulator Give simulated NDE data and images POD curves and false calls (NFI) Choice of detection criteria Two independent methods of determining POD Theoretical POD - From the Physical model “ Spot the Ball” - POD trial using the Simulated data Run on a standard PC in real time

9. Slide serial no 9 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Modular Approach Signal Background, Noise Detection Criteria POD PFI Use experimental or model-based inputs dependent on status of NDT technique

10. Slide serial no 10 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Detection Criteria – seek to reflect actual practice Single point POD & PFI Probability that a single pixel exceeds the reporting threshold () High false-call rates unless threshold set very high Multi-point POD & PFI Probability that a group of pixels exceed the reporting threshold () Simulates automated inspection system Reduced PFI without needing to set high threshold Integral “POD” approximation to visual detectability of a defect (human inspector). “Spot the ball” - POD “trials” on the computer

11. Slide serial no 11 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Physical model overview Inputs Calculations Component details Inspection details Modelling Outputs Theoretical POD Simulated images “Spot the ball”

12. Slide serial no 12 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch UT model: components User defines model region (or “space”) User builds up component from any number of rectangular regions added into this region Regions can be constant thickness or “wedges” in horizontal or vertical directions User shown image of component as it is built-up

13. Slide serial no 13 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch UT model: basis (simplified) Setup model region & 2-D “map” of attenuation Add defects to attenuation image Derive transmitted amplitude “map” Convolve to represent ultrasonic beamwidth Add “white” noise Allow for data recording system dynamic range Calculate “theoretical” POD & PFI values Display modelled image on screen: available for “spot the ball” POD & PFI assessment

14. Slide serial no 14 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch UT models: defects Through-wall location ignored Defects can be rectangular or elliptical. Defects can have any opacity between 0 & 1. Can model cracks, delaminations, single pores, inclusions & regions of excess resin. Added randomly to specified rectangular region of model. No overlaps!

15. Slide serial no 15 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Ultrasonic C-Scan DEMO DEMO

16. Slide serial no 16 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Definition of Model Space - Example Windows Menu

17. Slide serial no 17 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Ultrasonic C-Scan Example 1 DEMO

18. Slide serial no 18 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Ultrasonic C-Scan Example 2

19. Slide serial no 19 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Simulated POD Trial - “Spot the Ball” POD Simulated images brought up on screen Can have zero to any number of defects. Range of defects sizes or types. Can constrain location for defects Operator clicks on cursor where they thinks there is a defect Data automatically analysed for POD and False calls By repeating can build up statistics. Analyse by grouping or ‘Hit-Miss’ method Compare with Theoretical POD from physical model. Gives second independent measure of POD

20. Slide serial no 20 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch UT model: Example of output after “spot the ball” POD trial

21. Slide serial no 21 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch X-ray model: Example of output after “spot the ball” POD trial – Model POD to right

22. Slide serial no 22 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch ESA RTM Component and Simulated C-Scan Image

23. Slide serial no 23 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch X-ray model: basis Similar in concept to UT model: calculates modelled 2-D image (film radiograph) + POD & PFI values. Physical modelling of following processes in 2-D: Poly-chromatic beam of X-rays Energy dependent absorption of X-rays in material Scattering of X-rays within the component Absorption of X-rays in the film Unsharpness/spatial resolution due X-ray source size/film characteristics.

24. Slide serial no 24 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Validation Comparison with experimental POD trials Comparison of NDE data and simulated images with experiment. By examining trends. Theoretical POD v “Spot the Ball” POD Is the model consistent with the experimental results?

25. Slide serial no 25 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD validation UT: Multi-point POD values

26. Slide serial no 26 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Experimental v Model UT C-scan Experiment Model

27. Slide serial no 27 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Model validation: UT signal amplitude

28. Slide serial no 28 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Modelled radiograph of RTM component

29. Slide serial no 29 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Experimental radiograph of RTM component with added Al discs

30. Slide serial no 30 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch POD Model Validation - Measured & predicted density differences

31. Slide serial no 31 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Technique independent model overview Calculations Outputs Inputs Expt. image Identify defect Statistical Model POD & PFI Interactive analysis

32. Slide serial no 32 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Technique independent model: example of application

33. Slide serial no 33 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Where is a POD Model useful? Application at Design Stage Assessment of Historical data Extension of existing data Parametric or ‘what-if’ calculations Optimisation of inspection Planning or in parallel with experimental trials. Alternative to trial? Validation and Qualification False call data (PFI, NFI) Simulated POD trials Optimisation of inspection and detection criteria Input to integrity assessments and economic models (IVM) Quick and cost effective

34. Slide serial no 34 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Current POD Model applications Fabrication Hydrogen cracking LPG Vessels Offshore Composite Repair Group (JIP) Railway line inspection Airframe inspection UT inspection offshore jacket structures Modelling radiography of steel weld containing rogue weld metal Models to be available via web internet – DTI MMS15 Interactive Knowledge Base (IKB) Base on NDE of composites. www.aeat.co.uk/ndt/mms15 www.aeat.co.uk/ndt/mms15

35. Slide serial no 35 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Future Developments/ Exploitation Consultancy and model application Link to other design packages or CAD programmes. Extension to other NDE Methods and structures 3D implementation and visualisation Improved defect recognition modelling e.g. Neural Networks for automated defect detection (as NNXPOSE).

36. Slide serial no 36 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Conclusions Benefits in a modular approach to POD Modelling Own applications and advantages - quick, low cost, statistics, parametric studies, false calls, design... Modelling and simulation are complimentary to experimental measurements of reliability. Simulated trials in “ Spot the Ball” POD estimates offer a good halfway house between model and experiment. Deserves a higher profile as a compliment or alternative to experimental trials.

37. Slide serial no 37 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Human Reliability Empirical rules developed. Wall and Wedgwood ECNDT7 Utilise experimental data like PISC III curves. Human error worse for small defects. Distinguish hand-to-eye coordination and interpretation of images POD Actual = POD Intrinsic * HR Human Reliability Correction factor HR = H * ENV Normalise by threshold or 50% POD Consistent with European/US model

38. Slide serial no 38 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch NDE Simulation PISC III Action 7 PCSimone ultrasonic simulator - Inspection Validation Centre (IVC) AEAT Risley

39. Slide serial no 39 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Human Reliability POD PISCIII

40. Slide serial no 40 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Human Reliability - Long term operator fatigue - ultrasonic inspection 45 o Probe PISC III

41. Slide serial no 41 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Other POD Model EXAMPLES

42. Slide serial no 42 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Example - Parametric Variations Defect Tilt

43. Slide serial no 43 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Example - POD estimates for Historical Data Radiography 200mm thick forging X-Ray Radiography 1970’s Thick Section pressure vessel forging

44. Slide serial no 44 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Example -Voids in steel plate XPOSE

45. Slide serial no 45 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Example - Modelled weld containing rogue metal ESA models used to look at probability of detecting rogue metal in steel weld by radiography - Nuclear application

46. Slide serial no 46 © 2002 AEA Technology plc 3 rd European-American Workshop on Reliability of NDE and Demining September 10-13, 2002 in Berlin Paper 24 Wall and Burch Complementary approaches Inspection Qualification Inspection Validation Performance demonstration Inspection Value Method (IVM) Inspection Simulators (e.g PISC III Action 7)