1. Issues Related to Parameter Estimation in Model Accuracy Assessment DDDAS: June 6-7, 2013 1 Tom Henderson & Narong Boonsirisumpun ICCS 2013 Barcelona, Spain 6 June 2013 Funded in part by grant AFOSR-FA9550-12-1-0291.

2. Major Objectives: 1.Develop Bayesian Computational Sensor Networks – Detect & identify structural damage – Quantify physical phenomena and sensors – Characterize uncertainty in calculated quantities of interest (real and Boolean) DDDAS: June 6-7, 20132

3. Major Objectives (cont’d): 2. Develop active feedback methodology using model-based sampling regimes – Embedded and active sensor placement – On-line sensor model validation – On-demand sensor complentarity DDDAS: June 6-7, 20133

4. Major Objectives (cont’d): 3. Develop rigorous uncertainty models; stochastic uncertainty of: – System states – Model parameters – Sensor network parameters (e.g., location) – Material damage assessments DDDAS: June 6-7, 20134

5. DDDAS Aspects Addresses 3 of 4 DDDAS components: Applications modeling Advances in mathematics and statistical algorithms, and Application measurement systems and methods DDDAS: June 6-7, 20135

6. General Framework 1.World2. Observations 3.Model Code 4. Explanations & Predictions Observe, Measure Analyze Control Inform Generate Validate Constrain Verify DDDAS: June 6-7, 2013 6 Uncertainty Quantification

7. VVUQ for Sensor Networks DDDAS: June 6-7, 2013 7

8. Model Validation Issues: Input uncertainty: parameters, initial conditions, etc. Model discrepancy: fails to capture physics, scale, etc. Cost of computation DDDAS: June 6-7, 20138 Note: This and next 2 slides based on “Assessing the Reliability of Complex Models,” NRC Report, 2012.

9. Model Validation Identify sources of uncertainty Identify information sources Assess quality of prediction Determine resources required to improve validity DDDAS: June 6-7, 20139

10. Model Adequacy Measure Predict: quantity of interest (QoI) with acceptable tolerance for intended application with uncertainty range attached e.g., V(x) = 5 +/- 2 with 90% confidence DDDAS: June 6-7, 201310

11. Our Long-term Goal Bayesian inference network analysis of: Computational uncertainty results Information from large knowledge bases: – Maintenance log data – Human knowledge DDDAS: June 6-7, 201311

12. Model Accuracy Assessment DDDAS: June 6-7, 201312 (Figure based on Oberkampf [1])

13. Model Accuracy Assessment (MAA) Compare 7 parameter estimation approaches: – Inverse method – LLS – MLE – EKF – PF – Levenberg-Marquardt – Minimum RMS Error DDDAS: June 6-7, 201313

14. Model Accuracy Assessment (MAA) Can statistics produced by estimation technique characterize adequacy of the model? – Which method gives the best k estimate? – Which is least sensitive to noise? – Which has lowest time complexity? DDDAS: June 6-7, 201314

15. PDE Model: 2D Heat Flow DDDAS: June 6-7, 201315 Truncation Error:

16. Inverse Method DDDAS: June 6-7, 201316 At each location: Yields global estimate:

17. Linear Least Squares DDDAS: June 6-7, 201317 C is the Laplacian term and d is the temporal derivative: Yields global estimate:

18. Maximum Likelihood Estimate DDDAS: June 6-7, 201318 Take derivative of log likelihood function of T:

19. Extended Kalman Filter DDDAS: June 6-7, 201319 from temperature equation at each point from where

20. Particle Filter Method DDDAS: June 6-7, 201320 - Sample p particles from range of distribution - Use weight function to re-calculate particle probabilities - Re-sample particles from new distribution Continue until change in range is small

21. Levenberg-Marquardt Method DDDAS: June 6-7, 201321 Use Jacobian: Solve for k as:

22. Minimum RMS Method DDDAS: June 6-7, 2013 22

23. Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code DDDAS: June 6-7, 201323 Thermal Data Processing

24. Model of Phenomenon Simulated Data Measured Data Algorithms & Code Regular Mesh Temperatures DDDAS: June 6-7, 201324 Thermal Data Processing Sensors

25. Model of Phenomenon Test Generation Simulated Data Known Solution Data Algorithms & Code PDE’s, Material Points, other Sequential, parallel, multi-grid, adaptive mesh refinement Verification: Ensure Code Implements Model Noise Models ? DDDAS: June 6-7, 201325

26. Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code Parameter Estimation Adjust Model Parameters Model Parameter Estimation DDDAS: June 6-7, 201326

27. Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code Validation: Make sure Model matches Phenomenon ? Adjust Model DDDAS: June 6-7, 201327

28. Example Result (LLS, simulated) DDDAS: June 6-7, 201328

29. EKF Tracking Results DDDAS: June 6-7, 201329

30. EKF Predictive Results DDDAS: June 6-7, 201330

31. UQ in LLS Prediction DDDAS: June 6-7, 201331

32. Flat Heat Plate Schematic DDDAS: June 6-7, 201332

33. 2D Thermal Data Raw Thermal Data DDDAS: June 6-7, 2013 33 FLIR T420 high performance IR camera 320x240 pixel array 170x170 over plate Subsampled (smoothed) down to 17x17 array

34. K Estimate Results DDDAS: June 6-7, 201334

35. RMS Error of Prediction DDDAS: June 6-7, 201335

36. Time Cost of Methods DDDAS: June 6-7, 201336

37. Extrapolative Prediction DDDAS: June 6-7, 201337

38. Summary of Results Given the validation criterion that predicted temperature is within 2 degrees of measured temperature, the accuracy requirements are met. Distributions are determined for the thermal diffusivity parameter. DDDAS: June 6-7, 201338

39. Our Current Work DDDAS: June 6-7, 201339 Study of Bayesian Computational Sensor Networks for Structural Health Monitoring Using Ultrasound

40. Our Current Work (cont’d) DDDAS: June 6-7, 201340

41. Current High-Level Goals DDDAS: June 6-7, 201341 1. Develop Uncertainty Quantification for data driven structural health analysis process. Dongbin Xiu has joined the University of Utah and we have started discussions on this. 2. Quantify the effect of subject matter judgments with respect to inferences about VVUQ outcomes. Develop Bayesian inference network methods.

42. Current Specific Goals 1.Determine prior joint pdf’s describing knowledge of model parameter distributions. 2.Provide proof of robustness and stability of models under the various sources of perturbation (algorithmic, data, etc.). DDDAS: June 6-7, 201342

43. Current Specific Goals 3. Quantify validation processes to assess the appropriateness of the calibrated model for predictions of quantities of interest (e.g., damage existence, damage extent, model and sensor parameter values). 4. Obtain piezoelectric active sensor network experimental results on metallic plates DDDAS: June 6-7, 201343

44. Questions? DDDAS: June 6-7, 201344