Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Strategic Pressure Measurement System Characterization of the Mars Entry Atmospheric Data System Peter A. Parker, Mark A. Hutchinson, Michael Mitchell,

Similar presentations

Presentation on theme: "1 Strategic Pressure Measurement System Characterization of the Mars Entry Atmospheric Data System Peter A. Parker, Mark A. Hutchinson, Michael Mitchell,"— Presentation transcript:

1 1 Strategic Pressure Measurement System Characterization of the Mars Entry Atmospheric Data System Peter A. Parker, Mark A. Hutchinson, Michael Mitchell, and Michelle M. Munk National Aeronautics and Space Administration Langley Research Center Hampton, VA 6th International Planetary Probe Workshop June 25, 2008

2 2 Outline Mars Entry Atmospheric Data System (MEADS) Requirements Characterization Challenges Response Surface Methodology –Mathematical Model –Experimental Design and Execution Protocol –Design Performance –Uncertainty Quantification Integration into Flight Data Algorithm Summary of Approach

3 3 MEADS Objectives and Requirements Mars Science Laboratory (MSL) entry descent and landing instrumentation (MEDLI), entry atmospheric data system –Estimate vehicle attitude and atmospheric density from pressure measurements at 7 locations on the heat shield –Improve EDL simulation for robust Mars entry systems Defendable uncertainty in flight parameters relies on adequate measurement system characterization over extreme environments Uncertainty goal of 1% of reading through the range of 0.12 – 5.0 psia Characterization/Calibration deliverable products –Mathematical model to estimate flight pressure –Uncertainty estimates throughout the flight trajectory Deliverables are measurement system knowledge, not calibration data

4 4 System Description and Characterization Space Definition Measurement System Components 7 pressure transducers (sensors) Signal support electronics (SSE) Temperatures vary between SSE and sensors locations (start and entry) Pressure varies across port locations Sensors SSE Characterization Space Pressure0 to 5 psia Temp. Sensor-90 to +30 deg.C Temp. SSE-35 to +65 deg.C Predicted Trajectory

5 5 Characterization Challenges How should we characterize (calibrate) the measurement system to ensure defendable uncertainty estimates to meet research objectives? What is our modeling strategy? –How can we test if it is adequate? –How will the model be integrated with the flight data algorithm? –How do we quantify uncertainty over the environment? How do we build a test matrix (design) to support our model? –Which design points to choose – locations? –How many design points – data volume? –What is the quality of the design – performance?

6 6 Response Surface Methodology (RSM) An extension of Statistical Design of Experiments (DOE) Developed in the 1950’s in the chemical industry –50+ year successful track-record in industry and science –RSM-based calibration has been performed at Langley since 1999 Features of the Methodology Scientifically disciplined and mathematically rigorous decision-making framework to design, execute, and analyze experiments Systems engineering perspective - emphasizes integration –Efficient, strategic, tactical, objective, defendable –Not a replacement for good science and engineering Fundamental Principles –Design Efficiency –Randomization, Replication, Blocking

7 7 Model and Design – Simple Example Maximize Sxx = Spread the points apart Optimal: equal replication at 0 and 5 Calibration Model Model Predictions Confidence Interval on Prediction Goal: Minimize the width of the confidence interval = lower uncertainty Where should we set the pressure levels for calibration? exaggerated intervals for illustration zero intercept sensitivity (slope) error (noise)

8 8 Mathematical Model for MEADS Consider a second-order Taylor series expansion in 3 factors zero intercept adjustments as a function of temperature sensitivity adjustments as a function of temperature second-order effect of pressure (non-linearity) second-order effects of temperature on intercept assumed negligible based on system knowledge final model contains terms that are statistically significant

9 9 Testing Model Adequacy Lack-of-Fit Residual (total unexplained variance) Pure-Error first-order model higher-order model residual error (linear model) What is a good (adequate) model? It is able to reproduce the experimental data within the random noise of the measurement system, not interpolate between data points Analysis of Unexplained Variance Errors are partitioned into model lack-of-fit and experimental noise Replication provides pure-error

10 10 Experimental Design and Execution Design is partitioned into 3 blocks to isolate day-to-day variability –Model development –Confirmation (test the model) Temperatures are set in random order Replication throughout design space Comprehensive assessment of the measurement system performance over predicted trajectories Temperature Design Space

11 11 Pressure Design Space Distribution of Information (# of pts) 0-5 psia calibration (8) 0-1 psia low-end calib./conf. (3) 0.12 psia (850 pa) confirmation (1) 5.5 psia 3  max confirmation (1) random conf. at mid-range (1) Nested randomization Once a temperature combination is set, pressure settings are completely randomized Design Confirmation Minimum Pressure Nested Design within 0 to 1 psia Numbers indicate replicated design points

12 12 Why a Randomized Point Ordering? We want to model the measurement system, not the calibration apparatus Randomization defends against unknown systematic variation correlated with calibration factors (pressure, temperature) Systematic Slope = 4 exaggerated systematic variation for illustration Superimposing Systematic

13 13 Design Features Factors3 factors: T(sensor), T(SSE), Pressure Factor Levels3 of T(sensor) and T(SSE), 5 of Pressure Design ConstructionAxial points in Temp, Nested in Pressure Model SupportedSecond-order model (9 terms), without T(sens) x T(SSE) Unique points25 Lack of Fit16 degrees of freedom (df), detect 4 th order in Pressure Replication 5 reps of Temperature, 4 pure-error df 3 reps of pressure within each temp., 30 pure-error df Randomization Nested restricted randomization, Pressure randomized within randomized Temperatures Blockingone block defined as an approximate 24 hour day Confirmation Points5 temperature combinations, 6 levels of Pressure

14 14 Design Performance - Prediction Variance Scaled prediction variance is a multiple of the pure-error Contours depend on design and model, not the experimental data Lower prediction variance = lower measurement uncertainty Temperature Space T_sensor (deg.C) T_SSE (deg.C) Pressure Space Pressure (psia) Note: TxP interactions space is omitted for clarity

15 15 Uncertainty Quantification How do we quantify measurement system uncertainty and mission specific performance? Consider 2 components of uncertainty: 1.Pressure Measurement Uncertainty 2.Calibration Model Stability Measurement Uncertainty simple: pressure measurement uncertainty over the calibration space –metric: distribution of 3  prediction intervals (PI) mission specific: uncertainty along the predicted trajectory –metric: 3  PI along the trajectory Stability of Calibration Model Over repeated environmental excursions (vibration, thermal-vacuum, out-gassing, microbial reduction) periodic stability tests quantify the variability in model coefficients –metric: 3  interval of each calibration coefficient

16 16 factor (X) response (Y) Test 1 Test 2 Test m ……… time Stability Monitoring Stability tests are performed periodically at room temperature to monitor the calibration model coefficients, not the raw data  slope  intercept  variance Conceptual Simple Linear Example

17 17 Analysis of Stability Zero Int. Sens Control charts graphically monitor the model stability –red lines indicate expected level of common cause variation estimated from the replicated stability tests within the initial baseline calibration –a value beyond the red line indicates a statistically significant change in a calibration coefficient, signal – Decision Point Test Number (time axis) Example of Coefficient Monitoring

18 18 Integration into Flight Data Algorithm A forward model is developed for each pressure channel An inverse model is used for flight data reduction, with uncertainty Estimate of P uncorrected for interactions that are a function of P Interactions that are a function of P Solve iteratively to converge on a point estimate of P

19 19 Pressure Uncertainty Pressure uncertainty depends on the following components Location in design space (trajectory): Calibration design matrix, expanded in model form: Covariance matrix of response observations: Variance of model coefficients: Vector of partial derivatives with respect to each estimated coefficient: Confidence Interval:

20 20 Summary of Approach Characterization planning, design, modeling, and uncertainty strategically support defendable uncertainty estimates of flight parameters; satisfying the science objectives Design performance is quantitatively assessed before execution Execution incorporates strategic and tactical techniques –estimates experimental error (system noise), pure-error –defends against systematic variation in the apparatus –efficiently collect information over characterization space Modeling and uncertainty analysis –builds and tests adequate mathematical models –provides uncertainty estimates over the trajectory Provides a general framework applicable to measurement systems

21 21 Contacts and References Contacts for more information: Peter Parker Mark Hutchinson Michelle Munk Some Textbooks: Box, G.E.P., Hunter, W.G., Hunter, J.S. (1978), Statistics for Experimenters, John Wiley & Sons  Box, G.E.P. and Draper, N.R. (1987), Empirical Model-Building and Response Surfaces, John Wiley & Sons  Montgomery, D.C. (2004), Design and Analysis of Experiments (6th ed.), John Wiley & Sons.  Myers, R.H. and Montgomery, D.C. (2002), Response Surface Methodology, (2nd Ed.) John Wiley & Sons.  Wu, C.F. and Hamada, M. (2000), Experiments: Planning, Analysis, and Parameter Design Optimization, John Wiley & Sons.  NIST Engineering Statistics Handbook, Measurement Process Characterization, Some Articles, annotated  Kowalski, S. M., Parker, P. A., and Vining, G. G. (2007), “Tutorial on Split-Plot Experiments,” Quality Engineering, 19, pp (details of restricted randomization analysis) Parker, P. A. and Finley, T. D. (2007), “Advancements in Aircraft Model Force and Attitude Instrumentation by Integrating Statistical Methods,” AIAA Journal of Aircraft, 44, pp (general overview)  Parker, P. A. and DeLoach, R. (2001), “Response Surface Methods for Force Balance Calibration Modeling,” IEEE 19th International Congress on Instrumentation in Aerospace Simulation Facilities, Cleveland, Ohio. (detailed discussion of applying RSM to calibration, with examples)  Parker, P. A., Anderson-Cook, C. M., Robinson, T. J., and Liang, L. (2008), “Robust Split-Plot Designs,” Quality and Reliability Engineering International, 24, pp (general philosophy and design considerations for restricted randomization)  Tripp, J. S. and Tcheng, P. (1999), “Uncertainty Analysis of Instrument Calibration and Application,” NASA/TP (general framework of instrument uncertainty estimation)  Woodall, W.H., Spitzner, D.J., Montgomery, D.C., and Gupta, S., “Using Control Charts to Monitor Process and Product Quality Profiles,” Journal of Quality Technology, 36, pp , (general introduction to profile monitoring)

Download ppt "1 Strategic Pressure Measurement System Characterization of the Mars Entry Atmospheric Data System Peter A. Parker, Mark A. Hutchinson, Michael Mitchell,"

Similar presentations

Ads by Google