Presentation on theme: "Model calibration using. Pag. 5/3/20152 PEST program."— Presentation transcript:
Model calibration using
Pag. 5/3/20152 PEST program
Pag. Major steps of hydrologic modeling (Hydrologic) data Relate to model inputs Parameter estimation or model calibration Model -- parameters Predictions Prediction uncertainty Societal decisions
Pag. What is Model Calibration? 5/3/20154 PEST program Adjustment of the parameters of a mathematical or numerical model in order to optimize the agreement between observed data and the model's predictions. American meteorological society The process of adjusting model inputs so that model calculations match what we measure in the real-world. So, a good model calibration is not sufficient to develop a good model. Also need good data, a good model, and an adequate optimization method.
Pag. Real world data 5/3/20155 PEST program
Pag. Attempt 1 for model calibration 5/3/20156 PEST program
Pag. Attempt 2 for model calibration 5/3/20157 PEST program
Pag. Attempt 3 for model calibration 5/3/20158 PEST program
Pag. Calculation of residuals for model calibration 5/3/20159 PEST program
Pag. Statistical measures involving residuals 5/3/ PEST program
Pag. 5/3/ PEST program PEST is a nonlinear parameter estimation package capable of estimating parameters for any computer model. It solves a nonlinear least squares problem and minimizes the differences between the model's outputs and field measurements such as the calculated and measured discharges. PEST adapts to the model, the model does not need to adapt to PEST.
Pag. 5/3/ PEST program Minmizing Sum of Squared Residuals (SSR) The use of this equation, as an objective function to be minimized, implies certain assumptions about the residuals (Clarke, 1973): a)That the time series of residuals have zero mean and constant variance b)That the time series of residuals does not have a significant autocorrelation If confidence intervals are to be given for the estimated model parameters, then: c) residuals have to be distributed normally.
Pag. The role of PEST 5/3/ PEST program 1.(initial) Parameter Sensitivity Analysis : conducted on a larger parameter space, which allows excluding from the further analysis those parameters that have a relatively small impact on the model response.(initial) Parameter Sensitivity Analysis 2. Calibration: (parameter estimation): PEST “calibrates” a model by reducing the discrepancies between model outputs and field observations to a minimum in the weighted least squares sense.Calibration 3. Predictive analysis: Once a parameter set has been determined for which model behavior matches system behavior as well as possible, it is then reasonable to ask whether another parameter set exists which also results in reasonable simulation by the model of the system under study.Predictive analysis
Pag. PEST template, instruction, and control files 5/3/ PEST program 1.Template file: tells PEST how to write parameters in the model input fileTemplate file 2. Instruction file: tells PEST where it has to look for the model-generated output, and how to read it e.g. which column contains the simulated discharges.Instruction file 3. PEST conrol file: contains:User-supplied initial parameter values, range of permissible values that a parameter can take, observations, maximum number of iterations, etc.PEST conrol file
Pag. 5/3/ PEST program Optimization algorithm can be classified into two categories: 1.Global search e.g. Shuffled Complex Evolution-University of Arizona (SCE-UA, 1992). searches the entire parameter space and do a controlled random search in the direction of global optimum. 2. Local search e.g. PEST program (Doherty, 1994). Estimate parameters with observations using a gradient-based method.gradient-based method Gradient methods are much faster but can get stuck in local minima.
Pag. Local and global minima 5/3/ PEST program Response surface for an inverted goal function. Local minima are represented as peaks. It shows that for any given goal function, there exits many parameter sets for which the goal functions are not significantly different from each other, i.e., there are many potential solutions based on quite different parameter sets.
Pag. 5/3/ PEST program Based on Objective Function, the automated calibration procedures can be classified as: 1.Single objective procedures e.g. mean squared errors in PEST program. PEST trys to minimize the objective function in order to obtain a better fit between predicted and observed values. PEST offers the best parameter set. 2. Multiple objective procedures. The generated output from this approach is a set of solutions (called pareto solutions) instead of a unique solution.
Pag. When is a Model Calibrated? 5/3/ PEST program As the problem of parameter optimization is not unique, it is important that we define when a model is calibrated and what the magnitude of the prediction uncertainty is.
Pag. 5/3/ PEST program To see when a model is calibrated: 1.Calculate the 95% prediction uncertainty
Pag. 5/3/ PEST program 2. Normally, more than 80 percent of obsevations should be braketet by 95% confidence intervals (the upper and lower limits of the best parameter sets) a) is not a parameter calibration problem, b)calibration can obtain smaller uncertainty distribution, c) it can be expected that some measured data can fall outside the 95PPU.
Pag. 5/3/ PEST program 3. Upon reaching the above criteria, if there exits a significant r 2 and/or Nash-Sutcliff coefficient between the best simulation and the measured data for a calibration and a test (validation) data set, then the model can be considered calibrated.
Pag. Calibration issues 5/3/ PEST program After analysing residuals to check whether they are independent, homoscedastic and normally distributed with zero expectation; We need to consider following questions: 1.What is the data quality and error (data uncertainty)? How does it affect predictions? 2.What is the sensitivity of model matches to changes in model inputs? (non- uniqueness in solutions) 3.How good is good enough, for a well calibrated model?
Pag. 5/3/ PEST program Thank you
Pag. 5/3/ PEST program
Pag. 5/3/ PEST program WetSpa model parameter sensitivity for the Illinois River basin
Pag. 5/3/ PEST program Streamflow simulation using WetSpa at Illinois River basin, Oklahoma, USA
Pag. 5/3/ PEST program
Pag. PEST input template for WetSpa 5/3/ PEST program
Pag. PEST Instruction file for WetSpa 5/3/ PEST program
Pag. PEST conrol file for WetSpa 5/3/ PEST program