Eawag: Swiss Federal Institute of Aquatic Science and Technology Analyzing input and structural uncertainty of deterministic models with stochastic, time-dependent parameters Peter Reichert Eawag Dübendorf, ETH Zürich and SAMSI

SAMSI meeting Feb. 1, 2007 Contents Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion  Motivation  Approach  Implementation [Tomassini, Reichert, Künsch and Borsuk, 2007, subm.]  Results for a simple epidemiological model [Cintron-Arias, Reichert, Lloyd, Banks, 2007, initiated]  Results for a simple hydrologic model [Reichert and Mieleitner, 2007, in progress]  Discussion

SAMSI meeting Feb. 1, 2007 Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Motivation Problem:  In most applications, there is a bias in results of deterministic models.  This bias is typically caused by input and model structure errors, and not by correlated measurement errors.  Bias modelling describes this bias statistically, but does not directly support the identification of its causes and the formulation of improved model structures. Objective:  Search for improved model structures or stochastic influence factors to model input or parameters that reduce/remove the bias. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Approach Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Approach The proposed approach consists of the following steps: 1)Replace selected (constant) parameters of the deterministic model by a stochastic process in time. 2)Estimate the states of this process jointly with the model parameters. 3)Analyze the degree of bias reduction achieved by doing this for different parameters. 4)Analyze the identified time dependence of the parameters for correlation with external influence factors and internal model variables. 5)Improve the deterministic model if such dependences were found. 6)Explain the remaining stochasticity (if any) of the model by including the appropriate stochastic parameter(s) into the model description. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Implementation Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model Deterministc model: Consideration of observation error: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model Model with parameter i time dependent: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Time Dependent Parameter This has the advantage that we can use the analytical solution: The time dependent parameter is modelled by a mean-reverting Ornstein Uhlenbeck process: or, after reparameterization: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Inference We combine the estimation of  constant model parameters,, with  state estimation of the time-dependent parameter(s),, and with  the estimation of (constant) parameters of the Ornstein-Uhlenbeck process(es) of the time dependent parameter(s),. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Inference Gibbs sampling for the three different types of parameters. Conditional distributions: Ornstein-Uhlenbeck process (cheap) simulation model (expensive) Ornstein-Uhlenbeck process (cheap) Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Inference Metropolis-Hastings sampling for each type of parameter: Multivariate normal jump distributions for the parameters  and . This requires one simulation to be performed per suggested new value of . The discretized Ornstein-Uhlenbeck parameter,, is split into subintervals for which OU-process realizations conditional on initial and end points are sampled. This requires the number of subintervals simulations per complete new time series of . Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Estimation of Hyperparameters by Cross - Validation Due to identifiability problems we select the two hyperparameters (  ) by cross-validation: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Estimation of Hyperparameters by Cross - Validation For a state-space model of the form we can estimate the pseudo-likelihood from the sample: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Results for Epidemiological Model Results for a Simple Epidemiological Model  Model  Model Application  Preliminary Results Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model A Simple Epidemiological Model: Dushoff et al S :susceptible I :infected N :„total population“ N-S-I :resistant model parameters 2initial conditions 1standard dev. of meas. err. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model Application Influenza Outbreaks in the USA Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Model predictions and data Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis – constant parameters Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis – beta0 time varying Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Time dependent parameter: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Next Steps  More Fourier terms, more general periodicity.  Cross validation for choice of  and .  Interpretation of remaining stochasticity.  Inclusion of identified forcing in model equations.  Inclusion of stochastic parameter in model equations.  Different data set: measles. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary for Hydrologic Model Results for a Simple Hydrologic Model  Model  Model Application  Preliminary Results Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model A Simple Hydrologic Watershed Model (1): Kuczera et al Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model A Simple Hydrologic Watershed Model (2): Kuczera et al A B 7 model parameters 3 initial conditions 1 standard dev. of meas. err. 3 „modification parameters“ C Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model Application Model application:  Data set of Abercrombie watershed, New South Wales, Australia (2770 km 2 ), kindly provided by George Kuczera (Kuczera et al. 2006).  Box-Cox transformation applied to model and data to decrease heteroscedasticity of residuals.  Step function input to account for input data in the form of daily sums of precipitation and potential evapotranspiration.  Daily averaged output to account for output data in the form of daily average discharge. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Model Application Prior distribution: Estimation of constant parameters: Independent uniform distributions for the loga- rithms of all parameters (7+3+1=11), keeping correction factors (f rain, f pet, f Q ) equal to unity. Estimation of time-dependent parameters: Ornstein-Uhlenbeck process applied to log of the parameter. Hyper-parameters:  = 5d,  fixed, only estimation of initial value and mean (0 for log f rain, f pet, f Q ). Constant parameters as above. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Posterior marginals: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Max. post. simulation with constant parameters: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residuals of max. post. sim. with const. pars.: Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis, max. post., constant parameters Residual analysis, max. post., q_gw_max time-dependent Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis, max. post., s_F time-dependent Residual analysis, max. post., f_rain time-dependent Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis, max. post., f_pet time-dependent Residual analysis, max. post., f_Q time-dependent Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis, max. post., k_et time-dependent Residual analysis, max. post., q_latmax time-dependent Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Residual analysis, max. post., k_s time-dependent NSparameterNSparameter 0.90f_rain0.79f_pet 0.88s_F0.69q_gwmax 0.87k_s0.67q_latmax 0.83f_Q0.67k_et const. parameter: 0.64 Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Time-dependent parameters Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Time-dependent parameters Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Scatter plot of parameter vs model variables Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Scatter plot of parameter vs model variables Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Scatter plot of parameter vs model variables Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Scatter plot of parameter vs model variables Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Preliminary Results Scatter plot of parameter vs model variables Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Next Steps  Search for correlations between time dependent parameters and external influence factors and system variables.  Improve formulation of deterministic model and redo the analysis.  Cross validation for choice of  and .  Inclusion of stochastic parameter in model equations to explain remaining stochasticity. Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Discussion Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

SAMSI meeting Feb. 1, 2007 Discussion 1.Other formulations of time-dependent parameters? 2.Dependence on other factors than time? 3.How to estimate hyperparameters? (Reduction in correlation time always improves the fit.) 4.How to avoid modelling physical processes with the bias term? 5.Learn from more applications. 6.Compare results with methodology by Bayarri et al. (2005). Combine/extend the two methodologies? 7.How to improve efficiency? 8.How to combine statistical with contextual knowledge? 9.? Motivation Approach Implementation Results for epidem. Model Results for hydrol. model Discussion

Eawag: Swiss Federal Institute of Aquatic Science and Technology Thank you for your attention