1. Eawag: Swiss Federal Institute of Aquatic Science and Technology Problems of Inference and Uncertainty Estimation in Hydrologic Modelling Peter Reichert Eawag Dübendorf and ETH Zürich

2. SAMSI meeting Oct. 16, 2006 Contents Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities  Motivation  Errors and Uncertainties in Hydrologic Watershed Modelling  Suggested Problem Solutions  Working Group Opportunities

3. SAMSI meeting Oct. 16, 2006 Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

4. SAMSI meeting Oct. 16, 2006 Motivation Practice of Environmental Modelling: Mechanistic, deterministic description of system behaviour with a simple, additive, independent (measurement) error model – Strong autocorrelation of residuals, if temporal resolution of data is high. This severe violation of statistical assumptions leads to unreliable error estimates. The problem is aggravating, as temporal resolution of data and measurement accuracy are increasing. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

5. SAMSI meeting Oct. 16, 2006 Motivation Examples (1): Aquatic ecosystem modelling Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Phytoplanktion biomass Walensee Zürichsee Greifensee Mieleitner et al. 2006

6. SAMSI meeting Oct. 16, 2006 Motivation Examples (2): Climate modelling Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Tomassini et al. 2006

7. SAMSI meeting Oct. 16, 2006 Motivation Examples (3): Hydrologic modelling Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Yang et al. 2006

8. SAMSI meeting Oct. 16, 2006 Motivation Cause of the Problem and Challenges: The cause of this problem is not the inadequate model of the measurement process, but the neglection of input and model structure errors that are propagated through the model and dominate prediction uncertainty. Both input and model structure errors lead to very similar pattern in the residuals. The challenges are Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities to find good statistical descriptions of the random contributions of both error sources, to find procedures to support finding model structure improvements, and to separate the two error contributions.

9. SAMSI meeting Oct. 16, 2006 Motivation Universality of the Problem: This problem is typical for nearly all fields of dynamic modelling in the environmental sciences. The causes and techniques for problem analysis can be expected to be the same for different application areas, despite application-field specific interpretations and identified error models. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

10. SAMSI meeting Oct. 16, 2006 Motivation Hydrologic Modelling: Watershed hydrologic modelling is a particularly good study area for these problems as Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Data at high temporal resolution are available. Essentially the same problems occur with complex and very simple watershed models. (see next part of the talk for a justification of this statement.)  It seems to be a reasonable strategy to analyse the problem and test solutions with simple watershed models and transfer the promising solutions to the more complex case.

11. SAMSI meeting Oct. 16, 2006 Errors in Hydrological Modelling Errors and Uncertainties in Hydrologic Watershed Modelling Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

12. SAMSI meeting Oct. 16, 2006 Errors and Uncertainties in Hydrologic Watershed Modelling Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities  Overview of Hydrologic Processes  A Simple Hydrologic Watershed Model  More Complex Watershed Models  Sources of Error in Watershed Modelling

13. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling Overview of Hydrologic Processes The water balance in a watershed is affected by: rainfall, runoff, infiltration into the soil, evapotranspiration, transport through the soil (vertically and laterally), transport to shallow ground water, lateral transport in ground water, transport to deep ground water, exfiltration from soil and groundwater to surface water, transport in surface water. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

14. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling A Simple Hydrologic Watershed Model (1): Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Kuczera et al. 2006

15. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling A Simple Hydrologic Watershed Model (2): Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Kuczera et al. 2006

16. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling A Simple Hydrologic Watershed Model (3): Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Kuczera et al. 2006

17. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling More Complex Watershed Models Parameterization by soil properties (soil thickness, porosity, texture, conductivity, etc.). Higher vertical resolution of soil profile (layers, continuous vertical resolution). Higher horizontal resolution of watershed (accounting for variation in soil properties, land use, etc. within the watershed). Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities More complex models (with a higher spatial resolution) are primarily required for the prediction of the effect of land use change, not to improve the quality of the fit. These models are usually highly overparameterized, but do nevertheless not very much improve the fit.

18. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling Sources of Error in Watershed Modelling Input uncertainty Point measurements from rain gauges and potential evapo- transpiration measurements are extrapolated to the watershed area despite high local variation in rain intensity. Model structure uncertainty Many different „storage systems“ in parallel are represented by an „average storage“ or by storage systems parameterized using soil properties. All storage systems within a sub-basin are subject to the same input. Parameterization of „storage“ function. Output uncertainty Measurement error of stream flow (gauging curve and random error). Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

19. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling Difference – Simple vs. Complex Models As simple and complex models usually use the same input, they face the same problems outlined above. Only the use of higher (spatial) resolution in input could reduce some of these problems, not increase in model complexity (which was introduced for other reasons). It is a trend in real-time hydrologic modelling to do this with the aid of radar data. But still most of the hydrologic modelling studies must be based on rain gauge data. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

20. SAMSI meeting Oct. 16, 2006 Errors in Hydrologic Modelling Results for Simple Error Model When using an independent error model the result will usually be a small prediction uncertainty for the mean and a large standard deviation of the error term. The resiudals will show strong deviations from the indepencence assumption. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

21. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

22. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 1.„Ad-hoc“ Approaches Approaches based on increasing parameter uncertainty. (GLUE, SUFI, SUNGLASSES, etc.) 2.Improvement of Output Error Model Autoregressive output error models. 3.Input and Model Structure Error Models Storm multipliers. Bayesian model averaging. Use of a stochastic hydrological model. Stochastic, time-dependent parameters. Multi-criteria optimization. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

23. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 1. „Ad-hoc“ Approaches Approaches such as GLUE, SUFI, SUNGLASSES, etc. increase parameter uncertainty to cover most of the observations with a prediction uncertainty band. This is either done by introducing a „generalized“ likelihood function, the values of which are normalized and then interpreted as probabilities or by „ad-hoc“ selection of parameter subsets that lead to an adequate coverage of observations. Despite the poor statistical foundation, such techniques are quite popular in hydrology.  This is not the approach I would like to follow in the working group. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

24. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 2. Improvement of Output Error Model Use of an autoregressive error model instead of the independent error model. This approach is quite successful in the fulfilment of statistical assumptions (see example). However, it describes only the effect and not the cause of the errors and may lead to „statistical description“ of „physical phenomena“ (description of recession curves from „storages“ by the auto- regressive error model).  This is a nice intermediate step, but the effort must be on a description of the actual error sources. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

25. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 2. Improvement of Output Error Model (Example) Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Yang et al. 2006 residuals, no transformation residuals, Box- Cox transf. residuals, Box- Cox tr., var. sd. innovations, Box- Cox tr., var. sd. var. corr. time

26. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 2. Improvement of Output Error Model (Example) Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities Yang et al. 2006 residuals, no transformation residuals, Box- Cox transf. residuals, Box- Cox tr., var. sd. innovations, Box-Cox tr., var. sd., var. corr. time Auto- correlation

27. SAMSI meeting Oct. 16, 2006 Suggested Problem Solutions 3. Input and Model Structure Error Model Only recently better error models have been suggested. The essential elements are that  the high input uncertainty in total rainfall and potential evapotranspiration over the watershed must be considered explicitly,  a deterministic description is not adequate due to stochastic distribution of input over the watershed („the different storage systems“),  model structure (systematic) errors must be distinguished from random errors.  It would be an interesting SAMSI activity to discuss how to best do this and compare results of different approaches. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

28. SAMSI meeting Oct. 16, 2006 Working Group Opportunities Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

29. SAMSI meeting Oct. 16, 2006 Working Group Opportunities Research Questions / Options for Projects (1) 1.Compare results when making different model parameters stochastic and time-dependent. (Ongoing with a postdoc in Switzerland extending earlier work with continuous-time stochastioc parameters.) 2.Develop a better statistical description of rainfall uncertainty. (Option for a collaboration with climate/weather working groups.) 3.Explore alternative options for making parameters time-dependent. (Suggestions so far: storm-dependent parameters, time- dependent parameter as an Ornstein-Uhlenbeck process.) Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

30. SAMSI meeting Oct. 16, 2006 Working Group Opportunities Research Questions / Options for Projects (2) 4.Investigate on how to learn from state estimation of stochastic hydrological models. (Can the pattern of state adaptations lead to insights of model structure deficits or input errors?) 5.Develop uncertainty estimates when using multi- objective optimization. (How to use information on Pareto set for uncertainty estimation of parameters and results?) 6.Analyse differences in results of suggested approaches when using different models. (Is there a generic behaviour of different techniques when they are applied to different models/data sets?) Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

31. SAMSI meeting Oct. 16, 2006 Working Group Opportunities Research Questions / Options for Projects (3) 7.Improve the efficientcy of posterior maximisation and posterior sampling. (Efficiency becomes important when having complex watershed models in mind. Efficient global optimizers and sampling from multi-modal posterior distributions becomes then important.) 8.More questions will come up during discussions. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

32. SAMSI meeting Oct. 16, 2006 Working Group Opportunities Practical Considerations State estimation of time-dependent OU-parameters as well as the simple hydrological model are implemented in the UNCSIM package by PR. This package also provides a simple interface to complex hydrological models. Jasper Vrugt (LANL) can provide implementations of several simple hydrological models and analysis techniques in Matlab. The simple hydrological models can also easily be implemented in any other computing environment. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

33. SAMSI meeting Oct. 16, 2006 Working Group Opportunities How to Proceed? 1.Initiate a reading group for discussing key papers and suggestions of how to attack the problems. This could be a separate working group or a subgroup of the methodology working group. 2.Discuss and prioritize (according to expected chance of success) the „collection of suggestions“ developed under point 1 above. 3.Use preliminary results of project 1 to stimulte the discussions. 4.Decide on research plans for projects to work on. 5.Organise a workshop for discussing research plans and preliminary results with experts in the field. 6.Plan the group activities for the remaining part of the subprogram that lead to results to be published and presented at a closing workshop. Motivation Errors in Hydro- logic Modelling Suggested Pro- blem Solutions Working Group Opportunities

34. Eawag: Swiss Federal Institute of Aquatic Science and Technology Thank you for your attention