1. Martyn Clark (NCAR/RAL) Bart Nijssen (UW) Building a hydrologic model: Spatial approximations, process parameterizations, and time stepping schemes CVEN 5333 (Multiscale Hydrology) PHYSICAL HYDROLOGY & HYDROCLIMATOLOGY 2 December 2014, University of Colorado, Boulder

2. Outline Types of models ▫Data driven ▫Conceptual ▫Physically-based (or physically motivated) The necessary ingredients of a model (modeling in general) ▫State variables, process parameterizations, model parameters, model forcing data, and the numerical solution ▫Two examples:  Temperature-index snow model  Conceptual hydrologic model Physically-motivated snow modeling ▫Major model development decisions Impact of key model development decisions ▫General philosophy underlying SUMMA ▫Case studies: Reynolds Creek and Umpqua Summary and research needs

3. Assume very little knowledge of environmental physics: Infer hydrologic function as part of model calibration Bucket-style models do a great job of mimicking the hydrograph… and a poor job of representing important hydrologic processes Compensatory effects of model parameters (right answers for the wrong reasons)? Basic hydrologic modeling typology More complex – detailed depiction of a myriad of processes Many model parameters can be defined from geophysical attributes (parameter values have strong constraints) Challenging parameter estimation problem Longer run times, hard-coded parameters, high dimensional parameter space Process models Very simple (<50 lines of code) with few model parameters Do not explicitly represent important hydrologic processes (e.g., no trees) Easy to calibrate to mimic observed streamflow Poorly suited to simulate conditions different from the calibration period Bucket-style models Assume considerable knowledge of environmental physics: Prescribe hydrologic function as part of model development Perhaps too much confidence in selected model parameterizations (hard-coded parameters, single set of physics options)? Increases in model complexity create challenges for parameter estimation and uncertainty analysis  After all these years… still have an outstanding challenge: Improve performance of process-based models

4. Types of models Data-driven: Infer relationships from observations, without attempting to describe the underlying causal processes, e.g., ▫statistical model – regression between max. snow accumulation and summer streamflow ▫stochastic time series model – weather generators ▫machine learning – prediction of consumer preferences

5. https://archive.org/details/streamflowforeca914work

7. Types of models Data-driven: Infer relationships from observations, without attempting to describe the underlying causal processes, e.g., ▫statistical model – regression between max. snow accumulation and summer streamflow ▫stochastic time series model – weather generators ▫machine learning – prediction of consumer preferences Conceptual: Represent causal relationships without necessarily reflecting the underlying physical processes, e.g., ▫series of linear reservoirs to describe flow in a river ▫Budyko / Manabe bucket model to represent land surface hydrology

8. Example: SNOW-17 “SNOW-17 is a conceptual model. Most of the important physical processes that take place within a snow cover are explicitly included in the model, but only in a simplified form.” “SNOW-17 is an index model using air temperature as the sole index to determine the energy exchange across the snow-air interface. In addition to temperature, the only other input variable needed to run the model is precipitation.” Snow Accumulation and Ablation Model – SNOW-17, Eric Anderson, 2006 http://www.nws.noaa.gov/oh/hrl/nwsrfs/users_manual/part2/_pdf/22snow17.pdf

9. Types of models Data-driven: Infer relationships from observations, without attempting to describe the underlying causal processes, e.g., ▫statistical model – regression between max. snow accumulation and summer streamflow ▫stochastic time series model – weather generators ▫machine learning – prediction of consumer preferences Conceptual: Represent causal relationships without necessarily reflecting the underlying physical processes, e.g., ▫series of linear reservoirs to describe flow in a river ▫Budyko / Manabe bucket model to represent land surface hydrology Physically-based: Represent causal relationships as much as possible through a direct description of the underlying physical processes, e.g., ▫Richards equation for variably saturated flow in the vadose zone ▫Saint-Venant equations for 1D transient open channel flow Distinction between conceptual and physically-based is not always clear- cut and often a function of scale (time, space)

10. Example: The Community Land Model (CLM) Capable of simulating all dominant biophysical and hydrologic processes -Treetop-to-bedrock -Summit-to-sea

11. Most models contain elements of all three approaches model data- driven conceptualphysical

12. Spatial organization Lumped No explicit representation of space Distributed Explicit representation of space http://chrs.web.uci.edu/research/hydrologic_predictions/activities07.html SAC-SMADHSVM http://www.hydro.washington.edu/Lettenmaier/Models/DHSVM

13. Outline Types of models ▫Data driven ▫Conceptual ▫Physically-based (or physically motivated) The necessary ingredients of a model (modeling in general) ▫State variables, process parameterizations, model parameters, model forcing data, and the numerical solution ▫Two examples:  Temperature-index snow model  Conceptual hydrologic model Physically-motivated snow modeling ▫Major model development decisions Impact of key model development decisions ▫General philosophy underlying SUMMA ▫Case studies: Reynolds Creek and Umpqua Summary and research needs

14. The art of modeling: A realistic portrayal of dominant processes Need to define: 1)State variables (storage of water and energy); and 2)Fluxes that affect the evolution of state variables

15. The ingredients of a model: States, fluxes, parameters, and forcings State variables ▫Represent storage (mass, energy, momentum, etc.) ▫Evolve over time: state at time t is a function of states at previous times Fluxes ▫Represent exchange/transport ▫Rate of flow of a property per unit area Parameters ▫The (adjustable) coefficients in the flux equations Forcings ▫Time varying boundary conditions Rate of change of a state is associated with one or more fluxes different from zero

16. Forcings Initial state Parameters Computer code What is a model?

17. The necessary ingredients of a model: Model forcing data, model state variables, flux parameterizations, model parameters, and the numerical solution Example 1: A temperature-index snow model ▫The state equation ▫Flux parameterizations and model parameters ▫Numerical solution  Simple in this case, since fluxes do not depend on state variables State variable (also known as prognostic variable) Fluxes State variable: S= Snow storage (mm) Fluxes: a = Snow accumulation (mm/day) m= Snow melt (mm/day) Forcing data Model parameter Physical constant (can also be treated as a model parameter) Model forcing: p= Precipitation rate (mm/day) T a = Air temperature (K) Parameters: κ= Melt factor (mm/day/K) Physical constants: T f = Freezing point (K)

18. The necessary ingredients of a model: Model forcing data, model state variables, flux parameterizations, model parameters, and the numerical solution Example 2: A conceptual hydrology model State equation Figure from Hornberger et al. (1998) “Elements of Physical Hydrology” The Johns Hopkins University Press, 302pp.

19. The necessary ingredients of a model: Model forcing data, model state variables, flux parameterizations, model parameters, and the numerical solution Example 2: A conceptual hydrology model ▫The state equation ▫Flux parameterizations ▫Numerical solution  Care must be taken: model fluxes depend on state variables (numerical daemons) State variable Fluxes State variable: S= Soil storage (mm) Model forcing: p= Precipitation rate (mm/day) Model fluxes: e t = Evapotranspiration (mm/day) q b = Baseflow (mm/day Forcing data Model forcing: e p = Potential ET rate (mm/day) Parameters: S ps = Plant stress storage (mm) S max = Maximum storage (mm) k s = Hydraulic conductivity (mm/day) c= Baseflow exponent (-) Model parameter State variable

20. Pulling it all together: The general modeling problem Propositions: 1.Most hydrologic modelers share a common understanding of how the dominant fluxes of water and energy affect the time evolution of thermodynamic and hydrologic states ▫The collective understanding of the connectivity of state variables and fluxes allows us to formulate general governing model equations in different sub- domains ▫The governing equations are scale-invariant 2.Key modeling decisions relate to a)the spatial discretization of the model domain; b)the approaches used to parameterize individual fluxes (including model parameter values); and c)the methods used to solve the governing model equations. General schematic of the terrestrial water cycle, showing dominant fluxes of water and energy Given these propositions, it is possible to develop a unifying model framework The SUMMA approach defines a single set of governing equations, with the capability to use different spatial discretizations (e.g., multi-scale grids, HRUs; connected or disconnected), different flux parameterizations and model parameters, and different time stepping schemes

21. Outline Types of models ▫Data driven ▫Conceptual ▫Physically-based (or physically motivated) The necessary ingredients of a model (modeling in general) ▫State variables, process parameterizations, model parameters, model forcing data, and the numerical solution ▫Two examples:  Temperature-index snow model  Conceptual hydrologic model Physically-motivated snow modeling ▫Major model development decisions Impact of key model development decisions ▫General philosophy underlying SUMMA ▫Case studies: Reynolds Creek and Umpqua Summary and research needs

22. Snow modeling How should we simulate the dominant snow processes in this environment? ▫What are the dominant processes from a hydrologic perspective?  Snow accumulation: drifting; non-homogenous precipitation; rain-snow transition  Snow melt: Net energy flux for the snowpack; meltwater flow  Changes in snow properties: grain growth; snow compaction ▫What information do we need to simulate the dominant processes?  Model forcing data: Precip; temperature; wind; humidity; sw and lw radiation; (air pressure)  Model parameters: Drifting; snow albedo; turbulent heat fluxes; storage and transmission of liquid water in the snowpack

23. Starting point Governing equations that describe temporal evolution of thermodynamic and hydrologic states ▫Thermodynamics ▫Hydrology  Volumetric liquid water content  Volumetric ice content change in temperature melt/freeze fluxes at the boundaries change in liquid water melt/freeze fluxes at the boundaries evaporation sink change in ice content melt/freezecompaction fluxes at the boundaries sublimation sink Notes: 1)Fluxes are only defined in the vertical dimension, meaning that there is no lateral exchange of water and energy among elements (isolated vertical columns) 2)Spatial variability can be represented through spatial variability in model forcing (e.g., non-homogenous precipitation represented as drift factors; spatial variability in solar radiation), and spatial variability in model parameters (e.g., dust loading). 3)Most snow models follow these governing equations

24. Model decisions 1) Spatial discretization of the model domain  The size and shape of the model elements  Vertical discretization of each model element

25. Model decisions 2) Parameterization of the model fluxes (and properties)  Spatially distributed forcing data  Vertical flux parameterizations How do we represent snow albedo? How do we represent atmospheric stability? How do we represent thermal conductivity?

26. Model decisions 3) Specifying the model parameters  Spatially distributed forcing data  Vertical flux parameterizations How much snow is necessary to refresh albedo? What is the albedo decay rate? What is the minimum albedo?

27. Model decisions 4) Time stepping schemes  Operator splitting: It can be very difficult to solve equations simultaneously; most models follow a solution sequence  Iterative solution procedure: Many fluxes are a non-linear function of the model states; iterative methods typically used to estimate the state at the end of the time step (iSNOBAL exception)  Numerical error monitoring and adaptive sub-stepping: Dynamically adjust the length of the model time step to improve efficiency and reduce temporal truncation errors

28. Outline Types of models ▫Data driven ▫Conceptual ▫Physically-based (or physically motivated) The necessary ingredients of a model (modeling in general) ▫State variables, process parameterizations, model parameters, model forcing data, and the numerical solution ▫Two examples:  Temperature-index snow model  Conceptual hydrologic model Physically-motivated snow modeling ▫Major model development decisions Impact of key model development decisions ▫General philosophy underlying SUMMA ▫Case studies: Reynolds Creek and Umpqua Summary and research needs

29. Motivation Develop a Unified approach to modeling to understand model weaknesses and accelerate model development Address limitations of current modeling approaches ▫Poor understanding of differences among models  Model inter-comparison experiments flawed because too many differences among participating models to meaningfully attribute differences in model behavior to differences in model equations ▫Poor understanding of model limitations  Most models not constructed to enable a controlled and systematic approach to model development and improvement ▫Disparate (disciplinary) modeling efforts  Poor representation of biophysical processes in hydrologic models  Community cannot effectively work together, learn from each other, and accelerate model development

30. The method of multiple working hypotheses Scientists often develop “parental affection” for their theories T.C. Chamberlain Chamberlin’s method of multiple working hypotheses “…the effort is to bring up into view every rational explanation of new phenomena… the investigator then becomes parent of a family of hypotheses: and, by his parental relation to all, he is forbidden to fasten his affections unduly upon any one” Chamberlin (1890)

32. Objectives Advance capabilities in hydrologic prediction through a unified approach to hydrological modeling ▫Improve model fidelity ▫Better characterize model uncertainty

33. 33 (1) Model architecture soil aquifer (e.g., Noah)(e.g., VIC) aquifer soil (e.g., PRMS) (e.g., DHSVM) aquifer soil - spatial variability and hydrologic connectivity

34. (2) Process parameterizations

35. SUMMA: The unified approach to hydrologic modeling

36. Example Application: Simulation of snow in open clearings Different model parameterizations (top plots) do not account for local site characteristics, that is dust-on-snow in Senator Beck Model fidelity and characterization of uncertainty can be improved through parameter perturbations (bottom plots) Reynolds Creek Senator Beck

37. Example application: Interception of snow on the vegetation canopy Again, model fidelity and characterization of uncertainty can be improved through parameter perturbations Different interception formulations Simulations of canopy interception (Umpqua)

38. Example Application: Transpiration Biogeophysical representations of transpiration necessary to represent diurnal variability Interplay between model parameters and model parameterizations Rooting depth Hydrologic connectivity Soil stress function

39. Example Application: Importance of model architecture (spatial variability and hydrologic connectivity)  1-D Richards’ equation somewhat erratic  Lumped baseflow parameterization produces ephemeral behavior  Distributed (connected) baseflow provides a better representation of runoff

40. Outline Types of models ▫Data driven ▫Conceptual ▫Physically-based (or physically motivated) The necessary ingredients of a model (modeling in general) ▫State variables, process parameterizations, model parameters, model forcing data, and the numerical solution ▫Two examples:  Temperature-index snow model  Conceptual hydrologic model Physically-motivated snow modeling ▫Major model development decisions Impact of key model development decisions ▫General philosophy underlying SUMMA ▫Case studies: Reynolds Creek and Umpqua Summary and research needs

41. Summary Objectives ▫Better representation of observed processes (model fidelity) ▫More precise representation of model uncertainty Approach: Detailed evaluation of competing modeling approaches ▫Recognize that different models based on the same set of governing equations ▫Defines a “master modeling template” to reconstruct existing modeling approaches and derive new modeling methodologies ▫Provides a systematic and controlled approach to model and evaluation Outcomes ▫Provided guidance for future model development ▫Improved understanding of the impact of different types of model development decisions ▫Improved operational applicability of process-based models

42. Summary and research needs Model fidelity ▫Comprehensive review/analysis of different modelling approaches has helped identify a preferable set of modeling methods  Some obvious: biophysical representation of transpiration, two-stream canopy radiation, dust deposition on snow, etc. ▫Need to place much more emphasis on parameter estimation  A science problem rather than a curve-fitting exercise  Focus on relating geophysical attributes to model parameters  Use multiple datasets at different scales to reduce compensatory errors Model uncertainty ▫Improved understanding of suitable methods to characterize uncertainty in different parts of the model  Distinguish between decisions on process representation versus decisions on choice of constitutive functions ▫Recognize shortcomings of using multi-physics and multi-model approaches to characterize uncertainty  Competing models can provide the wrong results for the same reasons (albedo example) ▫Need to approach uncertainty quantification from a physical perspective  Inverse methods are plagued by compensatory interactions among different sources of uncertainty… difficult to infer meaningful uncertainty estimates  Progress possible through a more refined analysis of individual model development decisions

43. Exercises with a toy model Martyn Clark (NCAR/RAL) Dmitri Kavetski (University of Adelaide, Australia)

44. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. FUSE: Framework for Understanding Structural Errors

45. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Define development decisions: unsaturated zone architecture

46. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Define development decisions: saturated zone / baseflow

47. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Define development decisions: vertical drainage

48. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Define development decisions: surface runoff

49. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Build unique models: combination 1

50. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Build unique models: combination 2

51. PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi:10.1029/2007WR006735. Build unique models: combination 3 HUNDREDS OF UNIQUE HYDROLOGIC MODELS ALL WITH DIFFERENT STRUCTURE

52. Configure model analysis/inference software Copy the BATEA directory to a convenient place –The “convenient place” will henceforth and hereafter be known as the “KORE_INSTALLATION” (e.g., D:/) Set paths for your local file system –Open [KORE_INSTALLATION]BATEA\exe\bateau_fpf_director.dat and define the correct path to the template file (i.e., replace G:/ with KORE_INSTALLATION) –Open the template file defined in bateau_fpf_director.dat, and define the KORE_INSTALLATION (i.e., replace G:/ with whatever is correct) Define the forcing data –Open file Applications\MOPEX\INF_DAT\MOPEX.INF.DAT (in directory [KORE_INSTALLATION]BATEA\) and set the name of the data file (line 3) and indices defining the start of warm-up and inference period and the end of the inference period (last line). Define the model –Open file Applications\MOPEX\FUSEsettings\fuse_zDecisions-004.txt (in directory [KORE_INSTALLATION]BATEA\), and define your desired modeling decisions Start analysis/inference –Click on the appropriate executable in [KORE_INSTALLATION]BATEA\exe

53. Demonstration Quasi-Newton optimization –Optimization menu, select Quasi-Newton Manual Calibration –ManualC menu, select ChangeActiveD –Click DNYX button (top right) and move slider bars Model analysis – parameter x-sections –Analyse menu, select GridObjFunk Split-sample analysis –Save parameter file (ManualC menu, select writeParFile) and close BATEA –Modify the indices defining the start/end of the inference period: last line in file Applications\MOPEX\INF_DAT\MOPEX.INF.DAT –Start-up BATEA again and load parameter file (ManualC menu, select loadParFile) Experiment with a different model –Define desired modeling decisions in “fuse_zDecisions-004.txt” (in directory [KORE_INSTALLATION]BATEA\Applications\MOPEX\FUSEsettings\), and re- start BATEA Save model simulations –ManualC menu, select writeAllDataFile

54. Exercise Select a model and basin, calibrate the model using one time period, and evaluate the calibration using a different time period. Repeat the exercise for a different model structure and/or different basin. Document your observations.

55. QUESTIONS ???