Bayesian calibration and comparison of process-based forest models Marcel van Oijen & Ron Smith (CEH-Edinburgh) Jonathan Rougier (Durham Univ.)

ContentsContents 1.Bayesian calibration of a forest model 2.… what measurements to take? 3.Bayesian comparison of forest models

1. Bayesian calibration of a forest model

Process-based forest models Soil C NPP Height Environmental scenarios Initial values Parameters Model

Using data We need a method that: 1.Quantifies how uncertainty about inputs and model structure causes output uncertainty 2.Efficiently uses data, on inputs & outputs, to reduce uncertainties

BASic FORest model (BASFOR) BASFOR 12 output variables 39 parameters

BASFOR: Inputs BASFOR 12 output variables

BASFOR: Prior pdf for parameters

Example: Simulating growth of Norway spruce Skogaby

BASFOR: Prior predictive uncertainty Height Biomass Prior uncertainty for Skogaby

BASFOR: Predictive uncertainty BASFOR 12 output variables High output uncertainty 39 parameters High input uncertainty Data: measurements of output variables Calibration of parameters

Bayes’ Theorem P( |D) = P( ) P(D| ) / P(D)  P( ) P(D|f( )) “Posterior distribution of parameters” “Prior distribution of parameters” “Likelihood” of data, given mismatch with model output f = the model, e.g. BASFOR

Finding the posterior: MCMC MCMC: walk through parameter-space →set of visited points approaches the posterior parameter distribution P( |D) [e.g. using Metropolis-Hastings random walk] Sample of parameter vectors from the posterior distribution P( |D) for the parameters P( |D)  P( ) P(D|f( ))

MCMC: Metropolis-Hastings random walk Metropolis (1953) algorithm 1.Start anywhere in parameter-space: p (i=0) 2.Randomly choose p(i+1) = p(i) + δ 3.IF:[ P(p(i+1)) P(D|f(p(i+1))) ] / [ P(p(i)) P(D|f(p(i))) ] > Random[0,1] THEN: accept p(i+1) ELSE: reject p(i+1) i=i+1 4.IF i < 10 4 GOTO 2 Sample of parameter vectors from the posterior distribution P( |D) for the parameters

Forest data from Skogaby (Sweden) Planted: 1966, (2300 trees ha -1 ) Weather data: Soil data: C, N, Mineralisation rate Tree data: Biomass, NPP, Height, [N], LAI Skogaby

BASFOR: Prior predictive uncertainty Height Biomass Prior pred. uncertainty Data Skogaby Data: Göran Ågren

MCMC parameter trace plots: steps Steps in MCMC Param. value

Posterior marginal distributions for parameters

Parameter correlations 39 parameters

Bayesian calibration: overview Data Bayesian calibration

Prior & posterior predictive uncertainty Height Biomass Prior pred. uncertainty Posterior uncertainty (using data Skogaby)

Partial corr. coefficients (PCC) parameters – outputs 12 output variables 39 parameters

2. What kind of measurements would have reduced uncertainty the most?

Prior predictive uncertainty & height-data Height Biomass Prior pred. uncertainty Height data Skogaby

Prior & posterior uncertainty: use of height data Height Biomass Prior pred. uncertainty Posterior uncertainty (using height data) Height data Skogaby

Prior & posterior uncertainty: use of height data Height Biomass Prior pred. uncertainty Posterior uncertainty (using height data) Height data (hypothet.)

Prior & posterior uncertainty: use of height data Height Biomass Prior pred. uncertainty Posterior uncertainty (using height data) Posterior uncertainty (using precision height data)

Summary of procedure Data D ± σModel f Prior P( ) Calibrated parameters, with covariances Uncertainty of model output Sensitivity analysis of model parameters “Error function” e.g. N(0, σ) MCMC Samples of (10 4 – 10 5 ) Samples of f( ) (10 4 – 10 5 ) P(D|f( )) Posterior P( |D) PCC

3. Bayesian comparison of forest models

Uncertainty regarding model structure Soil C NPP Height Environmental scenarios Initial values Parameters Model Imperfect understanding

Bayesian comparison of two models Bayes Theorem for model probab.: P(M|D) = P(M) P(D|M) / P(D) The “Integrated likelihood” P(D|M i ) can be approximated from the MCMC sample of outputs for model M i ( * ) Model 1 Model 2 P(M 2 |D) / P(M 1 |D) = P(D|M 2 ) / P(D|M 1 ) The “Bayes Factor” P(D|M 2 ) / P(D|M 1 ) quantifies how the data D change the odds of M 2 over M 1 P(M 1 ) = P(M 2 ) = ½ (*)(*) harmonic mean of likelihoods in MCMC-sample (Kass & Raftery, 1995)

Bayes Factor for two big forest models MCMC 5000 steps Calculation of P(D|BASFOR) Calculation of P(D|BASFOR+) Data Rajec: Emil Klimo

Bayes Factor for two big forest models MCMC 5000 steps Calculation of P(D|BASFOR) Calculation of P(D|BASFOR+) Data Rajec: Emil Klimo P(D|M 1 ) = 7.2e-016 P(D|M 2 ) = 5.8e-15 Bayes Factor = 7.8, so BASFOR+ supported by the data

Summary of procedure Data D Prior P( 1 ) Updated parameters MCMC Samples of 1 (10 4 – 10 5 ) Posterior P( 1 |D) Model 1 MCMC Prior P( 2 ) Model 2 Samples of 2 (10 4 – 10 5 ) Posterior P( 2 |D) Updated parameters P(D|M 1 )P(D|M 2 ) Bayes factor Updated model odds

ConclusionsConclusions Bayesian calibration using MCMC: Improves model predictive capacity, by updating parameters Quantifies uncertainty in parameters and output Forest model calibration: Benefits from high-precision tree height measurement Bayesian model comparison: Same probabilistic approach as Bayesian calibration Bayes Factor shows how new data change the odds of models Aid in model development

AppendicesAppendices

Bayesian calibration of big models P( |D)  P( ) P(D|f( )) Calculating P( |D) costs much time: 1.Sample parameter-space representatively 2.For each sampled set of parameter-values: a.Calculate P( ) b.Run the model to calculate likelihood P(D|f( )) Sampling problem: Markov Chain Monte Carlo (MCMC) methods Computing problem: increased processor speed Solutions

Bayes Factor for two big forest models BASFOR 39 params BASFOR + 41 params (Penman eq., corrections) MCMC steps Calibration

Bayesian methods Bayes, T. (1763) Metropolis, N. (1953) Green, E.J. / MacFarlane, D.W. / Valentine, H.T., Strawderman, W.E. (1996, 1998, 1999, 2000) Jansen, M. (1997) Jaynes, E.T. (2003) Bayes’ Theorem MCMC Forest models Crop models Probability theory