1. Challenges in the Extraction of Decision Relevant Information from Multi-Decadal Ensembles of Global Circulation Models Dave Stainforth Acknowledgements: A. Lopez. F. Niehoerster, E. Tredger, N. Ranger, L. A. Smith Climate Change Workshop Statistical and Applied Mathematical Science Institute 18 th February 2010 1.Introduction and context. 2.The difficulties in predicting climate. 3.Domains of possibility. 4.Metrics. 5.Implications for future experiments. 6.[Transfer functions] Grantham Research Institute & Centre for the Analysis of Timeseries, London School of Economics and Political Science

2. Introduction Climate models can help us: –understand the physical system. –generate plausible storylines for the future. –build better models. Context: –responding to societal desire for predictions of the impacts of climate change –providing information to guide climate change adaptation strategies. minimise vulnerability/maximise resilience.vs. predict and optimise International adaptation – when is adaptation adaptation and when is it development? More uncertainty, please.

6. Climate Prediction – A Difficult Problem A problem of extrapolation: –Verification / confirmation is not possible. Model deficiencies: –Model inadequacy: they dont contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) –Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. Model interpretation: –Lack of model independence. Metrics of model quality –Observations are in-sample. –Ensembles are analysed in-sample. –Models which are bad in some respects may contain critical feedbacks in others. –Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

7. Types of Climate Uncertainty External Influence (Forcing) Uncertainty What will future greenhouse gas emissions be? Initial Condition Uncertainty (partly aleatory uncertainty) The impact of chaotic behaviour. Model Imperfections (epistemic uncertainty) Different models give very different future projections. Figure: IPCC – AR4

8. Uncertainty Exploration Type of Uncertainty:Response: Forcing UncertaintyEnsembles of Emission scenarios Initial Condition UncertaintyInitial Condition Ensembles (ICEs). (V. small. Typically max of 4; sometimes 9) Model Deficiencies.Multi-model ensembles e.g. CMIP III – O(10) Perturbed-parameter ensembles: - O(10000-100000) – climateprediction.net - O(100) – in-house teams e.g. MOHC

9. Climate Prediction – A Difficult Problem A problem of extrapolation: –Verification / confirmation is not possible. Model deficiencies: –Model inadequacy: they dont contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) –Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. Model interpretation: –Lack of model independence. Metrics of model quality –Observations are in-sample. –Ensembles are analysed in-sample. –Models which are bad in some respects may contain critical feedbacks in others. –Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

10. Consequences of Lack of Independence 1 See Stainforth et al. 2007, Phil Trans R.Soc A Climateprediction.net data

11. Consequences of Lack of Independence 2 From Stainforth et al. 2005

12. Can Emulators Help Out Here? No Even the shape of model parameter space is arbitrary so filling it in does not help in producing probabilities of real world behaviour.

13. An Aside: UK Climate Projections 2009 (UKCP09) - 1 Change in mean summer precip: 10%90% Murphy et al, 2004 UKCIP, 2009

14. An Aside: UK Climate Projections 2009: Change in Wettest Day in Summer Medium (A1B) scenario 2080s: 90% probability level: very unlikely to be greater than 2080s : 67% probability level: unlikely to be greater than

15. An Aside: A (Very) Basic Summary of My Understanding of the Process sample parameters, run ensemble, emulate to fill in parameter space, weight by fit to observations Emulate

16. An Aside: Issues Size of ensemble given size of parameter space. The ability of the emulator to capture non-linear effects. The choice of prior i.e. how to sample parameter space. The justification for weighting models. On what scales do we believe the models have information?

17. Choices of Model Parameters Most model parameters are not directly representative of real world variables. e.g. the ice fall rate in clouds, the entrainment coefficient in convection schemes. Their definition is usually an ad hoc choice of some programmer. (Possibly a long time ago, in a modelling centre far away.) Thus a uniform prior in parameter space has no foundation and testing the importance of such a prior is not a matter of tweaks around the edges (adding 15% to the limits, or exploring a triangular prior around central values); rather it is a matter of sensitivity to putting the majority of the prior points in one region:

18. An Aside: Issues Size of ensemble given size of parameter space. The ability of the emulator to capture non-linear effects. The choice of prior i.e. how to sample parameter space. The justification for weighting models. On what scales do we believe the models have information? Choice of parameter definition

19. Estimated distributions for climate sensitivity: upper bounds depend on prior distribution Frame et al, 2005 Uniform prior in sensitivity Uniform prior in feedbacks

20. Climate Prediction – A Difficult Problem A problem of extrapolation: –Verification / confirmation is not possible. Model deficiencies: –Model inadequacy: they dont contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) –Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. Model interpretation: –Lack of model independence. Metrics of model quality –Observations are in-sample. –Ensembles are analysed in-sample. –Models which are bad in some respects may contain critical feedbacks in others. –Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

21. Domains of Possibility 1 From Stainforth et al. 2005

22. Domains of Possibility 2 See Stainforth et al. 2007, Phil Trans R.Soc A Climateprediction.net data

23. Climate Prediction – A Difficult Problem A problem of extrapolation: –Verification / confirmation is not possible. Model deficiencies: –Model inadequacy: they dont contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) –Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. Model interpretation: –Lack of model independence. Metrics of model quality –Observations are in-sample. –Ensembles are analysed in-sample. –Models which are bad in some respects may contain critical feedbacks in others. –Non-linear interactions: selecting on a subset of variables denies the highly non- linear nature of climatic interactions.

24. Best Information Today / Best Ensemble Design For Tomorrow For tomorrow: Design ensembles to push out the bounds of possibility. For today: Use the best exploration of model uncertainty combined with the best global constraints.

25. Issues/Questions in Ensemble Design to Explore Uncertainty Emulators to guide where to focus parameter space exploration. (Potentially very powerful in distributed computing experiments.) How? Simulation management to minimise the consequence of in-sample analysis. How? Questions of how we describe model space to enable its exploration. How do we evaluate the spatial and temporal scales on which a model is informative? How do we integrate process understanding with model output in such a multi-disciplinary field. How do we integrate scientific information with other decision drivers. Better understanding and description of the behaviour non-linear systems with time dependent parameters. How do we evaluate information content?

26. Resolution.vs. complexity.vs. uncertainty exploration What processes do we need to include in our models? What do we need our models to do to answer adaptation questions? What would be the perfect ensemble? What should be the next generation ensemble?

27. Lets Be Careful Out There