© Crown copyright Met Office An Introduction to Long-range Forecasting Emily Wallace Nov 2012

© Crown copyright Met Office Content How is long-range forecasting possible? predictability vs chaos, drivers of predictability, what is predictable? Dynamical seasonal prediction systems Initialisation, coupled modelling, assessing uncertainty Hindcasts Bias correction, model climatology, skill assessment Products Standard products, bespoke products

© Crown copyright Met Office How is it possible?

© Crown copyright Met Office Predictability and chaos ‘chaos’ drywet The distribution is analogous to the climatology of a meteorological variable (here, rainfall). The ball drops can be seen as values corresponding to individual years. The precise bin in which a ball falls cannot be predicted (‘chaos’). If many drops are made, the ‘distribution’ of balls in the bins can be described.

Predictability and chaos wetdry ‘chaos’ large-scale influences © Crown copyright Met Office Individual ball drops are analogous to individual forecasts, all with similar starting points. The prediction consists in quantifying the difference between the two distributions (climatology and forecast). Example of large-scale influence: ocean temperatures The precise bin in which a ball falls still cannot be predicted (‘chaos’). The tilt of the table changes the shape of the distribution (‘predictability’). If many drops are made, the new distribution of balls in the bins can be described.

© Crown copyright 2011 time predictability ICs SSTs, surface, etc external forcings daysmonthsyears Sources of predictability: Initial conditions (pressure, temperature, etc) Boundary conditions (SST, soil moisture, etc); External forcing (emissions, solar etc) Seasonal: probabilistic forecast

© Crown copyright 2011 Example of forcing: sea surface temperature anomalies The forcing pattern is large scale and slow-varying in time. The impact is also large scale.

© Crown copyright 2011 Teleconnections: typical La Niña impacts

© Crown copyright 2011 Teleconnections: typical El Niño impacts

‘climate’ (seasonal averages), not ‘weather’ (conditions on specific days) large-area averages, not localised events range of outcomes, with probabilities attached to them (risk ) What is predictable at long range? How are long-range predictions done? statistical methods – using empirical relationships derived from historical records dynamical methods – using dynamical (climate) models

Statistical models – using empirical relationships derived from historical records Statistical models are… Cheap – equations are far less complex than dynamical models But… Require a long, good quality, observational dataset to train the model on Will produce poor predictions if the assumptions change Easy to over-tune

© Crown copyright Met Office Seasonal forecasting with dynamical models

A seasonal forecasting system requires: definition of starting point (initial conditions; data assimilation) model of the climate system description of uncertainties (ensembles) Dynamical methods: seasonal forecasting systems

© Crown copyright Met Office For seasonal forecasting, assimilation of ocean state is important Tropical Atmosphere Ocean array (TAO) ARGO floats SST Subsurface ocean

Climate simulation Vs. seasonal prediction © Crown copyright Met Office Climate model Observed state of ocean/atmos initialisation time Synchronised to real world Forecast from initialisation time Climate model Climate simulation Arbitrary or climatology start

A seasonal forecasting system requires: definition of starting point (initial conditions; data assimilation) model of the climate system description of uncertainties (ensembles) Dynamical methods: seasonal forecasting systems

Coupled and uncoupled seasonal forecast systems “Data ->system configuration” Coupled (1-tier) systems Model: Includes interactive 3D ocean model Initial conditions: atmosphere + 3D ocean GPCs: Exeter, ECMWF, Washington, Toulouse, Melbourne, Montreal, Tokyo, Beijing Forecast range: typically 6 months + Uncoupled (2-tier) systems Model: atmosphere only + prescribed SST. Atmosphere ‘forced’ with predicted (or persisted) SST anomalies. No 2-way atmosphere/ocean interaction Initial conditions: atmosphere (usually) + SST GPCs: Moscow, Seoul, CPTEC, Pretoria Forecast range: typically 3-5 months © Crown copyright Met Office

A seasonal forecasting system requires: definition of starting point (initial conditions; data assimilation) model of the climate system description of uncertainties (ensembles) Dynamical methods: seasonal forecasting systems

Uncertainty type 1: initial condition uncertainty © Crown copyright Met Office Climate model Data assimilation: ocean/atmos Run model forecasts from many slightly different initial conditions Forecasts may evolve differently Collectively, the ensemble estimates the range of uncertainty stemming from sensitivity to initial conditions forecast Ensemble prediction

Uncertainty type 2: uncertainty in model formulation When climate models are developed choices must be made on schemes to represent physical processes e.g. Convection scheme, radiation scheme... Forecasts from the same basic model and same initial state may give different forecasts when different physics schemes are used. Choice of physics scheme is often centre dependent Model formulation uncertainties are addressed by: Stochastically perturbing model variables (and/or tuneable physics parameters) as the model runs Combining ensembles from different modelling centres. Typically each centre will have made different choices in model formulation. Thus multi-model: e.g. LC-LRFMME, EUROSIP, APCC © Crown copyright Met Office

Addressing model formulation uncertainties © Crown copyright Met Office Uncertainty in initial atmospheric state Uncertainty in future atmospheric state Ensemble forecast from model 1 explores part of the future uncertainty (from initial condition uncertainty) Ensemble forecast from model 2 (i.e. perturbed physics), run from same set of initial states, typically explores additional future uncertainties (from model formulation uncertainty) Including representation of model formulation uncertainties gives better sampling of the true uncertainty.

Example ensemble predictions Met Office GloSea4 system Initial condition uncertainty (lagged analysis) 21 different initial ocean/atmos states used (daily lag) Model formulation uncertainty Stochastic (kinetic energy) perturbations to model wind field as the model runs © Crown copyright Met Office 22 nd Feb‘11 23 rd Mar’11 21 st Feb‘11 Forecast 2 perturbed runs from daily start dates ->14 runs to (7 months) each week, after 3 weeks we have a 42 member ensemble

GloSea4 ensemble prediction of Nino3.4 SST anomaly from March 2010 © Crown copyright Met Office

Ensemble ‘postage’ stamps © Crown copyright Met Office Ensemble mean: reinforces commonalities, masks uncertainties

© Crown copyright Met Office Qualities of the ensemble mean Considered the ‘most likely’ single (deterministic) prediction. Usually lies near centre of the ensemble distribution Picks out the dominant signal: Commonalities across the members ‘reinforce’ Differences across members tend to cancel Important ‘but’….. Quantitative information on uncertainty is removed by the averaging process

Seasonal prediction systems © Crown copyright Met Office Climate model Initialisation: Current state of ocean/atmos forecast Ensemble generation

© Crown copyright Met Office Hindcasts: Correcting model climatology

Hindcasts for bias correction Seasonal forecasts have biases. To adjust for biases we generate a set of retrospective forecasts (hindcasts) that describe the ‘climatology’ of the model Model climatologies are defined over all retrospective years and all members For GloSea4: 14 hindcast years, 12 members = 168 realisations of each season. Note: most systems have more ensemble members in the real-time forecast than in the hindcast set. © Crown copyright Met Office

Hindcast: a collection of forecasts of the past ( ) Bias corrected forecast obs Forecast members obs Hindcast mean Forecast members All Hindcast runs: ~ 12 members 14 years (96-09) Hcst mean Raw forecast

© Crown copyright 2011 PROBLEM: What does “climate” mean under climate change? Calculating anomalies: the importance of the reference period Does it matter?

© Crown copyright 2011 Ensemble-mean forecast for the average temperature anomaly over MAM Reference period Wallace and Arribas, 2012, MWR

© Crown copyright Met Office Hindcasts: Generating bias corrected probabilistic forecasts

Probabilistic forecasts and bias correction e.g. precipitation forecast © Crown copyright Met Office wet dry Observed climatology, Lower tercile (obs) Upper tercile (obs) Model climatology, e.g. wet bias Upper tercile (model) Lower tercile (model) Ensemble member Member is counted as a prediction of the average (obs) tercile category

Generating probability forecasts from the ensemble An estimate of the forecast probability of an event is the proportion of the ensemble members that predict the event © Crown copyright Met Office CategoryNo. Members that predict category Fraction of total ensemble members Forecast probability above55/955% average33/933% below11/911%

Seasonal prediction systems © Crown copyright Met Office Climate model Initialisation: Current state of ocean/atmos forecast Ensemble generation Retrospective forecasts (hindcasts) Skill assessment (verification) Forecast bias correction

© Crown copyright Met Office Hindcasts: Assessing skill

Skill of seasonal forecasting systems Skill is assessed on the hindcast (covering a number of past years) Can (and should) be done in several ways: Statistical assessment of skill Process based assessment

What is in Malaysia for me? © Crown copyright 2011 What’s in Malaysia for me?

© Crown copyright 2007 Statistical skill of forecast products, estimated from hindcasts: nal-to-decadal/gpc-outlooks/glob-seas-prob-skill ROC curves Reliability diagram Looking like an idiot Looking well prepared

© Crown copyright 2011 Forecasts are generated monthly using data from GloSea4 and ECMWF Skill (linear correlation) of 6-month forecasts from March to September is detailed below Skill (linear correlation) MarAprMayJunJulAugSep TS ACE Perfect forecasts would have a skill of 1.0 Deterministic skill assessment Skill of tropical storm seasonal forecast 1987–2009

Skill of seasonal forecasting systems Skill is assessed on the hindcast (covering a number of past years) Can (and should) be done in several ways: Statistical assessment of skill Process based assessment

© Crown copyright 2011 Time  Latitude  Colours: 5-day average rainfall in mm/day, 10°W-10°E Red line: Timing of monsoon onset, early July Time  Latitude  Mean observed rainfall (TRMM ) GloSea4 mean rainfall ( ), 25 April start dates Good agreement between observed and GloSea4 temporal evolution of monsoon and onset timing Some skill in predicting late/early onset (ROC score ~0.6) Seasonal forecasts with GloSea4 of timing of monsoon onset over Sahel

Improving ENSO forecasts Obs The westward extension of Nino is a common error in many climate models. It affects remote regions. High-res model has better ENSO pattern and teleconnections Low resolution High resolution

© Crown copyright 2011 ENSO teleconnections: precipitation JJA DJF Forecast (E-L)Observed (E-L) Skilful reproductions in the tropics – even for rainfall Red = El Nino is drier Blue = El Nino is wetter

© Crown copyright Met Office Examples of simple forecast products: decadal/gpc-outlooks

© Crown copyright 2007 ‘Raw’ products

© Crown copyright Met Office Examples of bespoke forecast products and information

GloSea4 Forecast probabilities for 2011 Short Rains (Sep-Nov) Early onset:Late onset: Courtesy of Michael Vellinga

Observations for 2011 Plots courtesy of Lizzie Good Courtesy of Michael Vellinga

Forecast products Deterministic forecasts Provides a best estimate and forecast range (±1 stdev interval) for: Numbers of named storms ACE index During the following 6 months Probabilistic forecasts Probability distributions Exceedance of thresholds (to aid assessment of risk) Help to quantify and communicate the inherent uncertainties in the forecast. Public forecast Tailored products

© Crown copyright Met Office Summary How is long-range forecasting possible? Large scale forcing that evolve slowly can make climate predictable Dynamical seasonal prediction systems Must include: initialisation, a climate model, and a way to assess uncertainty Hindcasts Due to model biases hindcasts are needed for correction of forecasts, They are also used to assess forecast quality, and can lead to model improvements Products We are developing new and exciting bespoke products and would like to make further developments with you

Questions and answers