Presentation is loading. Please wait.

Presentation is loading. Please wait.

Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 1 The Seasonal Forecast System at ECMWF Tim Stockdale European Centre for Medium-Range.

Similar presentations


Presentation on theme: "Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 1 The Seasonal Forecast System at ECMWF Tim Stockdale European Centre for Medium-Range."— Presentation transcript:

1 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 1 The Seasonal Forecast System at ECMWF Tim Stockdale European Centre for Medium-Range Weather Forecasts

2 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 2 Contents Seasonal forecasting with coupled GCMs Why use GCMs? How we make a forecast Basic calibration – how, why, and what are the problems? Model error Operational forecasts: ECMWF System 3 System design How good are the El Nino forecasts? How good are the atmospheric forecasts? Multi-model systems and forecast interpretation EUROSIP multi-model system Meaning of probabilistic forecasts

3 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 3 Sources of seasonal predictability KNOWN TO BE IMPORTANT: oEl Nino variability- biggest single signal oOther tropical ocean SST- important, but multifarious oClimate change- especially important in mid-latitudes oLocal land surface conditions- e.g. soil moisture in spring OTHER FACTORS: oVolcanic eruptions- definitely important for large events oMid-latitude ocean temperatures- still somewhat controversial oRemote soil moisture/ snow cover- not well established oSea ice anomalies- local effects, but remote?? oDynamic memory of atmosphere- most likely on 1-2 months oStratospheric influences- various possibilities Unknown or Unexpected- ???

4 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 4 Methods of seasonal forecasting Empirical forecasting oUse past observational record and statistical methods oWorks with reality instead of error-prone numerical models oLimited number of past cases means that it works best when observed variability is dominated by a single source of predictability oA non-stationary climate is problematic Two-tier forecast systems oFirst predict SST anomalies (ENSO or global; dynamical or statistical) oUse ensemble of atmosphere GCMs to predict global response oMany people still use regression of a predicted El Nino index on a local variable of interest Single-tier GCM forecasts oInclude comprehensive range of sources of predictability oPredict joint evolution of SST and atmosphere flow oIncludes indeterminacy of future SST, important for prob. forecasts oModel errors are an issue!

5 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 5 Step 1: Build a coupled model IFS (atmosphere) T L 159L62 Cy31r1, deg grid for physics (operational in Sep 2006) Full set of singular vectors from EPS system to perturb atmosphere initial conditions (more sophisticated than needed …) Ocean currents coupled to atmosphere boundary layer calculations HOPE (ocean) Global ocean model, 1x1 mid-latitude resolution, 0.3 near equator A lot of work in developing the OI ocean analyses, including analysis of salinity, multivariate bias corrections and use of altimetry. Coupling Fully coupled, no flux adjustments, except no physical model of sea-ice

6 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 6 Step 2: Make some forecasts Initialize coupled system (cf Magdalenas lecture) Aim is to start system close to reality. Accurate SST is particularly important, plus ocean sub-surface. Dont worry too much about imbalances Run an ensemble forecast Explicitly generate an ensemble on the 1st of each month, with perturbations to represent the uncertainty in the initial conditions; run forecasts for 7 months Worry about model error later ….

7 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 7 Creating the ensemble Wind perturbations Perfect wind would give a good ocean analysis, but uncertainties are significant. We represent these by adding perturbations to the wind used in the ocean analysis system. BUT only have 5 member ensemble, and no representation of other sources of uncertainty in ocean analysis (obs error,..) SST perturbations SST uncertainty is not negligible SST perturbations added to each ensemble member at start of forecast. BUT perturbations based on analyses that use the same input data Atmospheric unpredictability Atmospheric noise soon becomes the dominant source of spread in an ensemble forecast. This sets a fundamental limit to forecast quality. To ensure that noise grows rapidly enough in the first few days, we activate stochastic physics and use EPS singular vectors.

8 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 8 Rms error of forecasts has been systematically reduced (solid lines) …. RMSE and spread in different systems.. but ensemble spread (dashed lines) is still substantially less than actual forecast error. Substantial amounts of forecast error are not from the initial conditions.

9 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 9 Step 3: Remove systematic errors Model drift is typically comparable to signal Both SST and atmosphere fields Forecasts are made relative to past model integrations Model climate estimated from 25 years of forecasts ( ), all of which use a 11 member ensemble. Thus the climate has 275 members. Model climate has both a mean and a distribution, allowing us to estimate eg tercile boundaries. Model climate is a function of start date and forecast lead time. EXCEPTION: Nino SST indices are bias corrected to absolute values, and anomalies are displayed wrt a climate. Implicit assumption of linearity We implicitly assume that a shift in the model forecast relative to the model climate corresponds to the expected shift in a true forecast relative to the true climate, despite differences between model and true climate. Most of the time, assumption seems to work pretty well. But not always.

10 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 10 ECMWF Met Office Météo-France CFS Biases (eg JJA 2mT as shown here) are often comparable in magnitude to the anomalies which we seek to predict Plots courtesy Paco Doblas-Reyes

11 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 11 SST bias is a function of lead time and season. More recent systems have less bias, but it is still large enough to require correcting for.

12 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 12 Despite SST bias and other errors, anomalies in the coupled system can be remarkably similar to those obtained using observed (unbiased) SSTs …..

13 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 13 … and can also verify well against observations

14 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 14 Model errors are still serious … Models have errors other than mean bias Eg weak wind and SST variability in system 2 Strong underestimation of MJO activity Suspected too-weak teleconnections to mid-latitudes Mean state errors interact with model variability Nino 4 region is very sensitive (cold tongue/warm pool boundary) Atlantic variability suppressed if mean state is badly wrong Our forecast errors are larger than they should be With respect to internal variability estimates and (occasionally) other prediction systems Reliability of probabilistic forecasts is not particularly high

15 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 15 ENSO forecast quality – Feb. starts only Completion of TAO array ECMWF S3, 1 Feb Starts, first 3 months of forecast

16 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 16 ENSO forecast quality – all start months Model error partially masks the benefit of improved observations

17 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 17 System 3 vs Cycle 33r1 The most recent ECMWF cycles (such as 33r1, shown here) develop a too- strong cold tongue in the West Pacific. This results in a large drop in skill in SST forecasts in Nino 4. This relationship between mean-state error and forecast performance in the Nino- 4 region has been seen time and again in different model versions. The model version used for System 3 has a pretty good balance, but even it is not perfect.

18 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 18 Red: statistically corrected System 3 forecast Improvement is mainly seen in Nino 4, where there are clear indications of anomaly / mean-state interactions, and where state-dependent errors are visible. (Statistical model is a cross-validated linear regression, based on initial condition SST anomaly and model predicted SST anomaly) Statistical post-processing Not used operationally

19 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 19 Capturing trends is important. Some are handled well ….

20 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 20 … other trends are poorly handled

21 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 21 Volcanic influence on Eurasian winters? ECMWF System 3 hindcastAnalysed 2m T anomaly DJF 82/83 DJF 91/92

22 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 22 Operational seasonal forecasts Real time forecasts since 1997 System 1 initially made public as experimental in Dec 1997 System 2 started running in August 2001, released in early 2002 System 3 started running in Sept 2006, operational in March 2007 Burst mode ensemble forecast Initial conditions are valid for 0Z on the 1 st of a month Forecast is created typically on the 11 th /12 th (SST data is delayed up to 11 days) Forecast and product release date is 12Z on the 15 th. Range of operational products Moderately extensive set of graphical products on web Raw data in MARS Formal dissemination of real time forecast data

23 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 23 System 3 configuration Real time forecasts: 41 member ensemble forecast to 7 months SST and atmos. perturbations added to each member 11 member ensemble forecast to 13 months Designed to give an outlook for ENSO Only once per quarter (Feb, May, Aug and Nov starts) November starts are actually 14 months (to year end) Back integrations from (25 years) 11 member ensemble every month 5 members to 13 months once per quarter

24 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 24 How many back integrations? Back integrations dominate total cost of system System 3:3300 back integrations (must be in first year) 492 real-time integrations (per year) Back integrations define model climate Need both climate mean and the pdf, latter needs large sample May prefer to use a recent period (30 years? Or less??) System 2 had a 75 member climate, System 3 has 275. Sampling is basically OK Back integrations provide information on skill A forecast cannot be used unless we know (or assume) its level of skill Observations have only 1 member, so large ensembles are much less helpful than large numbers of cases. Care needed eg to estimate skill of 41 member ensemble based on past performance of 11 member ensemble For regions of high signal/noise, System 3 gives adequate skill estimates For regions of low signal/noise (eg <= 0.5), need hundreds of years

25 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 25 Example forecast products A few examples only – see web pages for full details and assessment of skill Note: Significance values on plots A lot of variability in seasonal mean values is due to chaos Ensembles are large enough to test whether any apparent signals are real shifts in the model pdf We use the w-test, non-parametric, based on the rank distribution NOT related to past levels of skill

26 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 26

27 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 27

28 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 28

29 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 29

30 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 30 Other operational plots for JJA 2009

31 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 31

32 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 32 Many other fields are available from the forecast system … ( … although these are not routinely plotted)

33 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 33 SST forecast performance Actual rms errors > model estimate of perfect model errors

34 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 34 More recent SST forecasts are better

35 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 35

36 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 36 At longer leads, model spread starts to catch up

37 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 37

38 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 38

39 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 39

40 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 40 How good are the forecasts? Temperature: actual forecastsTemperature: perfect model Deterministic skill: DJF ACC

41 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 41 How good are the forecasts? Precip: actual forecastsPrecip: perfect model Deterministic skill: DJF ACC

42 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 42 How good are the forecasts? Tropical precip < lower tercile, JJANH extratrop temp > upper tercile, DJF Probabilistic skill: Reliability diagrams

43 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 43 How good are the forecasts? Europe: Temp > upper tercile, DJF Probabilistic skill: Reliability diagrams

44 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 44 EUROSIP multi-model ensemble Three models running at ECMWF: ECMWF – as described Met Office – HADCM3 model, Met Office ocean analyses Meteo-France – Meteo-France model, Mercator ocean analyses Unified system Real-time since mid-2005 All data in ECMWF operational archive Common operational schedule (products released at 12Z on 15 th ) Common products Coordinated development strategy (sort of …) See EUROSIP User Guide on web for details, and also the recent ECMWF Newsletter article (Issue No. 118, Winter 2008/09)

45 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 45 ECMWFEUROSIP Summer 2009 temperature forecast (See Antjes lecture for more on multi-model ensembles...)

46 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 46 Model error and forecast interpretation Model error is large It dominates El Nino forecast errors Mean state and variability errors are very significant Errors cannot be easily fixed Products typically account for sampling error only Dont take model probabilities as true probabilities Estimating forecast skill can be difficult In many cases, data is insufficient to produce sensible estimates Makes interpretation of forecasts tough In the end we need trustworthy models (Multi-model ensembles are small, and only partially span the space of model errors)

47 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 47 Calibrated forecasts Frequentist or Bayesian Sampling uncertainties give rise to frequentist probabilities Model errors mean our forecast pdfs are wrong. We can try to represent our lack of knowledge as probabilities in a Bayesian framework. Calibration of probabilistic forecasts Given the model forecast pdf, the past history of forecast pdfs, the past history of observed outcomes, what is the best estimate of the true pdf? A Bayesian analysis depends on specification of the prior. Mathematically, no single right answer. We hope one day to implement a hierarchical Bayesian analysis, accounting for the major uncertainties and producing some general purpose calibrated probabilistic forecasts. Handling trends and non-stationarities is an additional challenge

48 Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 48 Some final comments Plenty of scope for improving model forecasts Nino SST forecasts still much worse than predictability limits Model errors still obvious in many cases Initial conditions probably OK in Pacific for some time, recently improved elsewhere by ARGO Model output -> user forecast Calibration and presentation of forecast information Potential for multi-model ensembles Timescale for improvements Optimist: in 10 years, well have much better models, pretty reliable forecasts, confidence in our ability to handle climate variations Pessimist: in 10 years, modelling will still be a hard problem, and progress will largely be down to improved calibration


Download ppt "Training Course 2009 – NWP-PR: The Seasonal Forecast System at ECMWF 1 The Seasonal Forecast System at ECMWF Tim Stockdale European Centre for Medium-Range."

Similar presentations


Ads by Google