1. Franco Molteni, Tim Stockdale, Frederic Vitart, Laura Ferranti
ECMWF operational and research activities on extended and long range predictions
Franco Molteni, Tim Stockdale, Frederic Vitart,
Laura Ferranti
WCRP/WGIP WMO/ETOPLSL session March 2014

2. The operational forecasting system
High resolution forecast: twice per day 16 km 91-level, to 10 days ahead
Ensemble Prediction System (ENS): twice daily 51 members, 30/60 km 62-level, to 15 days ahead
Extended range forecasts : twice a week (Mon/Thursdays) 51 members, 30/60 km 92 levels, to 1 month ahead
Long range forecasts: once a month (coupled to ocean model) members, ~80 km 91 levels, to 7 months ahead
Eurosip multi-model system : M. France, Met. Off. , NCEP and ECMWF
The operational forecasting system at ECMWF consists of:
- An high resol. Forecast
- An ENSEMBLE of 51 forecasts that also run twice a day up 15 days --- covering the Medium range predictions: beyond 72 hours and up 240.
- The Extended forecast system: These forecasts are issued twice a week (Mon/Thu) and run up to 32 days and they are an extension of the ENSEMble forecasting system
Long range ensemble forecast issued once a month that run up to 7 months.
This presentation is meant to be an introduction to the Extended range forecast: Currently extended range products are provided to the ECMWF Member and Co-operating States and to commercial users
WCRP/WGIP WMO/ETOPLSL session March 2014

3. Extended-range forecasts
A 51-member ensemble is integrated for 32 days twice a week (Mondays and Thursdays at 00Z)
Atmospheric component: IFS with the latest operational cycle and with a T639L91 resolution till day 10 and T319L91 after day 10.
Ocean-atmosphere coupling from day to NEMO (about 1 degree) every 3 hours.
Initial conditions:
Atmosphere: Operational 4-D var analysis + SVs+ EDA perturbations
Ocean: 3D-Var analysis (NEMOVAR) + wind stress perturbations
WCRP/WGIP WMO/ETOPLSL session March 2014

4. Extended-range forecasts
WCRP/WGIP WMO/ETOPLSL session March 2014

5. Extended-range forecasts:
Evolution of NAO skill at day
Difficulty for the MJO to cross the Maritime Continent
Too weak MJO teleconnections over the Euro-Atlantic sector
Too weak impact of the stratosphere on the troposphere
- Evolution of the MJO skill scores (bivariate correlations applied to WHI) since The MJO skill scores have been computed on the ensemble mean of the ECMWF re-forecasts produced during a complete year. The blue, red and brown lines indicate respectively the day when the MJO bivariate correlation reaches 0.5, 0.6 and 0.8.
- Evolution of the NAO skill. Correlation between the ensemble mean NAO index and the NAO index computed from ERA Interim) for the period day for all the cases when there is an MJO in the initial conditions (amplitude of the MJO index larger than 1 independently of the phase) and when there is no MJO in the initial conditions (amplitude of the MJO index less than 1)
Evolution of the MJO skill scores. The blue, red and brown lines indicate respectively the day when the MJO bivariate correlation reaches 0.5, 0.6 and 0.8.
From Vitart 2013
WCRP/WGIP WMO/ETOPLSL session March 2014

6. Extended-range forecasts:
Era-interim
Forecast
Too weak impact of the stratosphere on the troposphere
Lag correlation between NAO and SSW index as function of preceeding or following (positive x-axis) a SSW. The black line shows the lag correlation obtained from all the re-forecasts produced between October 2011 and March 2012 and covering the years 1995 to 2001 and for each ensemble member separately. The red line shows the corresponding verification using ERA Interim.
From Vitart 2013
Lag correlation between NAO and SSW index
WCRP/WGIP WMO/ETOPLSL session March 2014

7. Extended-range forecasts:
Evolution of RPSS of 2-metre temperature weekly mean anomalies over the northern Extra-tropics
This should improve the calibration and allow skill assessment. Evolution of the discrete ranked probability skill score (RPSS) of 2-metre temperature weekly mean anomalies over the northern Extratropics (North of 30N) since Only land points have been scored. The RPSS has been computed from terciles and for all the ECMWF re-forecasts covering all seasons. The red line shows the RPSS of day 12-18, the brown line represents the RPSS of day and the green line the RPSS of day
Future plans:
Extension of the re-forecasts: twice a week, 11-member ensemble.
Increased atmospheric horizontal resolution
- Possible extension to 45/60 days
- Increased oceanic resolution to a ¼ degree + sea ice model
WCRP/WGIP WMO/ETOPLSL session March 2014

8. Long range forecasts: IFS (atmosphere)
TL255L91 Cy36r4, 80km grid , full stratosphere
Initial conditions: ERA interim / operational
NEMO (ocean)
Global ocean model, 1x1 resolution, 0.3 meridional near eq.
Initial conditions: NEMOVAR (3D-Var) analyses
Sea ice
Specified values sampled from previous five years
Real time forecasts:
51 member ensemble forecast to 7 months
SST and atmos. perturbations added to each member
Re-forecasts:
(30 years) 15 members every month
51 members for Feb/May/Aug/Nov starts
WCRP/WGIP WMO/ETOPLSL session March 2014

9. Long range forecasts: ENSO forecasts
Well observed period it improves the skill both in term of acc and rmse spread is also improved despite the change in the persistence.
WCRP/WGIP WMO/ETOPLSL session March 2014

10. Long range forecasts: deterministic scores
MAM
JJA
SON
DJF
WCRP/WGIP WMO/ETOPLSL session March 2014

11. Long range forecasts: The dependence on ensemble size
Brier Skill Score (BSS) with climatology as a reference as a function of ensemble size.
Two versions of the BSS are used:
the uncorrected BSS as estimated from the ensemble directly.
- an analytical correction of the BSS extrapolating towards a hypothetical infinite ensemble size (Ferro, 2007) .
Impact of the seasonal forecast hindcast ensemble size on two estimates of the BSS. The blue dots represent the BSS as estimated from the finite ensemble. The red dots represent an extrapolation for an hypothetical ensemble of infinite size (Ferro, 2007). Different tercile events for 2m temperature and precipitation in DJF (Nov start dates) and JJA (May start dates) are shown for the Tropics, Europe and North America.
The score metrics we compare are:
Brier Skill Score (BSS) with climatology as a reference. Two versions of the BSS are used: the uncorrected BSS as estimated from the ensemble directly, and an analytical correction of the BSS extrapolating towards a hypothetical infinite ensemble size (Ferro, 2007).
The dependence on ensemble size
varies with the inherent predictability
of the event.

12. Long range forecasts:
Impact of different sources of uncertainty on the reliability skill estimates in the seasonal hind-casts as a function of ensemble size.
The error bars represent the 90% confidence intervals of the skill estimates.
The blue bars refer to the distribution of reliability obtained from bootstrapping the years of the hindcast period and reflect mainly the interannual variability in the 30-year period.
The magenta bars were obtained by randomly subsampling an ensemble of the given size from the
full set of 51 members.
The score metrics we compare are:
 Brier Skill Score (BSS) with climatology as a reference. Two versions of the BSS are used:
the uncorrected BSS as estimated from the ensemble directly, and an analytical correction
of the BSS extrapolating towards a hypothetical infinite ensemble size (Ferro, 2007).
 The slope of the reliability line. The reliability line is computed as a weighted linear
regression of all data points in a reliability diagram, see Figure 1. A reliability diagram is a
graphical display of the relationship between the forecast probabilities for an event and
the corresponding observed frequencies of the events given the forecasts.
slope = 1  perfect reliability slope = 0  no reliability slope > 0.5  BSS>0
The inter-annual variability of the hind-cast period (sample)
contributes the most to the uncertainties.
WCRP/WGIP WMO/ETOPLSL session March 2014

13. Long range Forecasts: Eurosip verification
2m temp. anomaly correlation for DJF over 14 years ( )
Sys4
One of the Eurosip system with best performance over Europe

14. Long range forecasts: Present research and future plans:
Effect of stochastic physics in mean climate and spread
Land surface initialization (SPECS)
Experimentation with dynamical sea-ice
Impact of enhanced resolution in both ocean and atmospheric components
Predictability of the AO and the role of initial conditions
Impact of sample and ensemble size on skill measures
Next system is likely features:
increased vertical levels and a better stratospheric physics
Dynamical sea-ice and enhanced resolution for the ocean
Impact of enhanced resolution in both ocean and atmospheric components (regimes, tropical storms, MJO)
Impact of sample and ensemble size on skill measures (particularly important for regions low signal-to-noise ratios)
WCRP/WGIP WMO/ETOPLSL session March 2014

15. Probabilistic skill scores – NDJFMA 1989-2008
Reliability Diagram
Probability of 2-m temperature in the upper tercile
Day 19-25
N. Extratropics
EUROPE
0.04
0.03
-0.06
-0.09
Slide 47: Reliability of the probability that 2-metre temperature anomalies are in the upper tercile for the period day for the Northern Extratropics (left panel) and Europe (right panel). The red (blue) line represents the reliability diagram obtained with all the cases with an MJO (no MJO) in the initial conditions. The numbers represent the Brier Skill Scores. Only land points have been included in the calculation of the reliability diagram and the Brier skill scores. This figure shows that the MJO has a major source of predictability for the time range day in the ECMWF forecasting system.
MJO in IC
NO MJO in IC
WCRP/WGIP WMO/ETOPLSL session March 2014

16. MJO forecast:
The time evolution of the MJO predicted by the EPS is described by a multivariate MJO index (Wheeler and Hendon 2004 Mon. Wea. Rev. vol. 132, 8 p ). The diagram represents 8 regions of the two dimensional phase space defined by the first two principal components (RMM1 and RMM2) of a combined fields (OLR zonal wind at 850 hPa and 200 hPa) averaged between 15S and 15N. Individual ensemble member values at day 1, 5, 10, 15 and 20 are represented respectively by a red, pink orange blue and green circles. The ensemble mean values (black triangles) are joined by a solid black line and the analysis values of the preceeding 30 days are joined by a grey line. The grey squares represent the analysis values of the preceeding 5, 10, 15, 20, 25 and 30 days. Points representing sequential values trace anticlock-wise trajectories around the origin,indicating systematic eastward propagation of the MJO. Large amplitudes (outside of the circle) signify strong cycles of the MJO, while weak activity appears as rather random motion near the origin.
WCRP/WGIP WMO/ETOPLSL session March 2014

17. Tropical storm density
Tropical storm density during the period October to March (left panels)
and when there is an MJO in Phase 2 or 3 (right panels). The top panels show observations
(from Joint Typhoon Warning Center), the middle panel shows the tropical storm densities
from the re-forecasts produced in 2011, and the bottom panel shows the tropical storm
densities from the re-forecasts produced in The tropical storm density is calculated by
computing the number of tropical storms passing within 500 kilometres and then normalise
that number by the total number of tropical storms over the whole basin
WCRP/WGIP WMO/ETOPLSL session March 2014

18. Weekly mean Accumulated Cyclone Energy (ACE)
The ACE of a tropical storm is calculated by summing the square of the estimated maximum sustained velocity of every active tropical storm at six-hour intervals. The ACE of a time period is the sum of the ACEs for each storm and takes into account the number, strength, and duration of all the tropical storms in the period. The 51-member ensemble forecast is compared with the model climatology (5-member ensemble over the past 18 years). A Wilcoxon-Mann-Whitney (WMW) test is then applied to evaluate if the predicted tropical storm frequency/location are significantly different from the climatology. The ocean basins where the WMW test detects a significance larger than 90% have a shaded background. Green bars represent the ensemble mean of the forecast and orange bars represent climatology. The values of each bar are written in black.
Available are also the tropical storm frequency: The "Tropical storms" have a relatively large range of wind speed (wind speed higher than 17m*s(-1)), so includes storms of hurricane strength. And Probabilities of tropical storms will strike within 300Km for weekly periods.
WCRP/WGIP WMO/ETOPLSL session March 2014

19. Skill of the ECMWF Monthly Forecasting System
ROC score: 2-meter temperature in the upper tercile
Day 5-11
Day 12-18
Day 19-25
Day 26-32
Slide 16 Map of ROC scores of the probability that 2-meter temperature averaged over the period day is in the upper tercile. Only the scores over land points are shown. The terciles have been defined from the model climatology. The verification period is Oct 2004-May Red areas indicate areas where the ROC score exceeds 0.5 (better than climatology). This plot shows that the coupled model performs better than climatology for the period days
For the period days 19-26, the skill is much lower than for days 12-18, as expected. The red is largely dominating overall, suggesting that the model generally performs better than climatology at this time scale. Europe seems to be a difficult region, with very low skill at this time range. Tropical regions display the strongest skill after 30 days, suggesting that the coupled model at this time range starts to behave more like seasonal forecasting.
WCRP/WGIP WMO/ETOPLSL session March 2014

20. Conclusion
SSTs, Soil moisture, stratospheric initial conditions and MJO are source of predictability at the intra-seasonal time scale. In particular the MJO has a significant impact on the forecast skill scores beyond day 20. Model improvements, particularly in simulating the MJO activity are likely to be beneficial for monthly forecasting.
The monthly forecasting system produces forecasts for days that are generally better than climatology and persistence of day Beyond day 20, the monthly forecast is marginally skilful. For some applications and some regions, these forecasts could however be of some interest.
WCRP/WGIP WMO/ETOPLSL session March 2014

21. Real time forecast
Reforecast ( )
51 members every months
15 members every months
51 members for May and Nov.
Previous studies (Déqué, 1997; Kumar et al., 2001) suggested that larger sizes of the reforecast ensembles would be needed for regions of low inherent predictability
The operational seasonal forecasts (System 4) are based on an ensemble of 51 members for their global predictions. However the ensemble size of the corresponding retrospective forecasts is 15 members over 30 years period ( ). Just for May and November starting dates the reforecast has been expanded to 51 members.
We study the impact of the ensemble size on probabilistic forecast skill, in particular on Brier Skill Score and reliability, for regions of high and low signal-to-noise ratios.