1. Monthly Forecasting at ECMWF
Frédéric Vitart
European Centre for Medium-Range Weather Forecasts

2. Index Main sources of predictability on the monthly time-scale
Madden Julian Oscillation
Soil Moisture
Stratospheric Initial conditions
The ECMWF monthly forecast system
Description
Some examples of forecasts
Skill

3. Forecasting systems at ECMWF
Product
ECMWF:
Weather and Climate
Dynamical Forecasts
Medium-Range
Forecasts
Day 1-10(15)
Monthly
Forecast
Day 10-32
Seasonal
Month 2-7
Slide 3: The monthly forecasting system fills the gap between two currently operational forecasting systems at ECMWF: medium-range weather forecasting and seasonal forecasting. Medium-range weather forecasting produces weather forecasts out to 15 days, whereas seasonal forecasting produces forecasts out to 7 months. The two systems have different physical bases. Medium-range weather forecasting is essentially an atmospheric initial value problem. Since the time scale is too short for variations in the ocean significantly to affect the atmospheric circulation, the ECMWF medium-range weather forecasting system is based on atmospheric-only integrations. SST anomalies are simply persisted. Seasonal forecasting (2-7 months forecasts), on the other hand, is justified by the long predictability of the oceanic circulation (of the order of several months) and by the fact that the variability in tropical SSTs has a significant global impact on the atmospheric circulation. Since the oceanic circulation is a major source of predictability in the seasonal scale, the ECMWF seasonal forecasting system is based on coupled ocean-atmosphere integrations.

4. Bridging the gap between seasonal forecasting and NWP
A particularly difficult time range: Is it an atmospheric initial condition problem as medium-range forecasting or is it a boundary condition problem as seasonal forecasting?
Some sources of predictability in the monthly time scale:
Sea surface temperature/Sea ice
Snow cover
Soil Moisture
Stratospheric Initial conditions
The Madden-Julian oscillation
Slide 4:The time range 10 to 30 days is a very difficult time range for weather forecasts. It is not clear if it is an atmospheric initial condition problem as medium-range weather forecast or if it is a boundary condition problem as seasonal forecast. Most likely, it is a combination of both. The time range 10 to 30 days is probably too long for the atmosphere to keep a memory of its initial conditions, and too long for the ocean variability to have an impact on the atmospheric circulation. Therefore, for a long time, it was assumed that there was almost no predictability in this time scale. However, two main sources of predictability in this timescale have been discovered in the recent years:
- The impact of the stratosphere on the troposphere
The Madden-Julian Oscillation (MJO)
Impact of soil moisture initial conditions
Impact of snow cover
Impact of surface and sea surface temperatures

5. Impact of soil moisture
Slide 5: Impact of land moisture initialization on the skill of a multi-model ensemble to predict 2-metre temperature anomalies for different time ranges (1-15 days, days, days and days) and different amplitude of soil moisture anomalies in the initial conditions.
Koster et al, GRL 2010

6. Impact of soil moisture
Slide 6: Same as Slide 5 but for the skill of the multi-model to predict precipitation anomalies.
Koster et al, GRL 2010

7. Stratospheric Sudden Warmings
Slide 7: A series of images shows the development of the first Stratospheric Sudden Warming monitored by satellite, in early As the air warms, the winds that normally circle the North Pole counterclockwise switch direction. The low pressure area over the pole, in blue, distorts and splits. These warming events may affect winter weather patterns on the ground. (Source: Chui and Kunz, 2009, symmetry).
Chui and Kunz, 2009

8. Stratospheric influence on the troposphere?
Slide 8: Baldwin et al. (2003) found that the stratospheric polar vortex varies relatively slowly compared to the tropospheric circulation, and that it has an impact on the troposphere. Slide 5 shows the vertical cross section of a composite of 18 weak vortex events as a function of time. This figure displays a propagation from the stratosphere to the lowest levels of the troposphere, well beyond 10 days. This suggests that there is predictability in the monthly timescale.
Baldwin and Dunkerton, 2001

9. Z1000 Response (Weak vortex-CTL)
Stratospheric influence on the troposphere?
D+1-D+10
D+11-D+20
Z1000 Response (Weak vortex-CTL)
From T. Jung et al 2005
D+21-D+30
D+31-D+40
Slide 9: This slide shows the impact of the stratospheric forcing on the geopotential height at 1000 hPa in a GCM. Close to the surface, the largest and most statistically significant response is found in the north-eastern North Atlantic and parts of the Arctic, at least after more than 20 days in the integration.

10. Synoptic Z500 Activity D+21-D+30
Stratospheric influence on the troposphere?
Synoptic Z500 Activity D+21-D+30
Slide 10: This slide shows the difference of synoptic activity in the range from D+21 to D+30 between strong and weak polar stratospheric vortex. The synoptic activity is computed by taking the standard deviation of the day-to-day Z500 changes. The largest and most statistically significant impact of the stratospheric forcing is found over Northern Europe and the north-eastern North Atlantic, highlighting the fact that extended range forecasts in the European region should benefit the most from stratosphere-troposphere coupling.
From T. Jung et al 2005

11. Stratospheric Sudden warming- January 2009
SSW Index (T50 gradient)
15/1/2009
8/1/2009
Slide 11: Stratospheric Sudden Warming (SSW) Index. The SSW index is calculated by integrating the gradient of temperature at 50 hPa between 30N and 0N around the globe. The black line represents the analysis. Each green line represents one ensemble member. The red line represents the ensemble mean, and the blue line represents the control run. The left panel represents the monthly forecast of 8 January 2009 and the right panel the monthly forecast of 15 January 2009.

12. Stratospheric Sudden warming- January 2009
15/1/2009 2mtm anomaly Forecast
Composite
Bad SW
Composite
Good SW
Analysis
Day 19-25
Day 26-32
Slide 13: 2-metre temperature forecasts with the ensemble members which predicted correctly a SSW and with the ensemble members which did not predict a SSW. Both members with and without SSW display cold anomalies over west Europe, but the forecasts obtained from the members with SSW (right panels) are closer to the analysis (left panel) than the forecast obtained from the members without SSW, particularly over Scandinavia. Therefore it seems that the correct prediction of the SSW helped to get a better 2-metre temperature forecast over Europe, although the impact is of second order.

13. The Madden-Julian Oscillation (MJO)
Slide 14: The MJO was first described by Madden and Julian (1971). By looking at rawinsonde data provided by a station in the central Pacific (Canton), they discovered large coherence between surface pressure, zonal winds and temperature at various levels over a broad period range that maximized between 41 and 53 days. By looking at data from other stations, they discovered a coherence in the day range between stations far removed from one another. The sum of evidence pointed toward an oscillation, which is an eastward movement of large-scale circulation.
From Madden and Julian (1972)

14. The Madden Julian Oscillation (MJO)
MJO life cycle
Slide 12 : The left panel shows the evolution of OLR anomalies during a typical MJO. The blue colour indicates increased convection (positive phase of the MJO) and the red colour indicates suppressed convection (negative phase of the MJO). The MJO starts with increased convection in the Indian Ocean. This convection propagates eastwards, and it reaches the Maritime continent by about day 9. It reaches the western Pacific by day 15, and stops at the dateline (180E). Then, the negative phase of the MJO starts, with suppressed convection over the Indian Ocean propagating eastward. The right panel shows the time evolution of OLR anomalies over a period of 3 years from observations (2001 to 2003). In this Hovmoeller diagram, MJO episodes are well visible, but they are not regular. There are periods of strong MJO activity, and periods with low MJO activity. The different MJO events display a lot of variability in intensity and main characteristics, in the same way as ENSO events do not look all the same. The non-regularity in the occurrence of the MJO, makes the forecast of the MJO non trivial.
(From NASA)
From

15. The Madden Julian Oscillation (MJO)
The MJO is a day oscillation
The MJO is a near-global scale, quasi-periodic eastward moving disturbance in the surface pressure, tropospheric temperature and zonal winds over the equatorial belt.
The Madden-Julian Oscillation (MJO) is the dominant mode of variability in the tropics in time scales in excess of 1 week but less than 1 season.
The MJO has its peak activity during Northern winter and spring.
Slide 15: This is a summary of the previous slide: the MJO is a day oscillation. The MJO is a near-global scale, quasi-periodic eastward moving disturbance in the surface pressure, tropospheric temperature and zonal winds over the equatorial belt. The Madden-Julian Oscillation (MJO) is the dominant mode of variability in the tropics in time scales in excess of 1 week but less than 1 season. The MJO has its peak activity during Northern winter and spring.

16. The Madden Julian Oscillation (MJO)
Why is the MJO so important?
Impact on the Indian and Australian summer monsoons (Yasunari 1979), Hendon and Liebman (1990)
Impact on ENSO. Westerly wind bursts produce equatorial trapped Kelvin waves, which have a significant impact on the onset and development of an El-Niňo event. Kessler and McPhaden (1995)
Impact on tropical storms (Maloney et al, 2000; Mo, 2000)
Impact on Northern Hemisphere weather
Slide 16: The MJO has a significant impact on the Indian summer monsoon (Yasunari 1979). Clouds associated with the active phase of the Indian monsoon propagate northward through the Indian Ocean and Indian subcontinent at about 1 degree latitude per day (Murakami, 1976). Yasunari (1979) associated these northward moving clouds to the MJO.
The MJO has also a significant impact on the Australian monsoon. Hendon and Liebmann (1990) noticed that 27 of the 30 monsoon onsets from 1957 to 1987 coincided with the arrival of clouds associated with the MJO.
Several papers have suggested an impact of the MJO on the onset and development of the 1997 El-Nino event. During an active phase of the MJO, intense near-surface westerly wind events develop over the tropical Pacific Ocean, known as westerly wind bursts (Kiladis et al, 1994). Kindle and Phoebus (1995) documented a significant impact of the westerly wind bursts on the ocean through the initiation of equatorial trapped Kelvin waves. Kessler and McPhaden (1995) suggested that westerly wind bursts have a significant impact on the onset and development of an El-Nino event.
The MJO impacts the low-level vorticity and vertical wind shear over the eastern North Pacific and the Gulf of Mexico (Maloney et al, 2000), which has a significant impact on the tropical storm cyclogenesis. Mo (2000) documented that the suppression of deep convection over the Pacific during the early phase of the MJO is conducive to more easterly wind in the upper troposphere over Atlantic, which implies less vertical wind shear of the horizontal wind, and therefore more favorable conditions for cyclogenesis.

17. Ting and Sardeshmukh JAS 1993
Impact of the MJO on Extratropics
Slide 17: Impact of imposing a Heat source in the Tropics on Z500 circulation in the Extratropics. This shows a heat source over the maritime continent (consistent with MJO EPF1) has little impact o the Z500 circulation. Imposing a heat source over the Indian Ocean and a cooling over the west Pacific (consistent with MJO EOF2) has a significant impact on the Extratropics.
Lin et al, MWR 2010
See also
Simmons et al JAS 1983
Ting and Sardeshmukh JAS 1993

18. From Wheeler and Hendon, BMRC
MJO Prediction
Combined EOF1
Combined EOF2
Slide 18: In order to assess the skill of the monthly forecasting system to predict an MJO event, the forecasts are projected into combined EOFs of U200, U850 and OLR (see Wheeler and Hendon, 2004). Those combined EOFs have been computed using NCEP reanalysis and the two dominant combined EOFs , which represent about 12% of the total variance each, give a good representation of the MJO propagation.
From Wheeler and Hendon, BMRC

19. MJO FORECAST
Slide 19: The time evolution of the MJO of a monthly forecast is described by a multivariate MJO index (Wheeler and Hendon 2004 Mon. Wea. Rev. vol. 132, 8 p ). The diagram represents 8 regions of the two dimensional phase space defined by the first two principal components (RMM1 and RMM2) of a combined fields (OLR, zonal wind at 850 hPa and 200 hPa) averaged between 15S and 15N. Individual ensemble member values at day 1, 5, 10, 15 and 20 are represented respectively by a red, pink orange blue and green circles. The ensemble mean values (black triangles) are joined by a solid black line and the analysis values of the preceding 30 days are joined by a grey line. The grey squares represent the analysis values of the preceding 5, 10, 15, 20, 25 and 30 days. Points representing sequential values trace anticlock-wise circles around the origin, which signifies systematic eastward propagation on the MJO. Large amplitudes (outside of the circle) signify strong cycles of the MJO, while weak activity appears as rather random motion near the origin. The solid blue line represents the verification computed from the ECMWF operational analysis.

20. Impact on Europe Cassou (2008)
Slide 20: Lagged relationships between the eight phases of the MJO (rows) and the four North Atlantic weather regimes (columns) from ERA40. This figure shows a significant impact of some phases of the MJO on the NAO (see Cassou 2008)

21. Experiment’s setting:
MJO prediction
Experiment’s setting:
46 day forecasts at T255L62 coupled to HOPE
15 members
Starting dates: 15 Nov/Dec/Jan/Feb/Mar/Apr
Model Cycle 32R3 (operational cycle from 11/07 to 06/08)
Slide 21: This slide describes an experiment which has been set up to assess the skill of IFS to represent and predict MJO events. With this experiment, we can also estimate the MJO teleconnections in the model.

22. Ensemble mean/ reanalysis Ensemble mean/ reanalysis
MJO Skill scores
Bivariate RMS error
Bivariate Correlation
Slide 22: MJO skill scores. Left panel shows the bivariate correlation and the right panel shows the bivariate RMS error (black lines).
Ensemble mean/ reanalysis
Ensemble mean/ reanalysis
Ensemble Spread
“Perfect Model”
Climatology

23. Impact on Precipitation anomalies (Summer)
Slide 23: Impact of the MJO on precipitation. The plots show the precipitation anomalies composited over all the cases in phases 2+3, 4+4, 6+7 and 8+1 of the MJO in the model (left panel) and in the analysis (right panel) for the period DJF The model seems to represent very well the northward propagation of the MJO and its teleconnections over Central America.

24. Impact on Tropical Cyclone Density (Summer)
Slide 24: Impact of the MJO on tropical storm activity. The plots show the anomalies of tropical storm density composited over all the cases in phases 2+3, 4+5, 6+7 and 8+1 of the MJO in the model (left panel) and in the analysis (right panel). The model seems to represent very well the impact of the MJO on tropical cyclones.
Vitart, GRL 2009

25. Impact on the Extratropics- Z500 anomalies
Slide 25: Impact of the MJO on the Extratropics. In agreement with Cassou (2008), composites of phase days of the MJO project into a positive NAO (top panels), and composites of Phase days of the MJO project into a negative phase of the NAO in the analysis (bottom panels). In the model, the teleconnections are consistent with reanalysis (right panels), but are too weak over the Atlantic. The impact of the MJO on the NAO seems to be underestimated in the model.

26. Impact of MJO on Z500 anomalies
1 std< AMP < 1.5 std
AMP > 2 std
1.5 std< AMP < 2 std
Slide 26: Impact of the MJO on the Z500 anomalies for phase days as a function of the amplitude of the MJO. Stronger MJOs have a stronger impact than weaker MJOs.
Interval = 5 metres

27. Impact on weather regimes in hindcasts
Phase3+10 days
Phase6+10 days
NAO-
NAO+
Atlantic ridge
Scandinavian blocking
Slide 27: Impact of the MJO on the frequency of 4 Euro-Atlantic weather regimes. Red bars are for Phase days and blue bars are for Phase days. This plot shows that the strongest impact of the MJO is on the frequency of NAO+.

28. T850 anomalies – NDJFM 1989-2008 Phase 3 + 10 days Phase 6 + 10 days
ERA
MODEL
Slide 28: Impact of the MJO on 2-metre temperature anomalies. Left panels are for Phase3+10 days and the right panels for Phase6+10 days. Top panels show results obtained with ERA Interim and bottom panels show results obtained from the model.
Degree C

29. Probabilistic skill scores – NDJFMA 1989-2008
Reliability Diagram
Probability of 2-m temperature in the upper tercile
Day 19-25
N. Extratropics
Europe
0.04
0.03
-0.06
-0.09
Slide 29: Reliability of the probability that 2-metre temperature anomalies are in the upper tercile for the period day for the Northern Extratropics (left panel) and Europe (right panel). The red (blue) line represents the reliability diagram obtained with all the cases with an MJO (no MJO) in the initial conditions. The numbers represent the Brier Skill Scores. Only land points have been included in the calculation of the reliability diagram and the Brier skill scores. This figure shows that the MJO has a major source of predictability for the time range day in the ECMWF forecasting system.
MJO in IC
NO MJO in IC

30. Impact of the Extratropics on the MJO?
Slide 30: Lagged probability of the NAO index to be in the upper (positive numbers) or lower (negative numbers) terciles as a function of the phase of the MJO. The columns indicate the phase of the MJO. The columns indicate the lag in pentads. Lag 0 corresponds to the MJO in phase 1 to 8. Positive lags correspond to the MJO event preceding the NAO. Negative lags correspond to the NAO event preceding the MJO event. (Lin et al, 2009). This table suggests that the MJO has an impact on the NAO (positive lag) and that the NAO has an impact on the MJO (negative lag). Therefore, there seems to be a 2-way interaction between NAO and MJO.

31. Impact of N. Extratropics on MJO forecast skill
Slide 31: Bivariate correlation (forecast skill) as a function of the forecast lead time: control integration (CNT, black), relaxation of the Northern Extratropics to initial conditions (red) and analysis data (red). The shaded area represent the 5% level of confidence intervals computed using a bootstrap resampling technique. This figure suggest that the Northern Extratropics impact the Madden Julian Oscillation.
Vitart and Jung

32. Relaxed to Observations
March 1997 Westerly Wind Burst -U850
Relaxed to
Initial Conditions
Relaxed to Observations
Obs.
Control
Slide 32: Hovmoeller diagrams of tropical (15S-15N) zonal wind anomalies at 850 hPa for the forecast started on 15 February The top of the panels (above the horizontal black line) shows the previous 30 days from ERA Interim. Results are shown for a) ERA Interim, b) N. Extratropics relaxed towards initial conditions, c) Control, and d) the N. Extratropics relaxed towards analysis. This figure shows that this MJO event was modulated by the Northern Extratropical circulation.

33. The ECMWF monthly forecasting system
A 51-member ensemble is integrated for 32 days twice a week (Mondays and Thursdays at 00Z)
Atmospheric component: IFS with the latest operational cycle and with a T639L62 resolution till day 10 and T319L62 after day 10.
Persisted SST anomalies till day 10 and ocean-atmosphere coupling from day 10 till day 32.
Oceanic component: HOPE (from Max Plank Institute) with a zonal resolution of 1.4 degrees and 29 vertical levels
Coupling: OASIS (CERFACS). Coupling every 3 hours.
Slide 33:Description of the VarEPS-monthly forecasting system. Each week, the coupled model is integrated forward to make a 32 day forecast with 51 different initial conditions, in order to create a 51-member ensemble.

34. The ECMWF VarEPS-monthly forecasting system
EPS Integration at T639
Initial condition
Heat flux, Wind stress, P-E
Day 10
Day 32
Coupled forecast at TL319
Slide 34: New monthly forecasting system. Previously the monthly forecasting system consisted of coupled ocean-atmosphere integrations with an atmospheric horizontal resolution at TL159. Now, the monthly forecasting system and the VarEPS system have been merged. This graphic displays the new configuration: atmosphere-only at TL639 forced by persisted SST anomalies till day 10 twice a day. After day 10, the atmospheric model at a TL319 resolution is coupled to an ocean model till day 32.
The coupled model consists of the ECMWF atmospheric model (the same cycle as the deterministic forecast), coupled to an ocean general circulation model, which is a version of the Hamburg Ocean Primitive Equation model (HOPE), developed at the Max Plank Institute for Meteorology, Hamburg. The ocean model has lower resolution in the extratropics but higher resolution in the equatorial region, in order to resolve ocean baroclinic waves and processes, which are tightly trapped at the equator. The ocean model has 29 levels in the vertical. The atmosphere and ocean communicate with each other through a coupling interface, called OASIS, developed at CERFACS, France. The atmospheric fluxes of momentum, heat and fresh water are passed to the ocean every 3 hours and, in exchange, the ocean sea surface temperature (SST) is passed to the atmosphere.
Ocean only integration

35. The ECMWF monthly forecasting system
Atmospheric initial conditions: ECMWF operational analysis
Oceanic initial conditions: “Accelerated” ocean analysis
Perturbations:
Atmosphere: Singular vectors + stochastic physics
Ocean: Wind stress perturbations during the data assimilation
Background statistics:
5-member ensemble integrated at the same day and same month as the real-time time forecast over the past 18 years (a total of 90 member ensemble). Initial conditions: ERA Interim. Produced once a week.
Slides 35 In order to initiate monthly forecasts, initial conditions for both the ocean and atmosphere are required. Atmospheric and land surface initial conditions are obtained from the ECMWF operational atmospheric analysis/reanalysis system. Oceanic initial conditions originate from the oceanic data assimilation system used to produce the initial conditions of the seasonal forecasting system 2. However, this oceanic data assimilation system lags about 12 days behind real-time. The lag is partially due to the fact that the SST, obtained by interpolating in time the weekly OIv2 SSTs produced by NCEP, can be up to 12 days behind real-time. A first option would be to wait for the oceanic initial condition to be created by the data assimilation system to start the forecast, as in seasonal forecasting. This would mean losing 12 days of forecast and is not therefore suitable for monthly forecasting. A second option would be to persist the SST anomalies of the latest ocean analysis. However, we have some information about the wind stress and heat fluxes during the last 12 days of the ECMWF atmospheric analysis; this information can be used to help determine the present ocean initial state. Therefore, the option that has been chosen for monthly forecasting consists in integrating the ocean model from the last ocean analysis forced by analyzed wind stress, heat fluxes and P-E. During this 'ocean forecast', the sea surface temperature is relaxed towards persisted SST, with a damping rate of 100 W/m2/K.
Because of model errors, a drift occurs in the coupled system. In order to evaluate this model drift, the coupled model is integrated with 5 different initial conditions (5-member ensemble) at the same day and month as the real time forecast, but over the past 18 years, creating a 90-member climate ensemble.

36. The ECMWF monthly forecasting system
Anomalies (temperature, precipitation..) -
Slide 36: Anomaly maps are similar to seasonal forecasting charts, but with weekly means instead of monthly means. Over each point of the map, atmospheric variables such as 2-metre temperature, total precipitation, mean sea-level pressure or surface temperature, have been averaged over a weekly period (week 1: day 5 to 11, week 2: day 12 to 18, week 3: day 19 to 25, and week 4: day 26 to 32) and also over the 51 members of the real-time forecast and the 60 members of the back statistics. The plots display the difference between the ensemble mean of the real-time forecast and the ensemble mean of the back-statistics. The product therefore displays the shift of the forecast ensemble mean from the estimated "climatological" mean (created from ensemble runs over the past 18 years). In addition, a Wilcoxon-Mann-Whitney test (WMW-test, see for instance Wonacott and Wonacott 1977) has been applied to estimate whether the ensemble distribution of the real-time forecast is significantly different from the ensemble distribution of the back-statistics. Regions where the WMW-test displays a significance less than 90% are blank. Regions where the WMW-test displays a significance exceeding 95% are delimited by a solid contour (blue or red depending on whether the anomaly is positive or negative respectively). The blanking of "non-significant" shifts does not mean that there is no signal in the blanked regions, but only that, with the particular sampling we have, we cannot be sure that there is a signal. For this reason, there are likely to be many areas where a signal is real but remains undetected.

37. The ECMWF monthly forecasting system
Probabilities (temperature, precipitation..) -
Slide 37: Probability and tercile maps are also produced. An example of tercile map for the period day is displayed on this slide.

38. The ECMWF monthly forecasting system
Slide 38: Probability and tercile maps are also produced. An example of tercile map for the period day is displayed on this slide.

39. Experimental product: Tropical cyclone activity
The ECMWF monthly forecasting system
Experimental product: Tropical cyclone activity
Slide 39: Forecast of tropical cyclone activity. This plot shows the probability of tropical storm strike within 300 km predicted by the monthly forecast starting on 8 April 2010 and for the period day

40. MJO Forecasts
Slide 40: An example of real-time MJO forecast. The grey line represents the evolution of the MJO over the past 30 days. Each dot represents the position of the MJO at day 1,5,10,15,20 in the PC1/PC2 phase space. See slides 18 and 19 for more details.

41. Precip anomalies : 26 July 2010 – 01 August 2010
Slide 41: Prediction of the Pakistan floodings in The top left panel shows the precipitation anomalies during the week 26/07/ /07/2010. The other panel shows the ensemble mean precipitation anomalies from the ECMWF monthly forecasts for the time ranges: day 5-11, day and day This figure suggests that this event was predicted three weeks in advance.

42. ROC score: 2-meter temperature in the upper tercile
Skill of the ECMWF Monthly Forecasting System
ROC score: 2-meter temperature in the upper tercile
Day 5-11
Day 12-18
Day 19-25
Day 26-32
Slide 42: Maps of ROC scores of the probability that 2-meter temperature averaged over the period day is in the upper tercile. Only the scores over land points are shown. The terciles have been defined from the model climatology. The verification period is Oct 2004-May Red areas indicate areas where the ROC score exceeds 0.5 (better than climatology). This plot shows that the coupled model performs better than climatology for the period days
For the period days 19-26, the skill is much lower than for days 12-18, as expected. The red is largely dominating overall, suggesting that the model generally performs better than climatology at this time scale. Europe seems to be a difficult region, with very low skill at this time range. Tropical regions display the strongest skill after 30 days, suggesting that the coupled model at this time range starts to behave more like seasonal forecasting.

43. Skill of the ECMWF Monthly Forecasting System
2-meter temperature in upper tercile - Day 12-18
ROC score
Reliability diagram
Persistence of day 5-11
Day 12-18
Monthly forecast day 12-18
Day
19-25
Persistence of day 5-18
Slide 43: The toughest test for monthly forecasting is a comparison with the persistence of the medium-range forecasts. If the monthly forecast does not perform better than persisting the EPS forecasts, then it is useless. The top left plot shows the ROC diagram obtained with the monthly forecasting system for days (red) and the persistence of the probabilities of days 5-11 (about the same as persisting the last week of EPS) (blue). The event is the probability that 2-meter temperature is in the upper tercile. The top right panel represents the reliability diagram of the monthly forecast for day (blue line) and persistence of day 5-11 (red line). This slide demonstrates that the monthly forecast of days performs better than persisting the medium range forecast, and therefore can be useful. The difference is statistically significant according to WMW test. This result is also valid for all the other variables and other probabilistic events. It can be concluded that the model show some skill over the northern Extratropics at this time range, and therefore there is no reason to stop the EPS at day 10.
The bottom panels show the same figures for the time range day and the persistence of day 5-18.
Monthly forecast day 19-32

44. OLR anomalies - Forecast range: day 15
Slide 44: Hovmoeller diagram of OLR from ERA40 (left panel) and forecasts at day 15 from 8 successive versions of IFS since the monthly forecasting system became operational. This slide shows that there has been a considerable improvement in the amplitude of the MJO since the monthly forecasting system became operational. However the propagation of the MJO is too slow over the Indian Ocean in the last model cycles.

45. MJO skill scores
Slide 45: ECMWF has produce monthly forecasts routinely since March 202. This figure shows the evolution of MJO skill scores since The MJO skill scores (bivariate correlations) have been computed on the model hindcasts produced during a complete year. For instance, 2002 indicates the hindcasts that were produced from March 2002 until March corresponds to the hindcasts produced over the past year (from March 2011 until March 2012). The blue line indicates the day when the MJO bivariate correlation reaches 0.5. The red line indicates the day when the MJO bivariate correlation reaches 0.6 and the brown line indicates the day when the MJO bivariate correlation reaches 08. This suggests that the MJO skill scores have significantly improved over the past 10 years. If we consider the 0.6 correlation as a measure of MJO predictability, then the gain is of about 9 days of predictability.

46. NAO skill scores All cases NDJFM NDJFM MJO in IC NO MJO in IC
Slide 46: The left panel shows the evolution of the NAO skill score (measured here as the correlation between the ensemble mean NAO index with ERA Interim re-analysis) from 2002 until This shows that the skill of the ECMWF models to predict the NAO has improved over the years. The right panel shows the same correlations but we separate the cases when there is an MJO in the initial conditions (red curve) and the cases when there is an MJO in the initial conditions (black curve). This suggests that the improved NAO prediction may come essencially from the improved prediction of the MJO.
MJO in IC
NO MJO
in IC

47. Performance of the monthly Forecasts
2-metre temperature ROC area over Northern Extratropics
Slide 47: Evolution of the skill (ROC area of 2-metre temperature) of the monthly forecasting system year by year since 2005 (Roc area of the probability that 2-metre temperature anomaly is in the upper tercile). There has been an improvement in the skill of the monthly forecasting system since The skill for day is now close to the skill of the day years ago.
Day 12-18
Day 19-25
Day 26-32

48. Conclusion
SSTs, Soil moisture, stratospheric initial conditions and MJO are source of predictability at the intra-seasonal time scale. In particular the MJO has a significant impact on the forecast skill scores beyond day 20. Model improvements, particularly in simulating the MJO activity are likely to be beneficial for monthly forecasting.
The monthly forecasting system produces forecasts for days that are generally better than climatology and persistence of day Beyond day 20, the monthly forecast is marginally skilful. For some applications and some regions, these forecasts could however be of some interest. There has been a clear improvement in the monthly forecast skill scores over the past 10 years. This improvement is likely to be related to improved prediction in the Tropics and most especially improved MJO prediction.