ECMWF User Meeting June 2006 The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden 1.Monthly instead of seasonal forecasting 2.The used of lagged forecasts (as a complement to the EPS) 3.Problems with weighting together different forecast systems For details see : forecast_products_user/Presentations2006/index.htm

ECMWF User Meeting June The seasonal forecasts Not used, partly because the forecasts seem to repeat themselves

ECMWF User Meeting June 2006 Warmer than normal The last four years ECMWF summer forecasts (issued in April) Warmer than normal Warmer than normal Warmer than normal Colder than normal Not warmer than normal Colder than normal Colder than normal

ECMWF User Meeting June The monthly forecast Used and found skilful, but tendencies of jumpiness in transitional periods

ECMWF User Meeting June April 27 April 2-3 week forecast of negative anomalies changed into 1-2 week forecast of positive Climate value Climate value

ECMWF User Meeting June 2006 Max +5 to 10 Max +10 to 15 Jumpiness experienced at a specific location 850 hPa temperature plume for Norrköping, southern Sweden clim

ECMWF User Meeting June 2006 Forecast for week June 2006Forecast for week June 2006 From 1June From 8June A very recent example of jumpiness (2m temp anom)

ECMWF User Meeting June The 21 day forecast (+9 days) Using the last three days 21 d forecasts enables us to inder the trends beyond day 10, even beyond day 15 For details see : forecast_products_user/Presentations2006/index.htm

ECMWF User Meeting June d20 d 30 d normal temperature statistics Last days ECMWF fc and EPS Last days Control +21 d forecasts Main method since summer 2003 ECMWF monthly forecast

ECMWF User Meeting June 2006 Consecutive daily 18-day lagged average forecast of 850 hPa temperature forecasts made from eight member averages of 21-day forecasts Already in in mid-April the lagged system provide hints about the temperature in early May 15 April15 May verif

ECMWF User Meeting June The problem of verification For details see : forecast_products_user/Presentations2006/index.htm

ECMWF User Meeting June 2006 Level of useful forecasts Introduction of more ECMWF data

ECMWF User Meeting June 2006 What to do? Two ways to go: 1.Political (cover up, play illusionist tricks or change the norms) 2.Scientific (go to the roots of the problem)

ECMWF User Meeting June 2006 ACC=98% Slope=0.8 I just happen to have some fresh verifications here, depicting the results during the first half of this year... Political trick: Selective sampling

ECMWF User Meeting June 2006 Anomaly correlation of monthly forecast for Stockholm (2 m temperature) More ECMWF input Verif Prog But it didnt look that bad….

ECMWF User Meeting June 2006 Scientific approach: The conventional verification disregarded three factors: 1. Variable range of variation between 2002 and More than one verification method should be used 3.Twelve forecasts per year is a too small sample For details see : forecast_products_user/Presentations2006/index.htm

ECMWF User Meeting June Lower correlation Smaller errors 2002 Higher correlation Larger errors

ECMWF User Meeting June 2006 f-a f-c a-c The RMSE in vector form yields angles as correlation measures β a f c ACC =cosβ

ECMWF User Meeting June 2006 a f-a f-c c Large variability high correlation (small β) but large errors β f a-c β a f Small variability low correlation (high β) and small errors f-a

ECMWF User Meeting June 2006 Introduction of more ECMWF data reduced the errors! Another verification method RMSE MABSE

ECMWF User Meeting June 2006 Verifying two years at a time (Lagged) verification over 24 months compared to over 12 months 12 months 24 months

ECMWF User Meeting June 2006 Mid-2006 Continued progress

ECMWF User Meeting June Swedish concerns about the quality of the centres EPS 1. Forecasters at SMHI and the Air Force do not find much use of the deterministic EPS compared to an elaborate use of the deterministic model 2. The scientists at SMHI and the MISU (Univ. Stockholm) are critical about the perturbations + (recently) the stochastic physics 3. My impression is not that the EPS is bad or has become worse, but has had problems to keep pace with the improvements of the deterministic model

ECMWF User Meeting June The size of the T42 EPS perturbations is very large The picture depicts the status before 1 February Since then the resolution of the deterministic system has increased by 50%, but the EPS perturbations which remain at their 1995 level of T42

ECMWF User Meeting June 2006 Before 2001 there was little quality difference between perturbed and non-perturbed forecasts, amounting beyond D+5 to an ACC difference. Since then it has increased to 10-15% ? Difference in ACC between the unperturbed Control and a randomly selected EPS member

ECMWF User Meeting June Over spreading in during the first hours made it difficult to use the EPS as BC for the HIRLAM 2.In cases of extreme or interesting events the signals often come 1-2 earlier in the T799 lagged system 3.In cases of consistent and skilful T799 performance the EPS keep the forecaster uncertain too long For more details see presentation at the OD Workshop November 2005

ECMWF User Meeting June 2006 The EPS perturbations make the forecasts 1 ½ days worse than Control! 1.5 days Unperturbed Control perturbed members The RMSE of individual EPS members The 2 m temperature forecasts for London winter Lagged EPS mean

ECMWF User Meeting June 2006 The RMSE of individual EPS members The 2 m temperature forecasts for London Feb-April 2006 perturbed members 1 day EPS mean Lagged Unperturbed Control

ECMWF User Meeting June 2006 Figure 2.1: Schematic image of the RMS error of the ensemble members, ensemble mean, and control forecast as a function of lead- time. The asymptotic predictability range is defined as the average difference between two randomly chosen atmospheric states. In a perfect ensemble system the RMS error of the ensemble members is a factor larger than the RMS error of the ensemble mean. Courtesy, L. Bengtsson, MISU climate RMSE(pert member)= (=sqrt2) RMSE (ensemble mean) Perturbed member Ensemble mean Control Tim Palmers Law

ECMWF User Meeting June 2006 Figure 4.2: Comparison of RMS error of the ensemble mean (green), the ensemble members (blue), the control forecast for the EPS (red) as well as the deterministic forecast as a function of lead-time (light blue). This comparison is made globally for the periods DJF (a) as well as JJA (d), and for regions 1 and 2 described in the text for the same time periods (plots b, c, e and f). The RMS errors are averaged over the globe and over the periods DJF and JJA. Courtesy L. Bengtsson, MISU

ECMWF User Meeting June Use of the last T799 runs forming lagged ensembles (work under development)

ECMWF User Meeting June 2006 EPS Mean and Lagged Mean 24 March 00 UTC + 84h

ECMWF User Meeting June 2006 Valid Sun 9 April 12 UTC EPS Mean and lagged ECMWF T799 4 April 00 UTC + 132h

ECMWF User Meeting June The public 6-10 day forecasts Once a week, four out of five forecasts verify

ECMWF User Meeting June day forecast presented on TV 26 January 2006

ECMWF User Meeting June 2006

Epsogram for Stockholm

ECMWF User Meeting June When does it pay to weight together forecasts? For details see : forecast_products_user/Presentations2006/index.htm E 1 < E 2

ECMWF User Meeting June 2006 f-a g-a a Ensemble mean error least for uncorrelated (orthogonal) errors f g β

ECMWF User Meeting June 2006 g f-a g-a a Correlated but equal errors β f

ECMWF User Meeting June 2006 g f-a g-a a Correlated, but non-equal errors Ensemble mean is not the optimal solution β f

ECMWF User Meeting June 2006 g f-a g-a a Correlated, but non-equal errors Weighted ensemble mean minimizes the error β f 90º

ECMWF User Meeting June 2006 a The weighted combination of two rather uncorrelated models (f 1 and g 1 ) can yield better forecast than the combination of two better, but correlated models (f 2 and g 2 ) When averaging orthogonality might compensate for lack of skill g1g1 f1f1 f2f2 g2g2

ECMWF User Meeting June 2006 g f-a g-a a At some stage any weighting will not improve the forecasts β f

ECMWF User Meeting June 2006 E 1 =f-a E 2 =g-a a Breaking point: when the fraction between the errors of the two systems equals the error correlation f g β

ECMWF User Meeting June How should x and y, the weights, be calculated taking the forecast error correlation into account? For details see : forecast_products_user/Presentations2006/index.htm E 1 < E 2

ECMWF User Meeting June 2006 Certain combinations of forecasts will not yield an improved weighted mean E 1 < E 2

ECMWF User Meeting June 2006 x 2 +y 2 b2b2 f-a g-a a f g β x y Pythagoras' Theorem not valid for the triangle a2a2 m β

ECMWF User Meeting June 2006 y2y2 m2m2 b2 β But Pythagoras' Theorem is valid for this right- angled triangle b 2 =y 2 +m 2 β

ECMWF User Meeting June 2006 a2a2 m2m2 β x2x2 …and for this right- angled triangle a 2 =x 2 +m 2 β

ECMWF User Meeting June 2006 y2y2 β m y a 2 -b 2 =x 2 -y 2 β a2a2 b2 x2x2

ECMWF User Meeting June 2006 A B β y2 x2x2 β b a 90º

ECMWF User Meeting June 2006 A B β y2 x2x2 β b a

ECMWF User Meeting June 2006 A B β y2 x2x2 β b a bcosβ acosβ

ECMWF User Meeting June 2006 abcos β b2b2 β (x+y) 2 a 2 The Cosine Theorem: (x+y) 2 = a 2 + b 2 - 2abcosβ

ECMWF User Meeting June 2006 a b x y β The Cosine Theorem: (x+y) 2 = a 2 + b 2 - 2abcosβ combined with a 2 - b 2 = x 2 - y 2 yields m

ECMWF User Meeting June 2006 The Cosine Theorem: (x+y) 2 = a 2 + b 2 - 2abcosβ combined with a 2 - b 2 = x 2 - y 2 yields a 2 -b 2 =x 2 -y 2 =(x+y) 2 -2y 2 -2xy a 2 -b 2 =a 2 +b 2 -2abcos(β) -2y 2 -2xy b 2 -a 2 =y 2 -x 2 =(x+y) 2 -2x 2 -2xy b 2 -a 2 =a 2 +b 2 -2abcos(β) -2x 2 -2xy 2b 2 =2abcos(β)+2y 2 +2xy b 2 -abcos(β)=y(x+y) 2a 2 =2abcos(β)+2x 2 +2xy a 2 -abcos(β)=x(x+y)

ECMWF User Meeting June 2006 E1E1 E2E2 a f g x y m cor(E1,E2) …and by replacing a and b with E 1 and E 2, the errors of the two systems and cos(β) with the correlation between E 1 and E 2 we have:

ECMWF User Meeting June 2006 E22E22 a f g y x E12E12 …which for uncorrelated errors boils down to …or the more familiar

ECMWF User Meeting June 2006 Hypothetical error correlations 50% 100% 0% D+0D+15 T799 vs T399 T799(T399) vs UKMO or an arbitrary eps-member If all three models have the same error magnitude and correlation then the weights are 33.3% But if the errors of T799 and T399 are more correlated than the errors of T799 (T399) versus UKMO the UKMO should be weighted the most Extension to three models??

ECMWF User Meeting June Future challenges Extending the monthly forecasts by including precipitation and provide forecasts separately form week1, week2 and week34 - and much more….