1. VERIFICATION Highligths by WG5

2. 2 Outlook The COSMO-Index COSI at DWD Time series of the index and its parts @ DWD starting @ 2003

3. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 3 Approach Application of a score like the UK-index - basic values Continuous elements Wind Skill score for wind and continuous elements related to persistence Skill score for categorical elements related to chance

4. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 4 Approach Application of a score like the UK-index - final score

5. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 5 Scores in MOVI and UKMO Continuous parameters: Reduction of variance RV = 1 – (RMSE prog / RMSE ref) 2 where ref = persistence Categorical parameters: ETS –ETS = (R – „chance“) / (T –“chance“) R= number of obs events correctly forecast T = number of events which were either observed or forecasted  global score S like COSMO-index COSI = S/S 0 x100

6. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 6 Parameters total cloud amount [threshold: 0-2, 3-6, 7-8 temperature [t2m, later: tmin, tmax] 10m- windvector precipitation [thresholds: 0.2, 2, 10 mm/6h]

7. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 7 Verification frequency Every 3h –T2m, 10m-wind and cloudiness: @ 00, 03,…, 18, 21 UTC later on: tmin & tmax over 12h 6h-sums: precipitation

8. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 8 COSI@DWD T 2m V 3.22 V3.19 COSMO-EU SSO DAY 1 DAY 2 DAY 3

9. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 9 COSI@DWD - Scores for day 1

10. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 10 COSI@DWD - Scores for day 2

11. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 11 COSI@DWD - Scores for day 3

12. U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009 12 COSI@DWD - Scores for different forecast times FS 06 12 18 24 30 36 42 48

13. Long period verification (seasonal trend) (from djf’04 to mam’09)  1.Statistical indices for low thres (0.2mm/24h) 2.Statistical indices for high thres (20mm/24h) Verification ovest last year (200806-200905)  1. Driving model comparison: ecmwf/Cosmo-I7/Cosmo-I2 2.Driving model comparison: ecmwf/Cosmo-ME/Cosmo-IT 3.Diurnal cycle for all the model

14. Seasonal trend (low thres) Bias reduction trend Seasonal cycle: big peak during summertime Biggest overestimation peak for cosmo-I7 Underestimation for cosmo-7 during latest seasons

15. Seasonal trend (low thres) Stable trend/slightly worsening in time Best performance during spring/summertime Worsening for Cosmo-7 during latest seasons

16. Seasonal trend (low thres) False alarm number reduction esp. in wintertime Worse performance during summertime, esp. for Cosmo-I7

17. Seasonal trend (low thres) Slightly improvement trend Seasonal cycle: better during moist seasons, worse during dry seasons

18. Seasonal trend (high thres) Bias reduction trend, at least during last year Seasonal cycle: big peak during spring-summertime (convective period)  seems to disappear during last summer (… why?) General good performance during last year Pronounced underestimation for Cosmo-7 during last seasons

19. Seasonal trend (high thres) Slightly improvement in time Worse performance during summertime (except 2007) Worsening for Cosmo-7 during last seasons

20. Seasonal trend (high thres) False alarm number reduction Worse performance during summertime

21. Seasonal trend (high thres) Slightly improvement trend Worse performance during summertime (except 2007)

22. Ecmwf vs Cosmo-I7 vs Cosmo-I2 Big gap between Ecmwf and Cosmo-model Cosmo-I7  slightly overestimation; Cosmo-I2  underestimation; Ecmwf  overest. for low thres/ underest. for high thres. I7 equivalent or slightly better then I2 (even if less false alarm for I2) 0.2 2 10 20 35 THRES. 0.2 2 10 20 35

23. Ecmwf vs Cosmo-I7 vs Cosmo-I2 fixed thres. - seasonal Big gap between Ecmwf and Cosmo-model Positive trend for both I7 and I2 I2  Underestimation tendency I7 is generally better JJA SON DJF MAM JJA SON DJF MAM

24. COSMOME vs ECMWF Temperature SON 2008 DJF 2008-9 MAM 2009 JJA 2009 0 2 3 0 2 3

25. COSMOME vs ECMWF Dew Point Temperature SON 2008 DJF 2008-9 MAM 2009 JJA 2009 0 2 3 0 2 3

26. COSMOME vs ECMWF – Wind Speed SON 2008 DJF 2008-9 MAM 2009 JJA 2009 0 1.5 2.5 0 1.5 2.5

27. COSMOME vs ECMWF – Precipitation SON - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 1 1 1

28. COSMOME vs ECMWF – Precipitation DJF - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 1 1 1

29. COSMOME vs ECMWF – Precipitation MAM - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 11 1

30. Big gap between Ecmwf and Cosmo-model IT and ME –> quite similar, it is difficult to decide the winner IT  tendency to overestimation Ecmwf vs Cosmo-Me vs Cosmo-IT 0.2 2 10 20 35 THRES. 0.2 2 10 20 35

31. Big gap between Ecmwf and Cosmo-model IT and ME –> quite similar, it is difficult to decide the winner IT  tendency to overestimation Ecmwf vs Cosmo-Me vs Cosmo-IT fixed thres. - seasonal JJA SON DJF MAM JJA SON DJF MAM

32. 2008-2009 diurnal cycle for all the versions Low thres High thres Bias overestimation peak during midday Spin-up problem for all the models especially COSMO-I7 and COSMO-I2 General worsening with forecast time The spin-up seems to disappear, underestimation during the first 6h Bias overestimation peak during midday COSMO-7 underestimates In general: slight improvement with respect to the previous year

33. To sum up(1) Long period verification (seasonal trend) (from djf’04 to mam’09)  1.General improvement trend 2.For low thres (rain/no rain): overestimation during spring/summertime with more probability of detection but also more false alarms 3.For high thres: the worse skills during spring/summertime (convective period) 4.General Cosmo-7 worsening during last year

34. To sum up (2) Verification over last year (200806-200905)  1.Good performance for Cosmo-EU 2.Similar performance Cosmo-I7/ Cosmo-ME (slightly better Cosmo- ME) 3. Cosmo-I7/ Cosmo-I2 comparison: similar skill, underestimation I2 4.Cosmo-ME/Cosmo-IT comparison: similar skill, overestimation IT 5.Diurnal cycle: bias overestimation peak during midday, in general slight improvement with respect to previous year

35. Some Conditional Verification using VERSUS -Verification of 2mT in Clear Sky Condition from the Model -Verification of 2mT in Clear Sky Condition from the Obs -Verification of 2mT in TCC Condition from the Model -Verification of 2mT in TCC Condition from the Obs Compared with “no condition” verification

36. Temperature CM SON 2008 MAM 2009 DJF2008-9 0 2 3 0 2 3

37. Conditional Verification Temper. with SC from Model SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3

38. Conditional Verification Temper. with SC from Obs SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3

39. Temperature CM SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3

40. Conditional Verification Temper. with TCC from Model SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3

41. Conditional Verification Temper. with TCC from Obs SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3

42. Temperature CM SON 2008 MAM 2009 DJF 2008-9 0 2 3 0 2 3