1. A.Montani; The COSMO-LEPS system. Present status and future plans of the COSMO-LEPS system Andrea Montani, D. Cesari, T. Diomede, C. Marsigli, T. Paccagnella ARPA-SIMC HydroMeteoClimate Regional Service of Emilia-Romagna, Bologna, Italy COSMO Genreal meeting Moscow

2. A.Montani; The COSMO-LEPS system. Outline Introduction:  present status of COSMO-LEPS;  migration to the 7-km system. Performance of the system:  time-series verification of COSMO-LEPS using SYNOP; Present activity:  COSMO-LEPS for TIGGE-LAM;  multi-model clustering. Future plans. Many regards from ECMWF Seminars and greetings for a good meeting to all WG4 members!!!

3. A.Montani; The COSMO-LEPS system. COSMO-LEPS (developed at ARPA-SIMC) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale Modelling, including Germany, Greece, Italy, Poland, Romania, Russia, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of high- impact and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)  generation of COSMO-LEPS to improve the forecast of high-impact weather in the short and early-medium range (up to fc+132h)

4. A.Montani; The COSMO-LEPS system. Dim 2 Initial conditions Dim 1 Dim 2 Possible evolution scenarios Dim 1 Initial conditions ensemble size reduction Cluster members chosen as representative members (RMs) LAM integrations driven by RMs LAM scenario COSMO-LEPS methodology

5. A.Montani; The COSMO-LEPS system. COSMO-LEPS suite @ ECMWF: present status d-1 dd+5 d+1d+2 d+4d+3 older EPS younger EPS clustering period 00 12 Cluster Analysis and RM identification 4 variables Z U V Q 3 levels 500 700 850 hPa 2 time steps Cluster Analysis and RM identification European area Complete Linkage 16 Representative Members driving the 16 COSMO-model integrations (weighted according to the cluster populations) employing either Tiedtke or Kain-Fristch convection scheme (randomly choosen) + perturbations in turbulence scheme and in physical parameterisations COSMO- LEPS clustering area suite runs as a “time-critical application” managed by ARPA- SIMC; Δx ~ 7 km; 40 ML; fc+132h; COSM0 v4.12 since Jul09; computer time (14.0 million BU for 2010) provided by the COSMO partners which are ECMWF member states. COSMO- LEPS Integratio n Domain

6. A.Montani; The COSMO-LEPS system. Upgrades during the “COSMO year” 16 November 2009: – archive of large-scale precipitation (62.2) for both COSMO-LEPS members and LM-DET; 30 November 2009: – implementation of COSMO-LEPS at 7 km (new domain, new perturbations (in types and values), lsso=.true., lforest=.true. ); – upgrade of COSMO to model version 4.8; 12 July 2010: – upgrade of INT2LM to model version 1.12; – upgrade of COSMO to model version 4.12.

7. A.Montani; The COSMO-LEPS system. Outline Introduction:  migration to the 7-km system. COSMO-LEPS 10 km (old) COSMO-LEPS 7 km (new)

8. A.Montani; The COSMO-LEPS system. Old system  x = 10 km  z = 40 ML  t = 90 s ngp = 306x258x40 = 3.157.920 fcst range = 132h initial conditions: interpolated from EPS members perturbations: type of convection scheme; tur_len; pat_len. Implementation of COSMO-LEPS at 7 km New system (COSMO-LEPS_7)  x = 7 km  z = 40 ML  t = 60 s ngp = 511x415x40 = 8.482.600 fcst range = 132h initial conditions: interpolated from EPS members merged with surface and soil-layer fields produced at DWD for COSMO-EU perturbations: type of convection scheme; tur_len; pat_len; crsmin; rat_sea; rlam_heat. COSMO-LEPS_7 tested from May to November 2009 (no merging yet) - to improve the forecast of near-surface parameters - to keep an “advantage” vs ECMWF EPS (running at ̴ 25 km) Why?

9. A.Montani; The COSMO-LEPS system. COSMO-LEPS_10 (Old) vs COSMO-LEPS_7 (New) Deterministic verification of T2M ensemble mean  Variable: 2-metre temperature.  Period: from June to November 2009.  Forecast ranges: fc+6h, fc+12h, …, fc+132h.  Scores: root-mean-square error, bias. Probabilistic verification of 12-hour cumulated precipitation  Variable:12h cumulated precipitation (18-06, 06-18 UTC).  Period: from June to November 2009.  Forecast ranges: fc 6-18h, fc 18-30h, …, fc 114-126h.  Scores: ROC area, BSS, RPSS, Outliers.  Thresholds: 1, 5, 10, 15, 25, 50 mm/12h.  Observations: SYNOP reports over either MAP D-PHASE region (450 reports/day) or the FULL-DOMAIN (1400 reports/day).  Method: nearest grid point; no-weighted fcst.

10. A.Montani; The COSMO-LEPS system. Bias and rmse of T2M Ensemble Mean  Consider bias and rmse for 3 months (24/5  24/8/2009) over MAPDOM ( ∼ 450 synop).  T2m forecasts are corrected with height.  Bias closer to zero and lower rmse for the 7-km suite.  Improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification.  Similar results over MAPDOM and over FULLDOM (not shown).  The signal is stable (same scores also for 6-month verification). ---- OLD rmse (10 km) ---- NEW rmse (7 km) —— OLD bias (10 km) —— NEW bias (7 km)

11. A.Montani; The COSMO-LEPS system. ROC area, BSS for 12-hour tp  Consider the event “10 mm of precipitation in 12 hours” for ROC area and BSS (from Jun to Nov 2009).  Better results for the 7-km suite, both for ROC area and BSS values.  The impact is more evident for BSS.  Reduction of 12-h cycle in 7-km runs.  The improvement is detectable for all forecast ranges and for both MAPDOM and FULLDOM. ROC area (both FULLDOM and MAPDOM)BSS (both FULLDOM and MAPDOM)

12. A.Montani; The COSMO-LEPS system. RPSS, OUTL for 12-hour tp  Consider scores not-dependent on one single threshold (from Jun to Nov 2009).  Better results for the 7-km suite in terms of RPSS.  The improvement is detectable for all forecast ranges and for both MAPDOM and FULLDOM.  The Percentage of outliers is only slightly reduced in the 7-km suite (solid lines), but the gap is very small.  The 7-km system has a positive impact in the reduction of the outliers BELOW THE MINIMUM for the MAPDOM (the same holds for FULLDOM, although not shown). % of outliers (only MAPDOM)RPSS (both FULLDOM and MAPDOM) COSMO-LEPS_7 implemented operationally on 1 December 2009

13. A.Montani; The COSMO-LEPS system. Outline Performance of the system:  time-series verification of COSMO-LEPS using SYNOP;

14. A.Montani; The COSMO-LEPS system. –SYNOP on the GTS Time-series verification of COSMO-LEPS Main features: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Dec 2002 to Jul 2010; PHASE region: 43-50N, 2-18E (MAP D-PHASE area); method: nearest grid point; no-weighted fcst; obs: synop reports (about 470 stations/day); fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h; thresholds: 1, 5, 10, 15, 25, 50 mm/12h; system: COSMO-LEPS; scores: ROC area, BSS, RPSS, Outliers, … both monthly and seasonal scores were computed

15. A.Montani; The COSMO-LEPS system. Time series of ROC area  Area under the curve in the HIT rate vs FAR diagram; the higher, the better …  Valuable forecast systems have ROC area values > 0.6.  Improvement of skill detectable for all thresholds along the years.  Poor performance of the system in Spring and Summer 2006 (both particularly dry), despite system upgrades.  Best performance in 2007 during DOP (D-PHASE Operation Period).  fc 30-42h: ROC area above 0.8 since mid-2007 and good scores in 2010.  fc 78-90h: ROC area ALSO above 0.8 in the last 10 months.

16. A.Montani; The COSMO-LEPS system. Seasonal scores of ROC area  Performance of the system assessed for the last 4 winters (DJF) and the last 5 springs (MAM).  Consider the “event” 10 mm/12h; valuable forecast systems have ROC area values > 0.6.  Need to take into account the different statistics for each season (e.g. MAM 2006 more rainy than the others).  Best performance for the last winter (with COSMO- LEPS at 7 km), for all forecast ranges.  As for spring, no particular impact of the new system. Scores better than in MAM 2009.  Faster decay of system performance in MAM.

17. A.Montani; The COSMO-LEPS system. Outliers: time series + seasonal scores  How many times the analysis is out of the forecast interval spanned by the ensemble members.  … the lower the better …  Performance of the system assessed as time series and for the last 4 winters.  Evident seasonal cycle (more outliers in winter).  Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010.  Need to take into account the different statistics for each season.  Best results for winter 2007-2008, with good performance of the new 7- km system.  Outliers before 10% from day 2 onwards.

18. A.Montani; The COSMO-LEPS system. Time series of Brier Skill Score  BSS is written as 1-BS/BS ref. Sample climate is the reference system. Useful forecast systems if BSS > 0.  BS measures the mean squared difference between forecast and observation in probability space.  BS equivalent to MSE for deterministic forecast.  The improvement of model performance is detectable for all thresholds along the years.  Fewer and fewer problems with high thresholds for shorter ranges.  Very good scores in 2010!!!.  fc 30-42h: BSS positive for all thresholds since April 2009.  fc 78-90h: good trend in 2010.

19. A.Montani; The COSMO-LEPS system. Ranked Probability Skill Score: time series + seasonal scores  A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.  Useful forecast systems, if RPSS > 0.  Performance of the system assessed as time series and for the last 4 winters (DJF).  the improvement of the system performance is detectable for all forecast ranges along the years;  RPSS positive for all forecast ranges since May 2008; encouraging trend in the last year.  Very good results for DJF 2009-2010 with COSMO-LEPS at 7 km!

20. A.Montani; The COSMO-LEPS system. Outline Present activity:  COSMO-LEPS for TIGGE-LAM;

21. A.Montani; The COSMO-LEPS system. Products: “high-priority” parameters (tp, lsp, t2m, td2m, u10, v10, gust10, mslp, orog, lsm) operationally generated for each ensemble member, from fc+0h to fc+132h every 3h, in GRIB2 format; produced at ECMWF; archived at ARPA-SIMC on the “native” rotated grid (0.0625 x 0.0625). COSMO-LEPS for TIGGE-LAM grib2 (TIGGE-LAM regular grid) grib1 (original rotated grid)

22. A.Montani; The COSMO-LEPS system. Main results Time-series verification scores. It is difficult to disentangle improvements related to COSMO-LEPS upgrades from those due to better EPS boundaries; nevertheless, positive trends can be identified :  increase in BSS and ROC area scores  reduction in outliers percentages;  system upgrades of Dec 2007 brought small but positive impact;  the increase in horizontal resolution had a clear positive impact last winter (also ECMWF EPS did well anyway …). Implementation of COSMO-LEPS_7km. The new system was tested in parallel suite for 6 months:  higher BSS and ROC area values for the probabilistic prediction of 12-h precipitation with respect to the operational one,  lower T2M errors of the ensemble mean,  positive impact of the introduction of the new perturbations. COSMO-LEPS_7km was implemented on 1 December 2009.

23. A.Montani; The COSMO-LEPS system. COSMO-LEPS_7km: –use the soil moisture analysis fields provided by DWD; –save COSMO-LEPS output files on model levels (up to fc+48h) for further downscaling; –test modifications of clustering methodologies: always select control runs by ECMWF EPS; consider shorter forecast ranges for clustering intervals (48-72h, 72-96h); –follow the outcome of ECMWF TAC-subgroup on BC project  possible modifications of the COSMO-LEPS suite; COSMO-LEPS for TIGGE-LAM: –develop coding, post-processing and archiving of COSMO-LEPS output files in GRIB2 format (test Fieldextra); –assistance to users. Future plans (1)

24. A.Montani; The COSMO-LEPS system. Future plans (2) Support calibration and verification. Carry on collaboration within research project (e.g. SAFEWIND, IMPRINTS). ECMWF Seminars 2011: tentative dates are 12-15 September 2011 ….. No more overlap with COSMO Meeting, please!

25. A.Montani; The COSMO-LEPS system. Thank you ! European Conference on Applied Climatology / EMS annual meeting 13 – 17 September 2010, Zurich (Switzerland) Session NWP2: Ensemble forecasting http://meetings.copernicus.org/ems2010

26. A.Montani; The COSMO-LEPS system. Operational set-up Core products:  16 perturbed COSMO-model runs (ICs and 3-hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC;  t = 132h; Additional products:  1 deterministic run (ICs and 3-hourly BCs from the high- resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC;  t = 132h;  1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC;  t = 36h.

27. A.Montani; The COSMO-LEPS system. Score dependence on the domain size (1)  Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom):  different statistics of the verification samples;  up to now, performance of the system over the 2 domains assessed only for 6 months (March- August 2007).  difficult to draw general conclusions

28. A.Montani; The COSMO-LEPS system. Score dependence on the domain size (2) RPSS  RPSS score… the higher the better… (and positive).  ROC area… the higher the better… (and above 0.6).  Smoother transitions from month to month in “fulldom” scores.  Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month.  Higher predictability with orographic forcing?  Need to check individual regions and/or to stratify for type of stations. OUTL ROC  Outliers percentage … the lower the better.

29. A.Montani; The COSMO-LEPS system. Bias and rmse of T2M Ensemble Mean  Consider bias (the closer to zero, the better) and rmse (the lower the better).  Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite.  The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification.  The signal is stable (similar scores for 1-month or 3-month verification).  Need to correct T2M forecasts with height to assess the impact more clearly.

30. A.Montani; The COSMO-LEPS system. The COSMO-LEPS suite @ ECMWF November 2002 – May 2004 d-1 dd+5 d+1d+2 d+4d+3 oldest EPS 12 middle EPS oldest EPS youngest EPS clustering period 00 12 Cluster Analysis and RM identification 4 variables Z U V Q 3 levels 500 700 850 hPa 2 time steps Cluster Analysis and RM identification European area Complete Linkage 5 Representative Members Driving the 5 COSMO-model integrations COSMO- LEPS Integratio n Domain COSMO- LEPS clustering area

31. A.Montani; The COSMO-LEPS system. The COSMO-LEPS suite @ ECMWF June 2004 – January 2006 d-1 dd+5 d+1d+2 d+4d+3 middle EPS youngest EPS clustering period 00 12 Cluster Analysis and RM identification 4 variables Z U V Q 3 levels 500 700 850 hPa 2 time steps Cluster Analysis and RM identification European area Complete Linkage COSMO- LEPS Integratio n Domain 10 Representative Members driving the 10 COSMO-model integrations employing either Tiedtke or Kain- Fristch scheme randomly choosen COSMO- LEPS clustering area Suite running in real time at ECMWF managed by ARPA-SIM; Δx ~ 10 km Fc length: 120h

32. A.Montani; The COSMO-LEPS system. COSMO-LEPS (developed at ARPA-SIM) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland). Why? Because the horizontal resolution of global-model ensemble systems is limited by computer time constraints and does not allow a detailed description of mesoscale and orographic-related processes. The forecast of heavy precipitation events can still be inaccurate (in terms of both locations and intensity) after the short range.

33. A.Montani; The COSMO-LEPS system. COSMO-LEPS project  combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of intense and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …) generation of COSMO-LEPS in order to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of the so-called “severe weather events”.

34. A.Montani; The COSMO-LEPS system. Semi-diurnal cycle in COSMO-LEPS scores  BSS score … the higher the better …  Performance of the system assessed for 5 different Summers (JJA). BSS  Evident 12-hour cycle in BSS scores (the same holds for RPSS, while less evident for ROC area scores).  Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).  The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range.  The bad performance in Summer 2006 is confirmed.