1. Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Report on Workshop « Stratified verification by weather classes and guidelines for forecasters » Geneva, 15-16 June 2010 Пьер Экерт

2. 2 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Goals of the workshop From classification to stratified verification Have in insight in classification methods and their relationship to the local weather, specially precipitation Look at (preliminary) stratified verification results Determine characteristics of classification methods useful for stratified verification

3. 3 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Goals of the workshop From stratified verification to guidelines Identify the kind of stratified verification useful to the forecasters Isolate the information which can be summarised in (text) guidelines Receive other requests from forecasters

4. 4 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Classification

5. 5 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Traditional manual classification for climate analysis (and verification), AWS is not suitable

6. 6 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller COST733 12 domains data: ERA40, SLP number of types fixed: 9, 18, 27  22 methods, 73 classifications (daily, 1958-2001) 12 domains data: ERA40, SLP number of types fixed: 9, 18, 27  22 methods, 73 classifications (daily, 1958-2001)

7. 7 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Classification methods A. Philipp et al., 2010, accepted pre-defined types 1.Definition of types 2.Assignent of cases (days) to these types ("deductive", "top down") derived types Both types and the allotment of days to these types are the result of one classification procedure ("inductive", "bottom up")

8. 8 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Classes and correlation to precipitation

9. 9 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Remaining 2 candidates pre-defined typesderived types subjectively def. types Rules/ Thresholds PCA (variants) Leader Algorithm Optimisation 3 3 6 22 55 GWT 10,18,26 PCACA 9,18,27 COST 733 catalogue candidates based on entirely diffent classification methods

10. 10 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Skill for precipitation (~Alps) PCACAGWT

11. 11 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller BSS Niederschläge (Winter) 26/27 WL18 WL9/10 WL GWT PCACA PERRET 31 WL SCHÜEPP 40 WL

12. COSMO Workshop on Stratified Verification and Guidelines Geneva 15-16 June 2010 Example of Application Mean Temperature January NEAADSWAAW

13. COSMO Workshop on Stratified Verification and Guidelines Geneva 15-16 June 2010 NEAAD - SWAAD Type NEAADType SWAAD

14. 14 COST 733 – Weather type classifications in Europe and at MeteoSwiss | COSMO workshop, Geneva, 25 June 2010 Reinhard Schiemann, Christoph Frei, Mark Liniger, Christof Appenzeller Conclusions w.r. to verification of models In COST733, the evaluation of the different weather types was done by looking on the ability to represent different precipitation patterns on the Alpine domain. For verification purposes we are more interested in differentiating weather classes where the models have difficulties from those where it performs well. Therefore, we should evaluate the weather type classifications again with the target measure "model skill" instead of precipitation. This can be done for a mean over the whole domain (RMSE), but also for spatial skill information (like e.g. the BIAS for each Pixel). Having the time series of both the weather type classification (one number per day) and the target quality measure (also 1 number a day) then we can calculate the BSS as was done in Schiemann and Frei (2009). We should do this only for the weather type classifications that were identified in the beauty contest (GWT, PCACA) and the modified GWT method. This evaluation would also help to interpret the results of stratified verification.

15. 15 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Stratified verification By season, by weather class or by type of event. Parameters  Temperature, dew point, precipitation, cloudiness, wind speed and gusts. Scores  ME, STD, POD, FAR, ETS, contingency tables, fuzzy scores or fields.

16. 16 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Presentation Class 1, summer a b c d e f g -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Stations / regions Bias Class 1, summer Synthetic map!

17. 17 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Recommendations for guidelines The guidelines should be self-contained. They can look like a cookbook, for instance for the use of a parameter or for the treatment of a specific situation. A light version can be at the disposal of the forecasted on duty (usually under time stress) when a longer version can be studied offline. This longer version can also be used as an education tool for newcomers. The shorter version can also be implemented as a seasonal factsheet. The seasonal factsheets should include (if possible) the expected changes of the current model version with respect to version which was running in the past season. Generally speaking the guidelines should be short, attractive and meaningful.

18. 18 Report on WG4 June 2010 workshop ¦ COSMO General Meeting September 2010 Pierre.Eckert[at]meteoswiss.ch Forecasters feedback Forecaster feedback should be organized either by a mailbox, a forum or regular discussions. At the end of each season a debriefing can be organized and a synthesis written. This can form a good base to the following corresponding season.