Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss A more reliable COSMO-LEPS F. Fundel, A. Walser, M. A. Liniger, C. Appenzeller COSMO General Meeting,15. September 2008

2 COSMO General Meeting 2008 André Walser Outline Motivation Method Products Verification Sensitivity Studies Conclusion & Outlook

3 COSMO General Meeting 2008 André Walser Why calibrate? OBS-CLEPS/(OBS+CLEPS)/ LT: 42h Jan Jul Q0.8Q0.95 COSMO-LEPS is not reliable, Probabilities might be wrong! Need for calibration

4 COSMO General Meeting 2008 André Walser COSMO-LEPS reforecasts (v 4.0) Setup Period of 30 years ( ) Deterministic run of COSMO-LEPS (1 member) 90h lead time, 12:00 initial time (every 3rd day) ERA40 as initial/boundary conditions Calculated on hpce at ECMWF Archived on Mars Convective scheme = Tiedtke/Kain-Fritsch Random physics (turlen & patlen) Limitations New climatology needed with each model version change Needs time and is costly (present setup ca. 2 Mio SBU/y.)

5 COSMO General Meeting 2008 André Walser “However, the improved skill from calibration using large datasets is equivalent to the skill increases afforded by perhaps 5–10 yr of numerical modeling system development and model resolution increases.” (Wilks and Hamill, Mon. Wea. Rev. 2007) “Use of reforecasts improved probabilistic precipitation forecasts dramatically, aided the diagnosis of model biases, and provided enough forecast samples to answer some interesting questions about predictability in the forecast model.” (Hamill et. al, BAMS 2006) “…reforecast data sets may be particularly helpful in the improvement of probabilistic forecasts of the variables that are most directly relevant to many forecast users…” (Hamill and Whitaker, subm. to Mon. Wea. Rev 2006) „...large improvements in forecast skill and reliability are possible through the use of reforecasts, even with a modernized forecast model. (Hamill et al., subm. Mon Wea. Rev 2007)“ Reforecasts in literature

6 COSMO General Meeting 2008 André Walser What‘s new? Calibration of a modern high-resolution LEPS Return Periods don‘t require observations Calibration of the entire model domain (in principle every parameter)

7 COSMO General Meeting 2008 André Walser PDF CDF x „return period“ OBSMODFCST x „calibrated return periods“ „raw return periods“ x „return period“ Calibration strategy Quantiles w.r.t. observations are not reliable Quantiles w.r.t model climatology are reliable

8 COSMO General Meeting 2008 André Walser Return period based Warnings domain averages No alert 6 times a year Twice a year Every 10 years 65 target regions: RR time series apply warnlevels 3 types of alerts:

9 COSMO General Meeting 2008 André Walser Products Corresponding return level [mm/24h] Probabilities to exceed return period with respect to the current month:

10 COSMO General Meeting 2008 André Walser Products Probabilities to exceed return period with respect to the current month:

11 COSMO General Meeting 2008 André Walser Products

12 COSMO General Meeting 2008 André Walser Products Obs. quantiles Obs. COSMO-7 Calibrated CLEPS

13 COSMO General Meeting 2008 André Walser Obs. quantiles Actual Obs. COSMO-7 Calibrates CLEPS Products

14 COSMO General Meeting 2008 André Walser 24h Tot_Prec ( UTC) Domain Switzerland interp. on CLEPS Grid (C.Frei) Apr06 – Aug07 (verification) (calibration) Observation Data Verification model topography [m]

15 COSMO General Meeting 2008 André Walser Reliability Diagram: Verification

16 COSMO General Meeting 2008 André Walser raw forecast overconfident, very limited skill strong improvement in reliability long lead-time forecasts more reliable calibrated raw Winter 06/07 Q0.8 24h precipitation + ( 18-42h ) + (66-90h)

17 COSMO General Meeting 2008 André Walser raw forecast overconfident modest improvement of reliability long lead-time forecasts more reliable calibrated raw Summer 06 & 07 Q0.8 24h precipitation + ( 18-42h ) + (66-90h)

18 COSMO General Meeting 2008 André Walser winter summer calibrated raw BSSD 24h precipitation

19 COSMO General Meeting 2008 André Walser Q0.8 24h precipitation 4 days lead-time Winter 06/07 Summer 06&07 Economic value

20 COSMO General Meeting 2008 André Walser Q0.8 Q0.95 Day 1 Day 4 Raw forecast skill < 0 BSSD: Rel. skill improvement (Apr 06–Aug 07)

21 COSMO General Meeting 2008 André Walser 1971 ….… ….… ….… ….… ….….. available reforecasts used reforecasts 2000 actual forecast Sensitivity Study: Calibration Period

22 COSMO General Meeting 2008 André Walser Sensitivity Study: Calibration Period short subset possible for the calibration of frequent events larger subset suggested for the calibration of extreme events points to relatively weak seasonality of frequent events

23 COSMO General Meeting 2008 André Walser 1971 ….… ….… available reforecasts used reforecasts actual forecast Sensitivity Study: Reforecast length

24 COSMO General Meeting 2008 André Walser years seem to be sufficient to calibrate frequent events extreme events require a large set of reforecasts More than 2 years of reforecasts are needed for a better than uncalibrated forecast Sensitivity Study: Reforecast length

25 COSMO General Meeting 2008 André Walser Conclusions Calibrating with reforecasts improves the forecast skill significantly Most effective in winter Beneficial for all users (i.e. all C/L ratios) Calibrated warnings without using observations are possible Calibrating frequent precipitation events does not require a large calibration period However, calibrating extreme events does

26 COSMO General Meeting 2008 André Walser Outlook Optimal use of reforecasts Publications Calibration of wind gusts Operational implementation of Calibrating & plot generation at CSCS (time-critical) Reforecasts at ECMWF (non time-critical)

27 COSMO General Meeting 2008 André Walser

28 COSMO General Meeting 2008 André Walser Special case BSS: M: Ensemble size Weigel et al. 2006, Mon. Wea. Rev. The BSS debiased