1. Evaluation of a Challenging Warm Season QPF month at HPC: June 2009 Brendon Rubin-Oster Richard Otto (with contributions from Mike Bodner, Keith Brill, David Novak and HPC management) June 3, 2010 1

2. Motivation For This Review HPC 1” Threat Score was for June 2009 Lowest since 1999 (0.119) FY 2009 goal is 0.290 NAM threat score of 0.111 is lowest since at least 1998 GFS threat score of 0.092 tied its lowest in 10 years (1999) HPC % Improvement over models: 18% NAM, 43% GFS, 16% ECMWF 0.131 2

3. (2009) 3

4. Seek To Answer the Following Was there a common theme associated with the busted forecasts? (MCS activity, scattered convection, synoptic scale forcing, mesoscale boundaries, etc.) What kinds of errors were observed? (timing/duration, placement, magnitude, etc.) How did the models perform? What (if anything) was so different about June 2009? 4 What were the causes of these errors?

5. Data & Methodology Selected model data archived at HPC for later review deterministic (NAM, GFS, ECMWF, CMC, UKMET) ensembles (SREF, CMCE, ECENS) 2 km base reflectivity radar composite archive (NMAPRAD) VIS/IR satellite & analysis data from internet (SPC archives, UCAR case studies) NCEP/NCAR reanalysis for monthly means Limitations on available data Surface: winds, moisture convergence, T, T d, θ e, moisture transport, 1000 mb frontogenesis, mixing ratio Thickness: 1000-850 mb & 850-700 mb Instability: CAPE, lifted indices Miscellaneous: Q-vectors, Corfidi vectors And others… 5

6. 6

7. What (if anything) was so different about June 2009? 7

8. Source: http://www.esrl.noaa.gov/psd/data/composites/day/ 500mb Geopotential Height means (m) 8

9. 250 mb mean zonal winds (m/s): June 1-30 Source: http://www.esrl.noaa.gov/psd/data/composites/day/ 9

10. Observed Precipitation (in) (June 1-30) Source: http://water.weather.gov/ 10

11. Percent of Normal Precipitation (June 1-30) Source: http://water.weather.gov/ 11

12. Was there a common theme associated with the busted forecasts?

13. Defining Modes of Convection ● Mesoscale Convective System (MCS):  organized cluster of thunderstorms  resides significant distance from synoptic boundaries  persists for at least 3 to 6 hours

14. Defining Modes of Convection ● Convection along a synoptic scale boundary:  thunderstorms developing within 150 km of a synoptic boundary

15. Defining Modes of Convection ● Convection along a mesoscale boundary:  thunderstorms developing within 150 km of a mesoscale surface boundary  consists of: sea breezes, surface trofs, outflow boundaries

16. Defining Modes of Convection ● Scattered convection:  thunderstorms not appearing to be associated with synoptic/mesoscale boundaries  does not meet MCS guidelines

17. Was there a common theme associated with the busted forecasts? 50% 22% 7% 14% 7%

18. What kinds of errors were observed? 29% 43% 7% 21%

19. What were the causes of these errors? 14% 22% 14% 22% 14%

20. How did the models perform during these 14 cases? 20 RankPerformerThreat Score 1NAM0.042 2HPC0.033 3GFS0.027 4ECMWF0.026 RankPerformerBias 1NAM1.353 2ECMWF1.462 3GFS1.932 4HPC2.810 Source: http://www2.hpc.ncep.noaa.gov/npvu/

21. 45mm / 1.78” PW June 26, 2009 case study

22. OBSERVEDNAM f036AUTO f036 HPC f036GFS f036EC f036

23. 24-hr radar loop {June 25 (1200Z)- June 26 (1200Z)}

24. Satellite loop June 25 (1415Z-2315Z)

25. Final Notes / Future Work? Explore how atypical the June 2009 observed phenomena and forecast performance were relative to other June cases HPC is currently applying MODE (method for object-based diagnostic evaluation) in real-time Use object oriented verification to objectively quantify errors Compare performance of high resolution mesoscale models