1. Edward Tollerud 1, Tara Jensen 2, John Halley Gotway 2, Huiling Yuan 1,3, Wally Clark 4, Ellen Sukovich 4, Paul Oldenburg 2, Randy Bullock 2, Gary Wick 4 (also Tressa Fowler, Barb Brown, Matt Pocernich, Paul Schultz, Chris Harrop, Isidora Jankov) 1 ESRL/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO 3 CIRES, Boulder, CO 4 ESRL/PSD, Boulder, CO Evaluation of QPF during the HMT-West Winter Exercise: A DTC/HMT Collaboration with USWRP Wally Clark http://esrl.noaa.gov/gsd/fab

2. USWRP/Test-bed Workshop 2010 HMT-DTC/USWRP Collaboration Goals and Objectives Implementation and demonstration of new verification capabilities for high-resolution NWP Develop DTC capabilities for ensemble model probabilistic prediction and verification Data Impact Studies Impact studies of model physics and parameterizations relevant to HMT research Collaborating Agencies and Motivations GSD: High-resolution ensemble NWP and verification PSD, USWRP: Atmospheric Rivers and QPF; Severe events NCAR/RAL: MET-based real-time and object–based verification; HWT interactions RFCs, WFOs, CA DWR: Operational reservoir management

3. Duality of Purpose: A Tale of Two Communities 1)Research for the USWRP, HMT, and etc. Very strong initial focus on QPF and extreme precipitation Important observational component What physical mechanisms lead to damaging rainfall? What verification tools can provide valuable diagnostic information? (eg. MODE vis a vis atmospheric rivers) 2) Operational Forecast Guidance for RFCs, CDWR, NWS, and etc. Baseline comparison with previous products (i.e, GFS forecasts) Establish usefulness of high-resolution WRF ensemble forecasts What is value of ensemble mean? What are limits to forecast lead times for QPF?

5. HMT Winter Exercise – Online Real-time Verification 1)MODE and standard scores 2)Selection of Vx data: gages, Stage IV, daily, 6h 3)Choice of format: valid time, initialization time, monthly summary, time series, bar plots, boxplots, variable thresholds, regional selections

6. Real-time QPF verification for HMT-West USWRP/Test-bed Workshop 2010 Initialized 12 UTC 1/17/2010, Verif. Stage IV, 0.1 in. threshold Good performance for ensemble mean (black ) and GFS (brown) Better performance overall for heavy rainfall periods http://verif.rap.ucar.edu/eval/hmt/2010/graphics/

7. 18 January 2010 19 January 2010 Wind turning: diurnal forcing or pure happenstance??

8. Constant Valid Time: 1200 UTC 01/20/2010 Spatial Frequency Bias,.01 inch threshold Full domain,.01 threshold Verified with 6h Stage IV Not large change with lead time except longest (4-5 day) Anomalously Large bias for ensemble mean Real-time QPF verification for HMT-West USWRP/Test-bed Workshop 2010

11. January Summary, ETS 24h forecast, full domain Verified at 24h gage sites Segregated by threshold Dramatic performance hit at higher thresholds No real ensemble member advantages in statistics but large intger-quartile range Real-time QPF verification for HMT-West

12. USWRP/Test-bed Workshop 2010 Real-time QPF verification for HMT-West January Summary, FAR 24h forecast, full domain Verified at gage sites Segregated by threshold Good Performance by Ensemble Mean, poor for GFS

13. PODY verified at points with 24h gage observations PODY verified at Stage IV grid points (6h)

14. Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Fields – 6h forecasts valid at 1200 UTC 20 January 2010

15. Ensemble Members, Ensemble Mean, and Observed (Stage IV) Precipitation Objects – 6h forecasts valid at 1200 UTC 20 January 2010

16. 72h48h24hSSM/I MODE Object Comparison of GFS Forecasts with SSM/I Observation for 25 February, 2004 (Clear Cut Case) Area=312 Area=369 Area=306Area=127 Wally Clark Object-based Atmospheric River Verification of Integrated Water Vapor http://esrl.noaa.gov/gsd/fab

17. Proposed Future Activities USWRP/Test-bed Workshop 2010 Expand and maintain real-time QPF verification demonstration system and analyze results (HMT-West and eventually HMT-East) Develop and assess prototype MODE-based spatial verification techniques to verify Eastern Pacific Atmospheric River events in near- real-time Addition of new ensemble-based probabilistic techniques to the MET, including spatial ensemble verification prototypes Investigate data impacts WRT verification data sets and initialization data

18. Prioritized Tasks: Details Adapt HWT-based probability scoring for HMT Incorporate GFS into skill scores for base-lining Add SREF to ensemble set Display Quantitative attributes from MODE Add wind verification for HMT-West Build 6h gage verification option Develop MODE-based ensemble verification utilities Include Variable duration option for verification periods

19. Prioritized Tasks: Details (Continued) Develop Credible Downscaling Applications for model Comparisons Extend Metviewer to Other Menu-Driven Procedures HMT-East Prototype Products Introduce Time-series Verification Options Publication of Winter Exercise Studies

20. Collaboration/Leverage with other DTC Tasks and Testbeds Probability and the HWT Verification of winds: QNSE Metviewer and Ensemble Processing for MET Ensemble Product Development, post-processing with DET HMT, ARs, and MET/MODE

21. Presentations and Publications Meeting/Workshop Presentations: BACIMO Workshop (Halley Gottway et al.) WRF User’s Conf. (Tollerud et al.; Halley Gottway et al.) Testbed Workshop (Tollerud et al.; Clark et al.) HMT West Tutorial (Jensen and Tollerud) Verification Colloquium (Tollerud et al.) Presentations planned for Future Weather Analysis and Forecasting; Hydrology QPF3 and GRAPES

22. Formal Publication Plans Diagnostic Results from a Real-time QPF Verification System for the HMT Winter Exercise Assessing the Performance of a High-resolution WRF ensemble Forecast System Relative to GFS Forecasts Object-based Verification for Historical Pacific Atmospheric Rivers Techniques for Spatial Verification using IWV and moisture Flux Objects QPF Verification Uncertainty and Other Impacts of Precipitation Dataset Choice