1. Findings and Recommendations from the Hydrologic Verification System Requirements Team Peter Gabrielsen, Julie Demargne, Mary Mullusky, Kevin Werner, Bill Lawrence, Scott Lindsey, Donna Page, Noreen Schwein, Scott Staggs, Tom Adams, and Bill Marosi RFC Verification Workshop, August 14-16 2007

2. 2 Final Report available at http://www.nws.noaa.gov/oh/rfcdev/ docs/Final_Verification_Report.pdf

3. 3 Outline Forecast Verification at NWS: needs and vision River Forecast Verification System Recommendations vs. Current/Future Work –Resources for Verification –Verification Plan –Verification Activities

4. 4 Forecast Verification at NWS Needs: –1996: National Research Council stated verification of hydrologic forecasts was inadequate –2006 *: Board on Atmospheric Sciences and Climate recommended NWS to expand verification of its uncertainty products and make it easily available to all users in near real time Vision: –Develop a comprehensive national system to verify hydrologic forecasts and guidance products which satisfy the needs of all users –Improve forecast services by analyzing sources of error and skill across the entire forecast process –Provide easy access to enhanced river forecast verification data to improve our scientific and operational techniques and services * “Completing the Forecast, Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts”

5. 5 River Forecast Verification System Goals: –Quantify quality of RFC forecasts and quality of forecast services –Monitor forecast quality over time –Monitor quality at various steps in the forecast process –Improve forecast quality –Assist prioritization of forecast system enhancements Uses: –Operational –Experimental/Research Customers: –Hydrologic forecasters –Scientists/Researchers –Hydrologic program managers –Emergency and water resources managers –Public

6. 6 River Forecast Verification System Components –Logistical verification –Deterministic verification –Probabilistic verification Verification System Capabilities –Data Archiving (attributes for time, service, basin and events) –Computing Metrics –Displaying Metrics (graphics and reports) –Disseminating Metrics and Data (understand quality & usefulness of forecasts) –Real Time Access to Metrics (understand errors in recent forecasts and over long term) –Error Analysis (including hindcast experiments) –Performance Measure Tracking (level of success)

7. 7 Verification Resources Recommended resources –OHD Program Manager for verification –RFC Verification Focal Points at each RFC –National Team for developing and implementing standardized formats and procedures to archive data required for verification Current and near-future resources –National Verification Focal Points: Julie Demargne & Mary Mullusky + support from others –13 RFC Verification Focal Points –2 Hydrologic Forecast Verification Teams for WR and NWS

8. 8 Proposed Verification Plan Recommendations from FY07 up to FY11 Task 1. Archive 2. Compute Metrics 3. Display Metrics 4. Dissemination 5. Real Time Access 6. Error Analysis 7. Performance Tracking Task 1. Archive 2. Compute Metrics 3. Display Metrics 4. Dissemination 5. Real Time Access 6. Error Analysis 7. Performance Tracking Logistical Verification Deterministic Verification Probabilistic Verification Grid Verification

9. 9 AHPS FY07 Proposed Verification Activities Deterministic Verification: –Enhance & release IVP –Develop error analysis, raw model forecasts, & deterministic hindcaster –Define data requirements & hardware for verification & hindcaster Ensemble Verification: –Enhance & release EVS –Enhance & release Ensemble Hindcaster + Develop error analysis –Evaluate existing archiving capabilities Logistical Verification: –Develop plan for archiving info –Implement logistical measures

10. 10 Current Verification Activities Deterministic Verification: –Deterministic Verification Tools: IVP enhancements & release –Develop Error Analysis: development of raw model forecast application & deterministic hindcaster, define data requirements & hardware Ensemble Verification: –Improve Ensemble Verification: EVS enhancements & release, development of requirements for archiving capabilities –Improve Ensemble Hindcaster: EH enhancements & release, development of uncertainty analysis capabilities Logistical Verification: –Develop plan for archiving info –Implement logistical measures

11. 11 Future Verification Activities Verification System Requirements & Planning Team to define the AHPS FY08 activities and update verification plan AHPS activities to be continued: –Deterministic Verification: IVP, Error Analysis –Ensemble Verification: EVS, Error Analysis –Logistical Verification: Logistical Measures 2 Hydrologic Forecast Verification Teams for WR and NWS to develop verification strategies using case studies

12. 12 Thank you