Northeast River Forecast Center Taunton, MA National Oceanic and Atmospheric Administration’s National Weather Service Hydrologic Ensemble Forecast Service (HEFS) Revisited Erick Boehmler Northeast River Forecast Center Taunton, MA National Oceanic and Atmospheric Administration’s National Weather Service

National Oceanic and Atmospheric Administration’s National Weather Service 2 Northeast River Forecast Center Taunton, MA HEFS Revisited HEFS Objective Meteorological Ensemble Forecast Processor (MEFP) >Capabilities >Methodology –Parameter estimation –Schaake Shuffle (Clark et al., 2004) Hydrologic Ensemble Processor Ensemble Postprocessor Ensemble Verification Service –Validation results Short – Long-range Products

National Oceanic and Atmospheric Administration’s National Weather Service 3 Northeast River Forecast Center Taunton, MA HEFS Objective Improve NWS hydrologic services FeatureESPHEFS Forecast time horizon Weeks to seasonsHours to years, depending on the input forecasts Input forecasts (“forcing”) Historical climate data with some variations between RFCs Short-, medium- and long- range weather forecasts Uncertainty modeling Climate-based. No accounting for hydrologic uncertainty or bias. Suitable for long-range forecasting only Captures total uncertainty and corrects for biases in forcing and flow at all forecast lead times ProductsLimited number of graphical products (focused on long- range) and verification A wide array of data and user- tailored products are planned, including standard verification

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD HEFS Purpose Quantify forecast uncertainty for Short-range (hours to days) – Flood watch and warning program. – Local emergency management. – Flood control system management. – Reservoir management. Medium-range (days to weeks) – Local emergency management preparedness. – Reservoir management. – Snowmelt runoff management. Long-range (weeks to months) – Water supply planning. – Reservoir management.

National Oceanic and Atmospheric Administration’s National Weather Service 5 Northeast River Forecast Center Taunton, MA HEFS Components

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD Meteorological Ensemble Forecast Processor Capabilities Forecast variables: precipitation and temperature. Forecast temporal horizon: up to about a year. Forecast spatial scale: basin. Forecast sources: – WPC/RFC single-valued forecasts > QPF for days 1-5 and QTF 1-7 – GEFS (days 1 -15) – CFSv2 (days1- 270). – Climatology (1 day – 1 year). Ensemble quality: bias-corrected. Multiple temporal scales: 6 hours to 3 months to capture forecast skill at various temporal scales.

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD MEFP Capabilities (Continued) Seasonality: accounted for by using moving window of user- specified size to pool data points in calibration. Temperature diurnal cycle: accounted for through equations relating 6-hr values and daily max and min values. Space-time coherence: preserved among upstream and downstream basins in forecast ensembles using Schaake Shuffle. Ensemble blending: Correlation-based. Ensemble forecasts are generated iteratively for all time scales and forecast sources from low correlation to high correlation. Operation modes: forecasting and hindcasting. Diagnostic tools: MEFPPE, GraphGen.

National Oceanic and Atmospheric Administration’s National Weather Service 8 Northeast River Forecast Center Taunton, MA MEFP Component Function MEFP Correct forcing bias Merge in time Downscale (basin) WPC (2-day Planned) GEFS (Day 1 -15) CFSv2 (Days 16 – 270) Climatology (Days ) NWS and external user applications Parameter Estimation MEFPPE Forecast Ensembles

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD MEFP General Methodology Objective: Produce reliable ensemble forcing variables that capture the skill and quantify the uncertainty in the source forecasts. Key Idea: Condition the joint distribution of single-valued forecasts and the corresponding observations using the forecast. Select source forecasts from multiple models to cover short- to long-range. Define durations useful for MEFP application. Use a common modeling framework (the meta-Gaussian model) for both precipitation and temperature.

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD MEFP General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation

National Oceanic and Atmospheric Administration’s National Weather Service 11 Northeast River Forecast Center Taunton, MA HEFS Components

National Oceanic and Atmospheric Administration’s National Weather Service 12 Northeast River Forecast Center Taunton, MA Ensemble Verification Service Supports verification of HEFS including for precipitation, temperature and streamflow Verification of all forecasts or subsets based on prescribed conditions (e.g. seasons, thresholds, aggregations) Provides a wide range of verification metrics, including measures of bias and skill Requires a long archive of forecasts or hindcasts GUI or command-line operation

National Oceanic and Atmospheric Administration’s National Weather Service 13 Northeast River Forecast Center Taunton, MA Forecast quality: validation results MEFP forcing Skill of the MEFP with GEFS forcing inputs Positive values mean fractional gain vs. climatology (e.g. 50% better on day 1 at FTSC1) MEFP temperature generally skillful, even after 14 days MEFP precipitation skillful during first week, but skill varies between basins Forecast lead time (days) Skill (fractional gain over climatology) “50% better than climatology”

National Oceanic and Atmospheric Administration’s National Weather Service 14 Northeast River Forecast Center Taunton, MA WALN6 (MARFC) Forecast quality: validation results CFSv2 GEFS Long-range forecasts Example of MEFP precipitation forecasts from Walton, NY Beyond one week of GEFS, there is little skill vs. climatology In other words, the CFSv2 adds little skill for the long-range (but forcing skill may last >2 weeks in flow) If climate models improve in future, HEFS can be updated Forecast lead time (days) Skill (fractional gain over climatology) MEFP precipitation forecast Walton, NY CLIM No skill after ~one week

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD HEFS Product Examples AHPS short-range probabilistic product

National Oceanic and Atmospheric Administration’s National Weather Service 16 Northeast River Forecast Center Taunton, MA HEFS Product Examples AHPS medium-range probabilistic products

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD Managing NYC water supply Croton; Catskill; and Delaware Includes 19 reservoirs, 3 lakes; 2000 square miles Serves 9 million people (50% of NY State population) Delivers 1.1 billion gallons/day Operational Support Tool (OST) to optimize infrastructure, and avoid unnecessary ($10B+) water filtration costs HEFS forecasts are central to OST. The OST program has cost NYC under $10M An early application of long-range HEFS forecasts

National Oceanic and Atmospheric Administration’s National Weather Service 18 Northeast River Forecast Center Taunton, MA Summary and conclusions Ensemble forecasts are the future Forecasts incomplete unless uncertainty captured Ensemble forecasts are becoming standard practice HEFS implementation, products, and validation is ongoing and expanding Initial validation results are promising HEFS will evolve and improve Science and software will improve through feedback Guidance will improve through experience We are looking forward to supporting end users!

National Oceanic and Atmospheric Administration’s National Weather Service 19 Northeast River Forecast Center Taunton, MA References Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., Wilby, R., The Schaake Shuffle: a method for reconstructing space–time variability in forecasted precipitation and temperature fields. Journal of Hydrometeorology 5 (1), 243–262. Demargne, J., Wu, L., Regonda, S.K., Brown, J.D., Lee, H., He, M., Seo, D.-J., Hartman, R., Herr, H.D., Fresch, M., Schaake, J. and Zhu, Y. (2014) The Science of NOAA's Operational Hydrologic Ensemble Forecast Service. Bulletin of the American Meteorological Society, 95, 79–98. Brown, J.D. (2014) Verification of temperature, precipitation and streamflow forecasts from the Hydrologic Ensemble Forecast Service (HEFS) of the U.S. National Weather Service: an evaluation of the medium-range forecasts with forcing inputs from NCEP's Global Ensemble Forecast System (GEFS) and a comparison to the frozen version of NCEP's Global Forecast System (GFS). Technical Report prepared by Hydrologic Solutions Limited for the U.S. National Weather Service, Office of Hydrologic Development, 139pp. Brown, J.D. (2013) Verification of long-range temperature, precipitation and streamflow forecasts from the Hydrologic Ensemble Forecast Service (HEFS) of the U.S. National Weather Service. Technical Report prepared by Hydrologic Solutions Limited for the U.S. National Weather Service, Office of Hydrologic Development, 128pp.

National Oceanic and Atmospheric Administration’s National Weather Service 20 Northeast River Forecast Center Taunton, MA

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD Model the Forecast / Observed Joint Distribution X Y Forecast Observed 0 Forecast Observed Joint distribution Model Space Joint distribution Sample Space PDF of Observed PDF of STD Normal PDF of Forecast NQT PDF of STD Normal X Y NQT Correlation(X,Y)

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the selected duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD Sample Conditional Joint Distribution 3.23 Forecast Observed Joint distribution Model Space x fcst X Y Obtain conditional distribution given a single-value forecast x fcst xixi xnxn Conditional distribution given x fcst Ensemble forecast Probability 0 1 … x1x1 xnxn x1x1 Ensemble members

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the selected duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation

National Oceanic and Atmospheric Administration’s National Weather Service Office of Hydrologic Development Silver Spring, MD Blend Ensembles with Schaake Shuffle 3.25 Ensemble members Blended ensemble members