Extracting probabilistic severe weather guidance from convection-allowing model forecasts Ryan Sobash 4 December 2009 Convection/NWP Seminar Series Ryan Sobash 4 December 2009 Convection/NWP Seminar Series

Identification of severe convection in high-resolution models Today’s state-of-the-art NWP models are run at resolutions capable of explicitly representing convection [< 4km]. Although these models do not explicitly predict the phenomena responsible for severe reports, the potential exists to produce guidance for these hazards if a relationship exists between the “intensity” of model convection and observed severe weather reports. If this relationship is robust, high-res model forecasts could be used to produce automated forecast guidance of convective hazards, just as traditional guidance is produced with environmental parameters.

Previous work – Identifying supercells Several studies have investigated the issue of mining output of operational models to identify severe convection, primarily supercells. Elmore et al. (2002) identified supercells (using a w-zeta correlation) in an operational high-res ensemble to provide guidance to forecasters on storm intensity/longevity. Shaded: UH > 25 m 2 s -2 Kain et al. (2008) identified rotating storms in real-time model output using an integrated helicity parameter (updraft helicity - UH) during Spring 2005 to obtain statistics on number/coverage of modeled supercells. Kain et al. (2008) preferred the UH parameter over a correlation parameter (supercell detection index - SDI). Both produced similar results. Kain et al. (2008)

Previous work – Identifying supercells The updraft helicity parameter is an measure of the vertical component of helicity associated with an updraft: The quantity is integrated between 2 km and 5 km AGL to identify rotating updrafts in the lower-to-middle troposphere. It is computed on the model grid using a mid-point approximation. Restricted to positive values (i.e. cyclonically rotating updrafts). Could potentially identify shear areas not associated with supercells?

Previous work – Spring Experiment 2008 During the 2008 NSSL/SPC Spring Experiment, severe convection was identified in a convection-allowing ensemble to produce guidance for forecasters. Along with UH, other fields associated with severe convection were identified in the model output. > large updraft helicity > strong low-level wind speed > moderately strong low-level winds co-located with linear reflectivity segments The grid points that met any of these criteria were flagged and accumulated over a 24 hour period (using hour model forecasts). These grid points are interpreted as the locations of “surrogate” severe weather reports.

Previous work – Spring Experiment 2008 A Gaussian smoother was applied following the procedure outlined in Brooks et. al. (1998) to produce a “practically perfect” Convective Outlook given a distribution reports. Subjective interpretations indicated this technique routinely captured the areas of severe reports.

Current Study An investigation of this guidance concept was undertaken. Although an ensemble was used during SE2008, a deterministic model run was used in this work. NSSL-WRF Configuration: > WRF-ARW V2.2 > Initialization time : 00 UTC > Forecast length : 36 hours > Horizontal resolution : 4 km > Physics : MYJ BL, WSM6 MP Domain

Current Study 5 fields were chosen to identify convection in the NSSL-WRF output: UH: updraft helicity (computed between 2 km and 5 km) [m2 s-2] UU: 10 m wind speed [m s-1] RF: 1 km AGL simulated reflectivity [dBz] UP: max. column updraft (below 400 hPa) [m s-1] DN: max. column downdraft (below 400 hPa) [m s-1] To capture intra-hourly convective-scale variations, the maximum value of each field within an hour was recorded. Grid points where a field exceeds a severe “threshold” were flagged and are referred to as surrogate severe reports. Focus on hour forecasts.

Current Study The thresholds were chosen subjectively during SE2008. To provide a more systematic examination, a range of thresholds were selected from each field’s frequency distribution during SE2008, near the original subjective thresholds. UH: 33 > 103 m 2 s -2 UU: 19 > 24 m s -1 RF: 53 > 56 dBz UP: 20 > 27 m s -1 DN: -5 > -7 m s -1 Specific thresholds based on percentiles of the distribution.

Current Study Each day’s surrogate reports are placed on an 80 km grid and smoothed using a Gaussian weighting function. Since the Gaussian weights sum to 1, the result can be interpreted as a probabilistic forecast. The final product is a surrogate severe probability forecast (SSPF). This post-processing technique was performed by Theis et. al. (2005) on precipitation forecasts to introduce a simple measure of spatial uncertainty into deterministic high-resolution forecasts.

Case: 29 May 2008 UH UURF DN UP OBS

Case: 29 May 2008 UH UURF DNUPOBS

Research Questions Does the SSPF provide skillful probabilistic guidance? Can the SSPF provide useful guidance to SPC forecasters? ? 1. What fields produce the most skillful guidance? 2. For a given field, which thresholds produce the most skillful guidance? 3. What forms of guidance are most appropriate? 4. How reliable are the probabilistic forecasts?

SSPF Verification ROC (relative operating characteristic) curves > assesses ability of forecasts to discriminate between different outcomes – is conditioned on observations > plot of probability of detection (POD) vs. probability of false detection (aka false alarm rate – FAR) > diagonal is POD=POFD (chance of forecasting event is equal to chance of forecasting a false alarm) > commonly summarized with ROC curve area Reliability diagrams > assesses ability of forecasts to produce reliable probabilities – is conditioned on the forecasts > ideally, want the forecasts of 30% to occur 30% of the time A skillful probabilistic forecasting system produces large ROC curve areas and is highly reliable. Images courtesy Austrailian Bureau of Meteorology Verification website

SSPF Verification SSPF-UH ROC curves for SE2008 ROC curve areas Decreasing threshold

SSPF Verification SSPF-UU ROC curves for SE2008 ROC curve areas Decreasing threshold

SSPF Verification SSPF-RF ROC curves for SE2008 ROC curve areas Decreasing threshold

SSPF Verification SSPF-UP ROC curves for SE2008 ROC curve areas Decreasing threshold

SSPF Verification SSPF-DN ROC curves for SE2008 ROC curve areas Decreasing threshold

SSPF Verification SSPF-UH Reliability diagram for SE2008 Increasing threshold No Skill Climatology

SSPF Verification SSPF-UU Reliability diagram for SE2008 Increasing threshold No Skill Climatology

SSPF Verification SSPF-RF Reliability diagram for SE2008 Increasing threshold No Skill Climatology

SSPF Verification SSPF-UP Reliability diagram for SE2008 Increasing threshold No Skill Climatology

SSPF Verification SSPF-DN Reliability diagram for SE2008 Increasing threshold No Skill Climatology

SSPF Verification Largest ROC curve areas using lowest thresholds for UH, UP, DN. ROC curve areas appear to asymptote approaching these thresholds. UH naturally reliable at mid-range (~50 m2s-2) thresholds, other fields tend to overforecast probabilities at all thresholds. Probabilistic forecasts which have large ROC curve areas, but are insufficiently reliable can be calibrated to improve reliability.

SSPF Verification SSPF forecasts were produced with constant sigma = 120 km. Can reliability of the forecasts be improved by changing this smoothing parameter? sigma = 120 kmsigma = 160 kmsigma = 240 km Too hot… Too cold… Just right?

SSPF Verification SSPF-UH Reliability for SE2008 sigma=80 kmsigma=160 km sigma=240 km Increase sigma

SSPF Verification SSPF-UU Reliability for SE2008 sigma=80 kmsigma=160 km sigma=240 km

SSPF Verification SSPF-RF Reliability for SE2008 sigma=80 kmsigma=160 km sigma=240 km

SSPF Verification SSPF-UP Reliability for SE2008 sigma=80 kmsigma=160 km sigma=240 km

SSPF Verification SSPF-DN Reliability for SE2008 sigma=80 kmsigma=160 km sigma=240 km

SSPF Verification Summary UH is the best predictor as indicated by ROC area, reliability diagrams. The best performing SSPF is UH with sigma ~ 200 km, using a threshold ~ 35 m 2 s -2. Calibration of other fields is more challenging… - Changes in sigma have desired effect for some forecast probabilities, but not for others. - Still have an overforecasting problem for UU, RF, UP, DN. UH is uniquely suited to identify severe convection. Increasing sigma decreases potential for higher probabilities. For rare-event forecasting, this may not be an issue.

Future work Work is underway to verify forecast over a 1.5 year period to determine if these findings are applicable under all seasons and regions. Discriminate between severe weather type. Applying the SSPF procedure to an ensemble of forecasts. Will help improve upon this proof-of-concept.

Research Questions Does the SSPF provide skillful probabilistic guidance? Yes. Can the SSPF provide useful guidance to SPC forecasters? Potentially. Additional calibration needed. Could be useful as a starting point in the analysis of model data. ? Day1 Surrogate Convective Outlook 29/1200Z – 30/1200Z Model: /00Z NSSL-WRF