Verification of Tropical Cyclone Forecasts Beth Ebert (BOM) Barb Brown (NCAR) Laurie Wilson (RPN) Tony Eckel (ERT) 8th TIGGE Working Group Meeting 22-24 February 2010, Geneva

What kind of TC forecasts? Deterministic TC track Intensity maximum wind central pressure temporal trend (rapid intensification) Wind field / radii Precipitation Storm surge Temporal consistency Ensemble Track distribution strike prob., cone of uncertainty Intensity distribution mean / median spread 90th percentile Prob (wind > threshold) Prob (precip > threshold) Prob (surge > threshold) Temporal consistency

Different users need different kinds of verification information Public and emergency managers Simple, graphical Focus on impact Forecasters Information on how to interpret forecasts Timing errors Modellers Systematic errors How to improve the model How to improve ensemble distribution / spread

Quality of deterministic forecasts What are the track errors (along-track, cross-track)? What are the intensity errors? Are temporal intensity trends correctly predicted? What is the error in timing of landfall? What is the error in forecast maximum wind (rain)? Multi-day total precipitation Is the spatial distribution of wind (rain) correct?

Quality of ensemble forecasts Does the ensemble enclose the observed track? Are the ensemble probabilities skillful and reliable predictions for strike probability (track) intensity (max wind, central pressure) wind precipitation Does the ensemble produce an appropriate spread for these variables?

Data issues Verification data Forecast data Reference forecasts Data sources Best track, Dvorak, surface instruments, radar, … Problems measuring in extreme conditions Forecast data Size of ensemble Model grid resolution Details of cyclone tracker Reference forecasts Statistical forecast (e.g., CLIPER)

Verification methods for deterministic TC forecasts Example 1 – visual comparison Analysis Model Track/intensity verification Courtesy Noel Davidson, BOM

Verification methods for deterministic TC forecasts Example 2 – along-track and cross-track errors Courtesy James Franklin, NHC

Verification methods for deterministic TC forecasts Example 3 – cumulative distribution of track errors Courtesy James Franklin, NHC

Verification methods for deterministic TC forecasts Example 4 – distribution of intensity errors HFIP High-Resolution Hurricane Test – DTC Final Report Sept 2009

Verification methods for deterministic TC forecasts Example 5 – rapid intensification Count Hours since onset of observed RI Observed High resolution model Low resolution model AOML / WRF – 69 cases HFIP High-Resolution Hurricane Test – DTC Final Report Sept 2009

Verification methods for deterministic TC forecasts Example 6 – rain intensity distribution Marchok et al. 2008

Verification methods for ensemble TC forecasts Example 1 – visual comparison

Verification methods for ensemble TC forecasts Example 2 – probabilistic scores and methods MOGREPS 120 h forecast for strike probability (within 120 km) TC Anja, November 2009 fake ^

Verification methods for ensemble TC forecasts Example 3 – spread of track and intensity forecasts track track intensity intensity 20-member FIM ensemble

New approaches for verifying TCs Spatial verification methods Precipitation and wind fields Storm characteristics location size intensity shape, etc.

New approaches for verifying ensemble TC forecasts Minimum spanning tree multi-variate rank histogram Ensemble of object properties ensemble mean object properties distributions – use standard methods for ensembles and probability forecasts relationship of TC genesis to the number of ensemble members predicting the TC at day 1+? correspondence ratio observation ensemble forecast ensemble forecast

Reporting guidelines Provide all relevant information Model(s), grid, range of dates, forecast lead times, verification data source, etc. Aggregation and distribution of results Confidence intervals / uncertainty

Verification tools for TCs US Developmental Testbed Centre (DTC) Tracker (Marchok) Code for verification of TC track and intensity Other international tool sets??

Document on TC verification – commented literature review to be written this year Contents Introduction Verification strategy Reference data Verification methods Reporting guidelines Summary References Appendices: a. Brief description of scores b. Guidelines for computing aggregate statistics c. Confidence intervals for verification scores d. Examples of graphical verification products