Forecasting of Atlantic Tropical Cyclones Using a Kilo-Member Ensemble M.S. Defense Jonathan Vigh

Acknowledgements Graduate Adviser: Dr. Wayne Schubert Master’s Committee Dr. Mark DeMaria Dr. William Gray Dr. Gerald Taylor Dr. Scott Fulton (MUDBAR) Schubert Research Group Data Sources: NCEP and TPC/NHC Mary Haley and NCL Developers Funding: Fellowship Support from Significant Opportunities in Atmospheric Research and Science Program (UCAR/NSF) and the American Meteorological Society NSF Grant ATM , NSF Grant ATM , NASA/CAMEX Grant NAG , and NOAA Grant NA17RJ1228

Outline The Big Picture Background The MUDBAR Model Design of a Kilo-Member Ensemble Postprocessing and Verification Results Case Studies Conclusions

Why study track? Major improvements in official track errors 72-h Official Track Forecast Errors -1.9% per year from % per year from Societal vulnerability increasing faster (e.g. Mitch, evacuation times) Even with accurate forecasts of intensity, wind field, rain – all for naught if the track is wrong

It’s Chaos Out There! The idea behind a forecast Perfect models and perfect initializations The nefarious atmosphere Error saturation and predictability limits Much of the track errors come from the major forecast errors of storms that follow erratic tracks Would be good to know in advance before large errors occur

Predictability Limits for a Barotropic Model (Leslie et al. 1998) (nm) Inherent Practical

Ensemble Background Definition: Any set of forecasts that verify at the same time. Idea is to simulate the sources of uncertainty present in the forecast problem Uncertainty in the initial state Uncertainty in the model Theory dictates that the mean forecast of a well- perturbed ensemble should perform better than any comparable single deterministic forecast

Types of Ensembles Monte Carlo simulations Lagged-average Forecasting Multimodel Consensus (Poor Man’s Ensemble) Dynamically constrained methods: Breeding of Growing Modes Singular Vector Decomposition

Questions and the thesis: Can a well-perturbed ensemble mean give a better forecast than any single realization? How many ensemble members are necessary to give the “right” answer? Is there a relationship between ensemble spread and forecast error? Can this relationship be used to provide meaningful forecasts of forecast skill? How accurately does the ensemble envelope of all track possibilities encompass the actual observed track?

The MUDBAR Model The nondivergent modified barotropic equation model (MUDBAR) of Scott Fulton Data enter the model through the initial condition (specify q) and the time- dependent boundary conditions (specify ψ on boundary, q on inflow)

Model Setup (Vigh et al. 2003) 6000-km square domain Optimized 3 grid configuration, 32 x 32 grid points Mesh spacing: 194, 97, and 48 km Each 120-h forecast takes 1.4 s on a 1 GHz PC (entire ensemble runs in ~1 h) Is able to reproduce the accuracy of the shallow water LBAR model

Bogussing Procedure The vortex profile of DeMaria (1987); Chan and Williams (1987): This bogus vortex is blended with the GFS initial wind field at the operationally-estimated storm position with the appropriate motion vector:

Ensemble Design Simple parameter-based perturbation methodology (fixed) Number and magnitudes of perturbations in each class chosen based on sensitivity experiments Five perturbations classes: 11 environmental perturbations (NCEP GFS ensemble) 1 control forecast 10 perturbed forecasts 4 perturbations to the depth of the layer-mean averaging of the wind very deep layer mean (1000 hPa – 100 hPa) standard deep layer mean (850 hPa – 200 hPa) Moderate depth layer mean (850 hPa – 350 hPa) Shallow depth layer mean(850 hPa – 500 hPa)

Ensemble Design, cont’d 3 perturbations to the model’s equivalent phase speed 300 m/sappropriate for Subtropical Highs 150 m/smiddle of the road 50 m/sappropriate for convective systems 3 perturbations to the bogus vortex size (V m ) V m = 15 m/ssmall vortex V m = 30 m/smedium-size vortex V m = 50 m/slarge vortex 5 perturbations to the storm motion vector All perturbations are cross multiplied to get an ensemble of: 11 x 4 x 3 x 3 x 5 = 1980 members! The Kilo-Ensemble

Postprocessing 1980 individual member forecasts – what to do now? Total ensemble mean (ZTOT), spread 20% cutoff used Subensemble means (for each perturbation), spread Calculation of spatial strike probabilities Value of probabilistic forecasting: Probabilities don’t hedge  The high tomorrow will be Capture the entire essence of the ensemble forecast

Verification Murphy (1993) talks about 3 types of ‘goodness’ for forecasts Consistency Quality Value Job of verification is to measure goodness Measures-oriented methods Distribution-oriented methods

Verification Procedures 293 cases from roughly 50 storms from the Atlantic Hurricane Seasons Only tropical and subtropical cases included All seasonal statistics are homogeneous Statistics calculated for the total ensemble mean and subensemble mean track forecasts: Mean track error x-bias y-bias Skill relative to CLIPER Frequency of superior performance

Other measures of ensemble performance Reliability of the ensemble envelope The outer envelope (0%) contained the retained the verification 80% of the time at 72-h, and 66% at 120-h Reliability of the spatial probabilities Spread vs. error relationship Large spread -> large error Small spread -> small error

Conclusions Ensemble mean forecast did not outperform the control forecast Ensemble strike probabilities seem within the realm of reality (reliability plot) Weak relationship between spread and error peaks at 60-h -> can estimate forecast skill Validity of barotropic model decreases at around 84- h, just as the benefits of the GFS environmental perturbations start to kick in

Questions: Can a well-perturbed ensemble mean give a better forecast than any single realization? How many ensemble members are necessary to give the “right” answer? Is there a relationship between ensemble spread and forecast error? Can this relationship be used to provide meaningful forecasts of forecast skill? How accurately does the ensemble envelope of all track possibilities encompass the actual observed track?

Possible reasons for performance degradation Reasons for poor ensemble performance: Barotropic dynamics are too simple Artificial edge biases Poor design – fixed perturbations not too good Spurious binary interactions between bogus vortex and GFS-analyzed vortex

Future Work Immediate future work (before Miami) Verify the strike probabilities using the Brier and the ROC scores Calculate a 26-member ensemble from just the 26 perturbations (without cross multiplication) Derive and verify cluster analysis forecasts Determine extent and effect of the binary interactions

Future Work (cont’d) Select an optimal subensemble for the particular forecast situation (error recycling) Redesign the ensemble to use relative perturbations Compare to other ensembles for track forecasting (GFS, GUNA, ECMWS, etc.)

Questions