1. Forecasting of Atlantic Tropical Cyclones Using a Kilo-Member Ensemble M.S. Defense Jonathan Vigh

2. Acknowledgements Graduate Adviser: Dr. Wayne Schubert Master’s Committee Dr. Mark DeMaria Dr. William Gray Dr. Gerald Taylor Dr. Scott Fulton (MUDBAR) Schubert Research Group Data Sources: NCEP and TPC/NHC Mary Haley and NCL Developers Funding: Fellowship Support from Significant Opportunities in Atmospheric Research and Science Program (UCAR/NSF) and the American Meteorological Society NSF Grant ATM-0087072, NSF Grant ATM-0332197, NASA/CAMEX Grant NAG5-11010, and NOAA Grant NA17RJ1228

3. Outline The Big Picture Background The MUDBAR Model Design of a Kilo-Member Ensemble Postprocessing and Verification Results Case Studies Conclusions

5. Why study track? Major improvements in official track errors 72-h Official Track Forecast Errors -1.9% per year from 1970-1998 -3.5% per year from 1994-1998 Societal vulnerability increasing faster (e.g. Mitch, evacuation times) Even with accurate forecasts of intensity, wind field, rain – all for naught if the track is wrong

6. It’s Chaos Out There! The idea behind a forecast Perfect models and perfect initializations The nefarious atmosphere Error saturation and predictability limits Much of the track errors come from the major forecast errors of storms that follow erratic tracks Would be good to know in advance before large errors occur

7. Predictability Limits for a Barotropic Model (Leslie et al. 1998) (nm)0244872 Inherent215280118 Practical4690138208

9. Ensemble Background Definition: Any set of forecasts that verify at the same time. Idea is to simulate the sources of uncertainty present in the forecast problem Uncertainty in the initial state Uncertainty in the model Theory dictates that the mean forecast of a well- perturbed ensemble should perform better than any comparable single deterministic forecast

11. Types of Ensembles Monte Carlo simulations Lagged-average Forecasting Multimodel Consensus (Poor Man’s Ensemble) Dynamically constrained methods: Breeding of Growing Modes Singular Vector Decomposition

12. Questions and the thesis: Can a well-perturbed ensemble mean give a better forecast than any single realization? How many ensemble members are necessary to give the “right” answer? Is there a relationship between ensemble spread and forecast error? Can this relationship be used to provide meaningful forecasts of forecast skill? How accurately does the ensemble envelope of all track possibilities encompass the actual observed track?

13. The MUDBAR Model The nondivergent modified barotropic equation model (MUDBAR) of Scott Fulton Data enter the model through the initial condition (specify q) and the time- dependent boundary conditions (specify ψ on boundary, q on inflow)

14. Model Setup (Vigh et al. 2003) 6000-km square domain Optimized 3 grid configuration, 32 x 32 grid points Mesh spacing: 194, 97, and 48 km Each 120-h forecast takes 1.4 s on a 1 GHz PC (entire ensemble runs in ~1 h) Is able to reproduce the accuracy of the shallow water LBAR model

15. Bogussing Procedure The vortex profile of DeMaria (1987); Chan and Williams (1987): This bogus vortex is blended with the GFS initial wind field at the operationally-estimated storm position with the appropriate motion vector:

16. Ensemble Design Simple parameter-based perturbation methodology (fixed) Number and magnitudes of perturbations in each class chosen based on sensitivity experiments Five perturbations classes: 11 environmental perturbations (NCEP GFS ensemble) 1 control forecast 10 perturbed forecasts 4 perturbations to the depth of the layer-mean averaging of the wind very deep layer mean (1000 hPa – 100 hPa) standard deep layer mean (850 hPa – 200 hPa) Moderate depth layer mean (850 hPa – 350 hPa) Shallow depth layer mean(850 hPa – 500 hPa)

17. Ensemble Design, cont’d 3 perturbations to the model’s equivalent phase speed 300 m/sappropriate for Subtropical Highs 150 m/smiddle of the road 50 m/sappropriate for convective systems 3 perturbations to the bogus vortex size (V m ) V m = 15 m/ssmall vortex V m = 30 m/smedium-size vortex V m = 50 m/slarge vortex 5 perturbations to the storm motion vector All perturbations are cross multiplied to get an ensemble of: 11 x 4 x 3 x 3 x 5 = 1980 members! The Kilo-Ensemble

20. Postprocessing 1980 individual member forecasts – what to do now? Total ensemble mean (ZTOT), spread 20% cutoff used Subensemble means (for each perturbation), spread Calculation of spatial strike probabilities Value of probabilistic forecasting: Probabilities don’t hedge  The high tomorrow will be 73... Capture the entire essence of the ensemble forecast

27. Verification Murphy (1993) talks about 3 types of ‘goodness’ for forecasts Consistency Quality Value Job of verification is to measure goodness Measures-oriented methods Distribution-oriented methods

28. Verification Procedures 293 cases from roughly 50 storms from the 2001- 2003 Atlantic Hurricane Seasons Only tropical and subtropical cases included All seasonal statistics are homogeneous Statistics calculated for the total ensemble mean and subensemble mean track forecasts: Mean track error x-bias y-bias Skill relative to CLIPER Frequency of superior performance

37. Other measures of ensemble performance Reliability of the ensemble envelope The outer envelope (0%) contained the retained the verification 80% of the time at 72-h, and 66% at 120-h Reliability of the spatial probabilities Spread vs. error relationship Large spread -> large error Small spread -> small error

58. Conclusions Ensemble mean forecast did not outperform the control forecast Ensemble strike probabilities seem within the realm of reality (reliability plot) Weak relationship between spread and error peaks at 60-h -> can estimate forecast skill Validity of barotropic model decreases at around 84- h, just as the benefits of the GFS environmental perturbations start to kick in

59. Questions: Can a well-perturbed ensemble mean give a better forecast than any single realization? How many ensemble members are necessary to give the “right” answer? Is there a relationship between ensemble spread and forecast error? Can this relationship be used to provide meaningful forecasts of forecast skill? How accurately does the ensemble envelope of all track possibilities encompass the actual observed track?

60. Possible reasons for performance degradation Reasons for poor ensemble performance: Barotropic dynamics are too simple Artificial edge biases Poor design – fixed perturbations not too good Spurious binary interactions between bogus vortex and GFS-analyzed vortex

61. Future Work Immediate future work (before Miami) Verify the strike probabilities using the Brier and the ROC scores Calculate a 26-member ensemble from just the 26 perturbations (without cross multiplication) Derive and verify cluster analysis forecasts Determine extent and effect of the binary interactions

62. Future Work (cont’d) Select an optimal subensemble for the particular forecast situation (error recycling) Redesign the ensemble to use relative perturbations Compare to other ensembles for track forecasting (GFS, GUNA, ECMWS, etc.)

63. Questions