Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third THORPEX International Science Symposium Monterey California, September 2009

Introduction Ensemble prediction systems account for the influence of spatio-temporal changes in predictability on forecasts Performance of an ensemble prediction system is flow dependent The goal of our study is to lay the theoretical foundation of a practical approach to predict spatio-temporal changes in the performance of an ensemble prediction system

Experiment Design We use an implementation of the Local Ensemble Kalman Filter (LETKF) on T62L28 resolution version of the NCEP GFS Experiments with Observations: Simulated Observations in Random Location: 2000 randomly placed vertical soundings that provide 10% coverage of model grid points (Kuhl et al. 2007, JAS). Simulated Observations at the Location of Conventional Observations: Observational noise added to “true states”, location and type taken from conventional observations Conventional Observations of the Real Atmosphere: Observations used to obtain the type and location for simulated observations (excludes satellite radiances)

Linear Diagnostics calculated in local regions using energy rescaling Explained Variance Fraction of forecast error contained in the space spanned by the ensemble Minimum value of zero when the error lies orthogonal to the space spanned by ensemble perturbations Maximum value of 1 when the ensemble correctly captures the space of uncertainty E-Dimension A local measure of complexity based on eigenvalues of the ensemble-based error covariance matrix in the local region (Introduced in Patil et al. 2001) Minimum value of 1 when the variance is confined to a single spatial pattern of uncertainty Maximum value of N when the variance is evenly distributed between N independent spatial patterns of uncertainty

Relationship between Explained Variance, E-Dimension, and Forecast Error shown for conventional observations Lower E-Dimension Higher Forecast Error Strong Instabilities Linear space provides an increasingly better representation of the space of uncertainty up to 120 hours Colors show mean E- dimension Joint Probability Distribution

Local Relative Nonlinearity a measure of linearity in the local regions  = || x a,f -x a,f || / ___(1/k)||  x a,f(k) || Modified from Gilmour et al (2001) Standard deviations of values computed using localization show a high degree of variability Time mean of globally averaged values for conventional observations Local Regions Global Distance between ensemble mean and control forecasts normalized by the average perturbation magnitude Forecast Lead Time

Correlation between relative nonlinearity and explained variance shown for conventional observations High values of explained variance at the 120 hour lead time are not due to strong linearity of the evolution of uncertainties

TV = Square of the magnitude of the error in the ensemble mean forecast TVs =Portion of TV which lies in the space spanned by the ensemble perturbations Vs = ensemble variance Evolution of Forecast Error shown for randomly placed simulated observations Forecast Lead Time For a perfect ensemble, TV and TVs would equal Vs at the initial time At initial time, TVs equals Vs therefore further inflating the variance would not improve analyses

Evolution of Forecast Error results shown for the Northern Hemisphere Extratropics Simulated obs Realsitic location Conventional obs. TV Vs Forecast Lead Time The total ensemble variance underestimates the forecast error captured by the ensemble TVs

Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at analysis time Modified from Ott et al (2002) Simulated obs Random location Simulated obs Realsitic location Conventional obs. eigen-direction Optimal performance in this measure would be indicated by 1 for all k Uncertainty is underestimated Uncertainty is overestimated d k =(  x t k ) 2 / k

Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at 120-hour lead time Conventional obs Simulated obs Realsitic location Simulated obs Random location By 120-hr lead time, the ensemble underestimates uncertainty in all directions The spectrum is steepest for observations of the real atmosphere d i < 1 : ensemble overestimates d i > 1 : ensemble underestimates eigen-direction

The leading direction of d-ratio calculated for temperature at 850hPa For realistically placed observations, the regions of largest underestimation are those of highest observation density Simulated, Realistically Placed Obs. of the real atmosphere

Conclusions The linear space spanned by ensemble perturbations provides an increasingly better representation of the space of uncertainties with increasing forecast time. The improving performance of the space of ensemble perturbations with increasing forecast time is not due to local linear error growth, but rather to nonlinearly evolving forecast errors that have a growing projection on the linear space. At analysis time, we find that the ensemble typically underestimates uncertainty more severely in regions of high observation density than for regions of low observation density. This result indicates that implementing a spatially varying adaptive covariance inflation technique may improve analyses.