1. Can we Predict the Impact of Observations on 3 to 6 day Winter Weather Forecasts? Masters Thesis Defense May 10, 2007 Kathryn J. Sellwood University of Miami, R.S.M.A.S., M.P.O. Dept.

2. Why winter weather? It’s not all about Hurricanes!

3. Observations Upstream in Storm Tracks Used to Improve Forecast in Downstream Locations Storm Tracks for 1994 winter season (left) 6 year average eddy kinetic energy (right) from James, 1994

4. Influence of Observations Propagates eastward Future Winter Storm Location Poorly Observed Region

5. Upstream Observations improve the Downstream Forecast More Accurate Forecast Early warning

6. IF we can predict the impact of observations on specific 3-6 day winter weather forecasts THEN Observations can be used to improve accuracy and extend effective time range of forecasts

7. “Signal” from Observations at t=0 (left) and t=80 hours (right) combined 200 h-Pa u,v and T squared signal at 0 hours (left) and 80 hours (right)

8. Outline Background: Targeting ETKF Technique Results from Previous Targeting Field Programs Research Methodology Results Conclusions Future Work

9. Operational Targeting Timeline Targeting Method Future Analysis Verification initialization time (targeting) time time t i t a t v Decision time  36-60 hours   1-7 days 

10. A A B A B Targeting in WSR target regions identified for 2 day east coast forecast Day 1 Observation time Day 2 -Verification time All WSR flight paths Operational flight path

11. Ensemble Transform Kalman Filter (ETKF) Quantifies impact of observations Estimates the forecast error covariance matrix from an ensemble Assimilates observational data using a Kalman filter Computes resulting reduction in forecast error variance 5400m and 5820m 500 h-Pa height ensemble

12. Forecast error covariance matrix computed from matrix of ensemble perturbations Z P f = Z f Z fT Kalman Filter equation used to obtain new error covariance matrix if observations are assimilated P q = P r – P r H qT (H q P r H qT +R q ) -1 H q P r Difference is the Signal Covariance Matrix S q S q =P r –P q Signal Variance = reduction in forecast error variance

13. ETKF Issues ETKF relies on linear theory Depends on ensemble quality Assumes Kalman filter data assimilation / operational scheme is 3D-Var DA scheme introduces small scale noise that contaminates signal (Hodyss and Majumdar, 2007)

14. Results from previous Field Programs Targeting is effective in reducing short term forecast errors (Langland et al, 1999, Szunyogh et al., 1999, 2000, 2002) ETKF effective for short range 1-3 day targeting (Majumdar et al., 2002-a, Szunyogh et al., 2000, 2002) Flow regime related to effectiveness of ETKF Data “signal” propagates in the form of upper tropospheric Rossby wave packets (Szunyogh et al., 2000, 2002) Downstream development maintains wave packets and influences signal propagation (Szunyogh et al., 2002, Majumdar et al., 2002-b) The presence of coherent wave packets beyond operational lead times is evidence of data influence in the medium range

15. Observations over Pacific improve 3 day forecast over U.S. east coast and 6 day European forecast Improvement at 78 hours (u,v,T) 144 hour improvement (u,v,T)

16. Main Objectives Quantify ETKF’s ability to predict signal variance in the medium range Determine scales at which ETKF is effective Explore influence of flow regime Determine whether ETKF can distinguish between promising targeting cases and those where observations would have minimal effect

17. 2 Comparison Methods Method 1: measures the spatial correlation between the ETKF predicted signal variance and the squared GFS signal Method 2: makes a quantitative evaluation of ETKFs’ skill in predicting signal variance

18. The Data Set Data is from the 2006 Winter Storm Reconnaissance Program 19 individual cases Forecast variables are 200 h-Pa winds and temperature ETKF signal variance derived from a 50 member ECMWF ensemble Signals calculated as the difference between 2 forecasts that are identical except 1 omits the WSR observations Forecast produced using NCEPs Global Forecast System Model (GFS) Both fields at 1 degree resolution

19. Methodology Spatial fields of ETKF “predicted” signal variance and GFS squared-signal (“verification”) are smoothed by averaging over lat-lon grid cells. Field domain includes 180° W to 20° E from 20 to 80°N Correlation coefficients between these smoothed spatial fields are calculated, at all lead times, for various grid spacings. Correlation coefficients for actual case-specific predictions are then assessed relative to a “no- skill” baseline, constructed by randomizing the predictions of all 19 weather cases in the sample.

20. The Randomized Baseline Baseline random correlations are computed for all lead times from 0 to 144 hours The ETKF predicted signal variance for each of the 19 cases is compared to the squared GFS signals from the 18 different cases Results in a distribution of 342 random correlations This baseline captures the (non) skill of case- independent spatial structure (like climatological storm tracks) ETKF’s skill for the individual cases is compared against this baseline

21. Case Specific vs. Random Correlations Same Case Similar Pattern Randomly Selected Case Less Correlated

22. Case specific (top) Random (bottom) at 108 hour lead time

23. Correlation skill of case-specific vs. randomized predictions (2° grid) Blue = Random Red = ETKF Blue = Random Red = ETKF

24. Same as previous but for 5° grid

25. 10° grid

26. 15° grid

27. 20° grid

28. 30° grid

29. Average ETKF Correlations (solid) and Random Correlations (dashed) as a Function of lead time and resolution

30. Significance test Are the ETKF correlation coefficients significantly greater than the random distribution? Use Kalmogorov-Smirnov test for the difference of two PDFs.

31. Example of K-S test

32. 4/ 16 data points 13/20 data points = Maximum difference between distributions

33. The Kolmogorov – Smirnov test Compares cumulative distribution function (CDF) Produces 2 statistics based on “D” value H statistic tests the null hypothesis that the 2 distributions are equal H = 0 cannot reject null hypothesis H = 1 can reject with 95% confidence P statistic gives probability that the 2 distributions are indistinguishable Test applied for all lead times and resolutions

34. 2° grid

35. 5° grid H = 1 for all lead times > 0

36. 10° grid H = 1 for all lead times > 0

37. 15° grid H = 1 for all lead times > 0

38. 20° grid H =1 for all lead times > 0

39. 30° grid H = 1 for all lead times > 0

40. ETKF significantly beats random for all grid spacing and lead times > 0 At 0 hour leads ETKF predictions are not significantly better than random climatology ETKF case-specific predictions exhibit significantly better than random skill for the time ranges (3-6) days that we are interested in Skill tends to improve (relative to random) at longer lead times Higher correlations at lower resolution (larger grid)

41. ETKF has been shown to have skill in predicting the general pattern of signal variance over a large domain but…  We want to apply ETKF to specific forecasts  Can the ETKF predict signal variance specifically in predetermined verification regions at 3-6 day lead times?  If so at what resolutions and for what size verification regions

42. Verification Regions Same methodology as full domain comparison ETKF predicted signal variance compared to squared GFS signal over 20° x 20°, 40°x 40° and 60°x 60° verification regions Verification regions selected using wave packet technique of Zimin et al., 2003 Verification regions placed at the leading edge of wave packet maximum in ETKF predicted signal variance

43. Typical 120 hour V.R.s 60 X 60 40 x 40 20 x 20

44. ETKF vs Random 20° Verification Region - 2° grid RED = ETKF BLUE = RANDOM

45. K-S test 20° Verification Region 2° grid RED = ETKF BLUE = RANDOM

46. ETKF vs Random 20° Verification Region - 5° grid

47. K-S test 20° Verification Region 5° grid

48. ETKF vs Random 40° Verification Region - 2° grid

49. K-S test 40° Verification Region 2° grid

50. ETKF vs Random 40° Verification Region 5° grid

51. K-S test 40° verification region 5° grid

52. ETKF vs Random 60° Verification Region - 2° grid

53. K-S test 60° Verification Region 2° grid

54. ETKF vs Random 60° Verification Region - 5° grid

55. K-S test 60° Verification Region 5° grid

56. ETKF vs Random 60° Verification Region - 10° grid

57. K-S test 60° Verification Region 10° grid

58. Influence of Verification region size and resolution on ETKF skill (average correlation coefficients ) Verification Region Resolution

59. ETKF significantly beats random for case specific verification regions ETKF skill is significant even at 0 hour leads for 40° and 60° verification regions Better correlations for 40° and 60° V.R.s Higher correlations for lower resolution (larger grid) Better skill relative to random at short leads for 20° V.R. and long leads for 40° and 60° V.R.s Skill on synoptic scales ~ 500-1000km

60. Skill varies from case to case Why does the ETKF perform so much better (or worse) in particular cases? What makes a good or bad targeting case?

61. Individual Cases 9 best and 6 worst cases are identified Flow regime is quantified by 3 indices Flow regime evaluated over 3 regional areas How Does Background Flow Regime Effect The ETKF’s Performance?

62. Flow Regime Regions Pacific Continental Atlantic

63. Flow Regime Indices Zonal Index (Namias, 1947) Ug at 700 h-Pa level between 35° and 55° N Blocking Index (Tibaldi and Moltini, 1990)  /  y at 500 h-Pa between 40° and 60° N Ug Anomaly (Horel, 1985) Ug – Ug(climo) at 55°N 165°W PNA Index (Hanson et al., 1993) NAO Index (Liu et al., 1995, Benedict et al., 2003)

64. Low Frequency Non-Zonal Patterns PNA(positive phase) NAO(negative phase)

65. Evaluation of Flow Indices: Best Cases Zonal Index & Blocking Index: Zonal Flow Regime all cases Ug Anomaly: 1 case local blocking all others zonal PNA: negative 5 cases positive 4 cases NAO: positive 5 cases negative 4 cases

66. Evaluation of Zonal Indices 6 worst cases Zonal Index & Blocking Index: 4 cases zonal flow 2 cases blocked flow Ug Anomaly: local blocking in all cases PNA: negative all cases - zonal NAO: negative 5 cases blocked positive 1 case - zonal

67. Influence of Flow Regime Best cases characterized by zonal flow especially in Pacific Observation region PNA and NAO likely to be a factor only when associated with blocking Worst cases characterized by blocked flow in the Pacific 2 Worst cases are during Atlantic blocking ETKF predicts poorly for blocked regimes but does not seem to be negatively affected by waves (meridional flow)

68. Can ETKF Distinguish Between Good and Bad Cases? ETKF initialized for 3-6 day Atlantic forecasts ETKF target regions compared to operational flight paths 9 best, 6 worst cases evaluated Were Flight Paths Deemed Sensitive to Observations for these Forecasts?

69. Did ETKF predict sensitivity in flight regions for 3-6 day lead times? Sensitive regions (shaded areas) for 4 day forecast in East Atlantic and WSR flight tracks for 2 of the best cases

70. Sensitive regions and WSR flight paths for two of the worst cases

71. Similar Results for all good/bad cases The best cases have observational sensitivity for 3-6 day forecasts in the WSR flight region In the worst cases the WSR flight path misses the target region for the longer term forecast Results suggest ETKF is capable of identifying sensitive regions for targeted observations on these time scales ETKF is apparently able to distinguish good and bad targeting cases

72. Conclusions ETKF is effective out to 6 days ETKF has skill in case-specific verification regions ETKF is effective on synoptic scales Best prediction at low resolution Optimal verification region size is 40° ETKF can identify good and bad cases Blocked flow regime degrades ETKF skill THE ETKF TARGETING METHOD CAN BE EXTENDED IN MANY INSTANCES TO MEDIUM RANGE FORECASTING

73. Future / Related Work Determine if the ETKF predicted signal variance is associated with a reduction in forecast error variance in the medium range Explore the energy dynamics associated with signal propagation Take a closer look at how downstream development influences targeting

74. I would like to express my great appreciation to … my committee members Brian Mapes, Istvan Szunyogh and especially my advisor, Sharan Majumdar the many experienced people who provided valuable advice and information including Dan Hodyss, Yucheng Song, Fuqing Zhang and many others my family (even though they still can’t figure out my schedule) my RSMAS friends and colleagues the dolphins at the Seaquarium Thank you all !