1. 94th American Meteorological Society Annual Meeting
Northern Hemisphere Forecast Skill During Extreme Winter Weather Regimes Ryan N. Maue (WeatherBELL Analytics - Tallahassee) and Rolf H. Langland (NRL- Monterey)
94th American Meteorological Society Annual Meeting
Atlanta, GA
February 2 – 6, 2014

2. Research Objectives Framework & Goals Definitions
Use current NWP deterministic & ensemble systems to analyze large-scale flow patterns and relate to medium-range forecast skill “dropouts”
Diagnose causes of low-predictability flow regimes including dropouts: inadequate observations, large analysis uncertainty, and/or model error growth
Link to skill of Teleconnection Indices such as AO/NAO, EPO, WPO and PNA
Forecast skill metrics: 5 or n-day 500 hPa geopotential anomaly correlation (NH: 20°-80°N),
Forecast dropout: an individual or collection of several consecutive forecasts that have significantly lower 500 hPa geopotential anomaly correlation skill – compared to monthly/seasonal mean
(AC < 0.8)
Low-predictability regime: particular hemispheric-scale configuration of upper-level flow that leads to below average forecast skill

3. Anomaly Correlation: Definition
500-hPa geopotential height
Northern Hemisphere 20°-80° N
Forecast anomaly from climatology* at each grid point (m)
Analysis anomaly
The AC is common forecast skill metric used by operational centers
Forecasts with AC > 0.6 are considered as providing potential positive skill
Not perfect metric, but used in concert w/ e.g. mean squared error
* NCEP CFSR + CDASv2 (reanalysis) climatology

4. Archive of Analysis and Forecast Fields
Historical record of analysis and forecasts from current NWP deterministic and ensemble systems
Key is archive and/or real-time access to forecast products
Valuable resource includes the ESRL/PSD GEFS Reforecasts Version 2 (Hamill et al. 2013) as frozen forecast model for post-processing purposes
MODEL TIME PERIOD GRID/FIELDS SOURCE
ECMWF
Oct 2006-present
T799-T1279L137
~14 km
ECMWF TIGGE
WeatherBELL
GFS
Feb 2004-present
T382-T574L64
0.5° -1°
NOMADS/NCDC
NOGAPS / NAVGEM
Jan 2004-present
(Feb 2013 upgrade)
T239-T319-T359
0.5°
NRL MONTEREY
ESRL/PSD GEFS Ref v2
Dec 1984-present
T254/T190
ESRL/PSD

5. NH: 500hPa height AC -- 45-day running means
Anomaly Correlation: Forecast skill
NH: 500hPa height AC day running means
0.9

6. different models “dropout” on the same cases
Predictive Skill Regimes
From N Hemi AC, ECMWF & GFS: Centered means of 30-days and 7-days are calculated
The 7-day minus 30-day mean represents a seasonally adjusted measure of skill
Above-average skill
Below-average skill
Models tend to have low and high skill on the same forecast(s)- but ECMWF skill drops off less than GFS
Anom Corr / predictive skill is strongly dependent on atmospheric flow regime:
different models “dropout” on the same cases

7. Anomaly Correlation: GFS recent skill
Periods of significant skill drop-off & success
Nov Dec Jan 2013

8. Anomaly Correlation: ECMWF 5-day
Nov Dec Jan 2013

9. Anomaly Correlation: ECMWF 6-day
Compare to 5-day:
Nov Dec Jan 2013

10. Anomaly Correlation: ECMWF 10-day
Nov Dec Jan 2013

11. NH: 120-hour 500-hPa height AC 45-day running means
Anomaly Correlation: forecast skill
NH: 120-hour 500-hPa height AC 45-day running means
Seasonal AC scores are highly correlated
NH Winter skill >> Summer Skill.
Both models showed record-high skill during winter
Longer time series shows improvement in skill due to major model configuration changes:
Jan 26, 2010: ECMWF T799  T1279
July 28, 2010: GFS T382  T574 and physics major upgrade
Significant Summer skill improvements 2012 & 2013 from GFS
ECMWF
GFS
5-day forecast skill leveling off during cold-season
Still significant room for warm-season improvement

12. Example: Arctic Oscillation
AO is the first EOF of sea-level pressure (1000 hPa geopotential height) variations north of 20°N latitude
How do NWP systems perform during the + and – phase of the AO, as well as through transitions (e.g. Archambault et al. 2010)– during the Northern Hemisphere cold season?
POSITIVE PHASE
NEGATIVE PHASE
J. Wallace, University of Washington

13. Arctic Oscillation
Winter 2009 – 2010
Sustained Negative AO

14. November – January Negative AO, followed by strong Positive
Arctic Oscillation
Winter 2010 – 2011
November – January Negative AO, followed by strong Positive
forecast
Major Transitions

15. AC AO Arctic Oscillation ECMWF – Winter 2009 – 2010
500 hPa NH Anomaly Correlation
Arctic Oscillation Index AC AO
Transitions
Dec 2009
Feb 2010

16. AC AO Arctic Oscillation ECMWF – Winter 2010 – 2011
500 hPa NH Anomaly Correlation
Arctic Oscillation Index
ECMWF – Winter 2010 – 2011
500 hPa NH Anomaly Correlation
Arctic Oscillation Index AC AO
Lower Skill
Very High Skill
Lower Skill
Dec 2010
Feb 2011

17. Run in real-time daily at 00z Advantage: Frozen Forecast Model
ESRL/PSD GEFS v2 Reforecasts
T254 ensemble reforecasts based upon the historical CFS Reanalysis init conditions
Run in real-time daily at 00z Advantage: Frozen Forecast Model
0.95
0.9
0.85
Nov Mar 2011
0.8

18. Run in real-time daily at 00z Advantage: Frozen Forecast Model
ESRL/PSD GEFS v2 Reforecasts
T254 ensemble reforecasts based upon the historical CFS Reanalysis init conditions
Run in real-time daily at 00z Advantage: Frozen Forecast Model
0.95
0.9
0.85
Nov 2013
Jan 2014
0.8

19. GEFSv2 Arctic Oscillation Daily Index Error1 -1
Nov Jan 2014

20. GEFSv2 Arctic Oscillation Daily Index Error
Function of analysis phase
5-day Forecast Error
10-day Forecast Error

21. Box and Whiskery Visualizing Arctic Oscillation Forecast Index
Control, mean, std dev, extremes

22. 2. Bubbles Visualizing Arctic Oscillation Forecast Index
Analyzed AO, 10-day forecast bubbles

23. 3. Color-coded members Visualizing Arctic Oscillation Forecast Index
NCEP GEFS 16-day perturbed + control

24. Summary and Future Directions
A multi-year archive of high-resolution operational center deterministic forecasts has been developed to study “dropouts” in medium-range forecast skill
The models tend to “dropout” during the same forecasts, in “lower-predictability” flow regimes
The Arctic Oscillation (AO) index is (to some extent) anti-correlated with medium-range forecast skill, as measured by the 5-day anomaly correlation of 500mb height
Value of frozen model w/many years of data (> 25) to evaluate model performance during particular large-scale flow regimes
Value of multi-model & ensemble forecast-forecast correlations for medium-range extreme events

25. Acknowledgements Data: ECMWF TIGGE, NCEP/NOMADS, ESRL/PSD
NRC Postdoc at NRL Monterey ( )
advisor: Dr. Rolf H. Langland
Updated work based on Langland and Maue (Tellus A, 2012):
Recent Northern Hemisphere mid-latitude medium-range deterministic forecast skill
Request: NCEP reforecast system similar to ECMWF hindcasts for post-processing of current model systems & “Model Climatology” for extreme events