1. National Weather Service Model Flip-Flops and Forecast Opportunities Bernard N. Meisner Scientific Services Division NWS Southern Region Fort Worth, Texas National Weather Service The views expressed herein are those of the author and do not necessarily reflect the position of the National Weather Service.

2. National Weather Service Model Flip Flops Primary Motivation: Primary Motivation: Introduction of 1200 UTC MEXMOS guidance caused concerns among some forecasters. Introduction of 1200 UTC MEXMOS guidance caused concerns among some forecasters. Perception was that guidance from 1200 UTC model runs would be substantially different from the 0000 UTC guidance. (Flips and Flops) Perception was that guidance from 1200 UTC model runs would be substantially different from the 0000 UTC guidance. (Flips and Flops) Secondary Motivation: Secondary Motivation: Given substantial improvements in MOS guidance, are there still opportunities for forecasters to add value, particularly beyond Days 1 and 2? (Forecast Opportunties) Given substantial improvements in MOS guidance, are there still opportunities for forecasters to add value, particularly beyond Days 1 and 2? (Forecast Opportunties)

3. National Weather Service Model Flips and Flops How to define them? Current model run lies outside envelope of previous ensemble. Run-to-run change in MOS max/min temperatures >10 o F. Run-to-run change in MOS max/min temperatures exceeds monthly Mean Absolute Error.

4. National Weather Service Verification 108 Hr Forecast 102 Hr Forecast 96 Hr Forecast

5. National Weather Service Model Flip Flops Data Used: MEXMOS Guidance for DFW and PIT Airports. (Have also looked at a few other sites in NWS Southern Region.) Flip (and Flop) Criterion: Run-to-run change in Max/Min temperature guidance greater than 10 o F.

6. National Weather Service Flip-Flop Spreadsheet Model “Flips” Run-to-run change exceeds this criterion All these flips were in the proper direction

7. National Weather Service Flip-Flop Spreadsheet Model “Flip/Flop” Run to run change exceeds this criterion for two consecutive model runs This flop was in the wrong direction.

8. National Weather Service No discernable difference between 0000 and 1200 runs 00 12

9. National Weather Service October 2005November 2005 December 2005 Mean Absolute Guidance Errors - KDFW January 2006

10. National Weather Service DFW Max/Min Temperature 10 o F Threshold MonthFlips Number Correct Flip – Flops Number Correct Oct 2005 33-- Nov73-- Dec9474 Jan 2006 85-- Feb551- Mar10532 Apr20-- TOTAL42 25 (60%) 11 6 (55%)

11. National Weather Service Model Flips…and Flops Summary: No substantial difference between 0000 and 1200 UTC model guidance. No substantial difference between 0000 and 1200 UTC model guidance. Perception of systematic run-to-run changes is not valid. Perception of systematic run-to-run changes is not valid. Flips (and flops) occur during the cool season. Flips (and flops) occur during the cool season. Typically occur for just one verification time. Typically occur for just one verification time. Model flips are rare; flip-flops are very rare. Model flips are rare; flip-flops are very rare. 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts) 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts) Flips are most common for Days 4-6. Flips are most common for Days 4-6. Flip-flops are most common for Days 5-6. Flip-flops are most common for Days 5-6.

12. National Weather Service Forecast Opportunities National Weather Service

13. Forecast Opportunities Data Used: MEXMOS Guidance for DFW and PIT Airports. (Have also looked at a few other sites in NWS Southern Region.) Forecast Opportunity Criterion: Max/Min temperature guidance error greater than 10 o F. Caveat: Smaller errors at certain thresholds can be significant! 28 o F vs 33 o F; 99 o F vs 104 o F

14. National Weather Service Forecast Opportunities Spreadsheet Observed temperature cells turn pink when they exceed climate normal by this amount Performance of National Weather Service Forecasts Compared to Operational, Consensus, and Weighted Model Output Statistics by Jeffrey A. Baars and Clifford F. Mass

15. National Weather Service Forecast Opportunities Spreadsheet These cells turn brown when day-to-day change in observed max/min exceeds this amount

16. National Weather Service Forecast Opportunities Spreadsheet These cells turn blue when guidance error exceeds this amount

17. National Weather Service Forecast Opportunities Spreadsheet Forecast Opportunities occur for calendar days with large departures from climatology.

18. National Weather Service Percent of Forecasts: 14%

19. October 2005 8% November 2005 9% December 2005 14% Forecast Opportunities - KDFW Threshold 10 o F January 2006 13%

20. October 2005 32% November 2005 34% December 2005 43% Threshold 5 o F January 2006 35% Forecast Opportunities - KDFW

21. October 2005 8% November 2005 9% December 2005 14% Threshold 10 o F January 2006 13% Forecast Opportunities - KDFW

22. February 2006 14% March 2006 16% April 2006 6% Threshold 10 o F May 2006 4% Forecast Opportunities - KDFW

23. National Weather Service June 2006 0% July 2006 0% Threshold 10 o F September 2006 4% August 2006 1% Forecast Opportunities - KDFW

24. National Weather Service Forecast Opportunities Forecast Opportunities: Typically occur for calendar days and can be consistent from one run to the next. Typically occur for calendar days and can be consistent from one run to the next. Are most common for Days 3-7. Are most common for Days 3-7. Frequently occur for days on which the observed max/min temperature departs substantially from the climatological norm. Frequently occur for days on which the observed max/min temperature departs substantially from the climatological norm. MEXMOS guidance typically underestimates the observed departure from normal. MEXMOS guidance typically underestimates the observed departure from normal. Rarely occur for days when the temperature change from the previous day is large. Rarely occur for days when the temperature change from the previous day is large. GFS seems to handle these events well. GFS seems to handle these events well.

25. National Weather Service Future Work Flips and Flops: Can one formulate an objective definition of a model flip (and flop)? Can one formulate an objective definition of a model flip (and flop)? Is it possible to anticipate model flips? Is it possible to anticipate model flips? When a flip occurs, can we determine whether the flip is in the correct direction? When a flip occurs, can we determine whether the flip is in the correct direction? Forecast Opportunities: How can we identify Forecast Opportunities? How can we identify Forecast Opportunities?

26. National Weather Service Questions? mailto: bernard.meisner@noaa.gov National Weather Service