Presentation is loading. Please wait.

Presentation is loading. Please wait.

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver.

Similar presentations


Presentation on theme: "Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver."— Presentation transcript:

1 Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver Spring, Maryland x132

2 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Question: If it only thunders for 15 minutes out of 2 hours (the length of my TEMPO group), will my False Alarm Ratio be 1 hour 45 minutes = 0.88 ? 1 hour 45 minutes = 0.88 ? 2 hours 2 hours

3 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Answer: Absolutely not! We do not compute False Alarm Ratios (FAR) from the TEMPO forecast, alone. TEMPOs are not intended to be applied to the entire TEMPO period. We only compute FARs for the  Prevailing Forecast  “Operational Impact” Forecast

4 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM KT P6SM SKC= FM KT P6SM SKC= Prevailing Forecast = No TS. Therefore, the prevailing forecast FAR is undefined. Reminder, FAR = False alarm time Reminder, FAR = False alarm time Forecast time Forecast time Prevailing Forecast

5 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= What about the Operational Impact Forecast (OIF)? From 18-20z, two TAFs in effect. Which drives the OIF during this time period? Answer: It depends. Operational Impact Forecast

6 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Apply each of these steps to every 5 minute interval of the TAF. Step 1 Variability Test Variability test changes +/- 90 minutes Variability test changes +/- 90 minutes Reminder - TS is a binary element (yes or no) Reminder - TS is a binary element (yes or no) Step 2 Pass test – OIF is defined as the forecast equal to the observation (for binary elements). Pass test – OIF is defined as the forecast equal to the observation (for binary elements). Fail test – OIF is defined as the “worst” (most pessimistic) of the two forecasts. Fail test – OIF is defined as the “worst” (most pessimistic) of the two forecasts.

7 TAF Example 1 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1907z TS ended 1921z TS ended 1921z

8 TAF Example 1 Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS No TS No TS TS Yes No TS 1910 TS No TS TS Yes TS 1915 TS No TS TS Yes TS 1920 TS No TS TS Yes TS 1925 No TS No TS TS Yes No TS 1930 No TS No TS TS Yes No TS No TS No TS TS Yes No TS

9 Example 1 Operational Impact Forecast F’cstYesF’cstNo Obs’dYes 3 0 Obs’dNo POD = 3/3 = 1.00 FAR = 0/3 = 0.00

10 Example 1 TEMPO Evaluation # Hours TEMPO 2 hours (24) Justified TEMPO 100% (24/24) Justified TEMPO Hit 12.5% (3/24) TEMPO S/B FM --- (0/0) TEMPO Benign --- (0/0) TEMPO Hurt --- (0/0)

11 TAF Example 2 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z TS ended 1821z

12 TAF Example 2 Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 No TS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS

13 Example 2 Operational Impact Forecast F’cstYesF’cstNo Obs’dYes 3 0 Obs’dNo POD = 3 / 3 = 1.00 FAR = 5 / 8 = 0.63

14 Ex. 2 - What Went Wrong? Both PODs were great (1.0) Ex. 2 – much higher FAR (0.63 vs. zero) Variability Test very ImportantVariability Test very Important Failing Test means you will likely contribute heavily to FAR statistic.Failing Test means you will likely contribute heavily to FAR statistic. Last 34 minutes of Example 2 failed this test!Last 34 minutes of Example 2 failed this test! Limit TEMPO valid times to when you are relatively sure that two or more changes will occur within 90 minutes.Limit TEMPO valid times to when you are relatively sure that two or more changes will occur within 90 minutes. Don’t be afraid to amend frequently!!Don’t be afraid to amend frequently!!

15 Example 2 TEMPO Evaluation # Hours TEMPO 2 hours Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 15.8% (3/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)

16 Caveat The +/- 90 minute variability test that is performed at the end of each 5-minute period of the TAF does “look back” to the times that preceded the valid period of the TEMPO group, but it does “look back” to the times that preceded the valid period of the TEMPO group, but it does not “look back” to times that preceded the valid period of the TAF. does not “look back” to times that preceded the valid period of the TAF.

17 Extending Operational Impact Rules to Other Elements Variability Test (Step 1) Look for two or more changes +/- 90 minutes. For ceiling and visibility, the variability test looks for category changes. For wind direction, wind speed, and gusts, the variability test uses threshold values to define change. Sig Wx Type – Begin or end.

18 Ceiling Categories All changes are defined categorically: > 3000 feet > 3000 feet feet feet feet feet feet feet feet feet < 200 feet < 200 feet

19 Visibility Categories All changes are defined categorically: > 5 statute miles > 5 statute miles 3 to 5 miles 3 to 5 miles 2 to < 3 miles 2 to < 3 miles 1 to < 2 miles 1 to < 2 miles ½ to < 1 mile ½ to < 1 mile < ½ mile < ½ mile

20 Wind Changes Wind Direction: Change >= 40 degrees Wind Speed: Change >= 8 knots (+/-) Wind Gusts: Change >= 10 knots (+/-)

21 Extending Operational Impact Rules to Other Elements Step 2 Var. Test Passes => OIF is forecast w/ smallest error Ceiling / visibility (categorical error) Ceiling / visibility (categorical error) Wind (actual error in degrees, knots) Wind (actual error in degrees, knots) Tie: OIF = Prevailing Forecast Tie: OIF = Prevailing Forecast Var. Test Fails => OIF is the forecast w/ worst weather Highest wind speed / wind gusts Highest wind speed / wind gusts Lowest ceiling / visibility Lowest ceiling / visibility Wind direction – OIF is undefined.

22 Returning to TAF Example 2 KABC z KT P6SM SCT030 TEMPO SM +TSRA BKN008CB TEMPO SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z TS ended 1821z

23 TAF Example 2 Thunderstorm Database Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 No TS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS

24 TAF Ex 2: Ceiling Database Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 None (6) None (6) 800 (3) Yes None (6) (3) None (6) 800 (3) Yes 800 (3) (3) None (6) 800 (3) Yes 800 (3) (4) None (6) 800 (3) Yes 800 (3) 1825 None (6) None (6) 800 (3) Yes None (6) …… 1935 None (6) None (6) 800 (3) Yes None (6) 1940 None (6) None (6) 800 (3) No 800 (3) 1945 None (6) None (6) 800 (3) No 800 (3) 1950 None (6) None (6) 800 (3) No 800 (3) 1955 None (6) None (6) 800 (3) No 800 (3) 2000 None (6) None (6) 800 (3) No 800 (3)

25 TAF Ex 2: OIF Results F O R E C A S T F O R E C A S T (1) (2) (3) (4) (5) (6) (1) (2) (3) (4) (5) (6) (1) (1) O (2) B S (3) 2 E R (4) 1 V E (5) D (6) 5 16 (6) 5 16

26 Example 2 TEMPO Evaluation (Ceiling) # Hours TEMPO 2 hours (24) Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 10.5% (2/19) Just TEMPO Imp TAF 5.3% (1/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)


Download ppt "Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver."

Similar presentations


Ads by Google