# Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver.

## Presentation on theme: "Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver."— Presentation transcript:

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver Spring, Maryland 301-713-0090 x132 Charles.Kluepfel@noaa.gov

KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Question: If it only thunders for 15 minutes out of 2 hours (the length of my TEMPO group), will my False Alarm Ratio be 1 hour 45 minutes = 0.88 ? 1 hour 45 minutes = 0.88 ? 2 hours 2 hours

KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Answer: Absolutely not! We do not compute False Alarm Ratios (FAR) from the TEMPO forecast, alone. TEMPOs are not intended to be applied to the entire TEMPO period. We only compute FARs for the  Prevailing Forecast  “Operational Impact” Forecast

KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 22005KT P6SM SKC= FM2100 22005KT P6SM SKC= Prevailing Forecast = No TS. Therefore, the prevailing forecast FAR is undefined. Reminder, FAR = False alarm time Reminder, FAR = False alarm time Forecast time Forecast time Prevailing Forecast

KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= What about the Operational Impact Forecast (OIF)? From 18-20z, two TAFs in effect. Which drives the OIF during this time period? Answer: It depends. Operational Impact Forecast

KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Apply each of these steps to every 5 minute interval of the TAF. Step 1 Variability Test Variability test --- 2 changes +/- 90 minutes Variability test --- 2 changes +/- 90 minutes Reminder - TS is a binary element (yes or no) Reminder - TS is a binary element (yes or no) Step 2 Pass test – OIF is defined as the forecast equal to the observation (for binary elements). Pass test – OIF is defined as the forecast equal to the observation (for binary elements). Fail test – OIF is defined as the “worst” (most pessimistic) of the two forecasts. Fail test – OIF is defined as the “worst” (most pessimistic) of the two forecasts.

TAF Example 1 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1907z TS ended 1921z TS ended 1921z

TAF Example 1 Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS... 1905 No TS No TS TS Yes No TS 1910 TS No TS TS Yes TS 1915 TS No TS TS Yes TS 1920 TS No TS TS Yes TS 1925 No TS No TS TS Yes No TS 1930 No TS No TS TS Yes No TS... 2000 No TS No TS TS Yes No TS

Example 1 Operational Impact Forecast F’cstYesF’cstNo Obs’dYes 3 0 Obs’dNo 0 21 21 POD = 3/3 = 1.00 FAR = 0/3 = 0.00

Example 1 TEMPO Evaluation # Hours TEMPO 2 hours (24) Justified TEMPO 100% (24/24) Justified TEMPO Hit 12.5% (3/24) TEMPO S/B FM --- (0/0) TEMPO Benign --- (0/0) TEMPO Hurt --- (0/0)

TAF Example 2 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z TS ended 1821z

TAF Example 2 Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 No TS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS... 1935 No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS

Example 2 Operational Impact Forecast F’cstYesF’cstNo Obs’dYes 3 0 Obs’dNo 5 16 16 POD = 3 / 3 = 1.00 FAR = 5 / 8 = 0.63

Ex. 2 - What Went Wrong? Both PODs were great (1.0) Ex. 2 – much higher FAR (0.63 vs. zero) Variability Test very ImportantVariability Test very Important Failing Test means you will likely contribute heavily to FAR statistic.Failing Test means you will likely contribute heavily to FAR statistic. Last 34 minutes of Example 2 failed this test!Last 34 minutes of Example 2 failed this test! Limit TEMPO valid times to when you are relatively sure that two or more changes will occur within 90 minutes.Limit TEMPO valid times to when you are relatively sure that two or more changes will occur within 90 minutes. Don’t be afraid to amend frequently!!Don’t be afraid to amend frequently!!

Example 2 TEMPO Evaluation # Hours TEMPO 2 hours Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 15.8% (3/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)

Caveat The +/- 90 minute variability test that is performed at the end of each 5-minute period of the TAF does “look back” to the times that preceded the valid period of the TEMPO group, but it does “look back” to the times that preceded the valid period of the TEMPO group, but it does not “look back” to times that preceded the valid period of the TAF. does not “look back” to times that preceded the valid period of the TAF.

Extending Operational Impact Rules to Other Elements Variability Test (Step 1) Look for two or more changes +/- 90 minutes. For ceiling and visibility, the variability test looks for category changes. For wind direction, wind speed, and gusts, the variability test uses threshold values to define change. Sig Wx Type – Begin or end.

Ceiling Categories All changes are defined categorically: > 3000 feet > 3000 feet 2000-3000 feet 2000-3000 feet 1000-1900 feet 1000-1900 feet 500-900 feet 500-900 feet 200-400 feet 200-400 feet < 200 feet < 200 feet

Visibility Categories All changes are defined categorically: > 5 statute miles > 5 statute miles 3 to 5 miles 3 to 5 miles 2 to < 3 miles 2 to < 3 miles 1 to < 2 miles 1 to < 2 miles ½ to < 1 mile ½ to < 1 mile < ½ mile < ½ mile

Wind Changes Wind Direction: Change >= 40 degrees Wind Speed: Change >= 8 knots (+/-) Wind Gusts: Change >= 10 knots (+/-)

Extending Operational Impact Rules to Other Elements Step 2 Var. Test Passes => OIF is forecast w/ smallest error Ceiling / visibility (categorical error) Ceiling / visibility (categorical error) Wind (actual error in degrees, knots) Wind (actual error in degrees, knots) Tie: OIF = Prevailing Forecast Tie: OIF = Prevailing Forecast Var. Test Fails => OIF is the forecast w/ worst weather Highest wind speed / wind gusts Highest wind speed / wind gusts Lowest ceiling / visibility Lowest ceiling / visibility Wind direction – OIF is undefined.

Returning to TAF Example 2 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z TS ended 1821z

TAF Example 2 Thunderstorm Database Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 No TS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS... 1935 No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS

TAF Ex 2: Ceiling Database Obs’n Prevail TEMPO Pass VAR Test? OIF Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 None (6) None (6) 800 (3) Yes None (6) 1810 600 (3) None (6) 800 (3) Yes 800 (3) 1815 600 (3) None (6) 800 (3) Yes 800 (3) 1820 1500 (4) None (6) 800 (3) Yes 800 (3) 1825 None (6) None (6) 800 (3) Yes None (6) …… 1935 None (6) None (6) 800 (3) Yes None (6) 1940 None (6) None (6) 800 (3) No 800 (3) 1945 None (6) None (6) 800 (3) No 800 (3) 1950 None (6) None (6) 800 (3) No 800 (3) 1955 None (6) None (6) 800 (3) No 800 (3) 2000 None (6) None (6) 800 (3) No 800 (3)

TAF Ex 2: OIF Results F O R E C A S T F O R E C A S T (1) (2) (3) (4) (5) (6) (1) (2) (3) (4) (5) (6) (1) (1) O (2) B S (3) 2 E R (4) 1 V E (5) D (6) 5 16 (6) 5 16

Example 2 TEMPO Evaluation (Ceiling) # Hours TEMPO 2 hours (24) Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 10.5% (2/19) Just TEMPO Imp TAF 5.3% (1/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)

Download ppt "Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver."

Similar presentations