NWS TAF Performance: Are we any good? Last year I put together a report card for a visit with Southwest Airlines. – Airlines have little/no access to TAF stats. 2000/3 % improvement over LAMP; 0-6 hours; for every SWA/AirTran city. – We have to be better than an automatic product. – Shows which offices are paying attention.
FWD 1000/3 Performance From Stats on Demand: … A review of previous years forecast performance showed that the percent frequency of occurrence of IFR conditions is a reasonable first estimate predictor for overall forecast performance for a given time period, or region... % Occurrence of IFR & BLO National POD/FAR Results 10 63 / 39 8 58-62/42-45 6 57-60/42-45 4 56-59/44-46 FWD: 5 70 / 32
Optimizing WFO Aviation Service & Forecast Performance The Problem? Office culture can be stuck in bad habits. Forecasters dont see effects of their TAFs. – AFP must become lobbyist for Customers. – Long term advocate for improvement…there are no quick fixes. Forecasters must take ownership of their TAFs. – Individual feedback!
My Forecast Philosophy Optimism beats pessimism; or when in doubt, leave it out. (more later) On cautious TAFs: – If TAFs could crash airplanes, the country would be littered with aluminum (copper theft would decrease). On too much detail: – TAFs are not a portrait of the atmosphere theyre a stick figure. As long as theyre anatomically correct, theyre doing their job. – (this is PPTAF philosophy as well).
More Philosophy 2000/3 is the most important TAF category!! – TAF is mandated by FAA for airlines, not GA. – 1000/3 is a GA category, not an airline category. – Costs the airlines money. – Occurs more often than lower categories. This is where AFPs should place greatest emphasis. – 2000/3 focus will not degrade 1000/3 performance.
False Alarm Rate (FAR) (cig/vis) 4 possible outcomes for a TAF (consider 2000/3): – Hit: fuel added, good forecast. – Correct Negative: no extra fuel, plane lands at destination. – Missed event: no extra fuel, plane lands at destination anyway (vast majority of cig/vis cases). – FA: fuel always added, not needed. 100% of FAs penalize airlines – FAs always add cost – much worse than missed event.
Optimism beats pessimism (FAR) If you are on the fence about an occurrence, and you bet on the lower category, youre basically betting 50/50 on a low probability event. By the way, Las Vegas wants you to stop by on your way home. DFW/DAL annual % occurrence: (AVNFPS Climo tools) (SOD) – VFR 82% Above 2000/3 90% – MVFR 12% Below 2000/3 10% – IFR 5% – LIFR & Below <1% Caution/pessimism affects FAR much more than POD.
FWD TAF Rules: Forecast probable weather, not possible weather (use AFD for possibilities). Dont cross thresholds unless conditions will probably occur. – 2000/3 is most important category, then 600/2 (400/1), 1000/3 then others. No TEMPOs in the Outlook Period. If you have confidence, write a new line.
More FWD TAF Rules: No ¼ SM in the main body. Period. – Not our job to shut down operations. We forecast weather, & let airlines decide whether to fly. No PROB30s. (more later) Forecast TSRA during probable times. Use VCTS & CB for when less likely. Use the AVNFPS cig/vis trend tool. The TAF police will always find you!
Individual Feedback Individual feedback is key to lowering FAR & improving performance. – Many think the problem lies elsewhere. Youre going to hear excuses…facts will trump them. – Shoes 4 stages of feedback whining, crying, grumbling, & finally acceptance. Many forms of feedback choose what works best for you.
Individual Feedback Run individual TAF statsat least a years worth to get a good sample size. – Annotate/circle/comment/point out good & bad. – The biases are a good place to find where individual improvements can be made (pessimistic forecasters). Show collated statistics so each individual can see where they stand in the group. – Note: LAMP performance can vary considerably person by person.
Individual Feedback % Improvement vs LAMP 2009 Fcstr D had no clue they wrote that many TEMPOs
Individual FeedbackForecaster D Did not beat MOS his/her first year at FWD Issued a 24 hr TAF with 18 hours of TSRA Note how low FAR improves scores & POD actually increased. Forecaster D LAMP
Warm Season Afternoon Thunderstorms Afternoon precip occurs @ KDFW 4% of time (SOD). When precip occurs (AVNFPS climo distribution):
Thunderstorms two TAFs (warm season): TAF KDFW 061800/072400 18012KT P6SM SCT060CB TEMPO 0620/0623 3SM TSRA BKN015CB FM070100…. TAF KDFW 061800/072400 18012KT P6SM SCT060CB TEMPO 0620/0623 6SM TSRA BKN050CB FM070100…. Whats the difference? #1 takes 3 hour hit on category.
Warm Season Afternoon Thunderstorms Were not ignoring the flight category, were forecasting the probable category. Part of MFL problem?
Feedback Works FY 2011 MFL CSI 15.4 % Improvement over LAMP FY 2010 MFL CSI 18.8 % Worse than LAMP MFL LAMP
Miscellaneous D2D Map background All ASOS/AWOS voice lines. Get current conditions (no specials from AWOS). SVR WX wind gusts as well.
Summary Stress optimistic forecasting (improve FAR) & PPTAF principles. Focus on 2000/3, but dont forget other important categories. Individual feedback works. Dig into aviation climatology to find areas ripe for improvement (AVNFPS climo tools & SOD). Every office has bad forecasts…learn from them. Improvements will take sustained effort, especially in the beginning. The office culture will change, & service/scores will improve.
Summary Constant abrasion produces the pearl; its a disease of the oyster. Lenny Bruce Questions/Comments?