NWS TAF Verification Brandi Richardson NWS Shreveport, LA.

Slides:

Advertisements

Similar presentations

Verification of Probabilistic Forecast J.P. Céron – Direction de la Climatologie S. Mason - IRI.

Advertisements

Canadian Aviation Forecast Verification

Optimizing WFO Aviation Service & Forecast Performance Dan Shoemaker Aviation Curmudgeon WFO FWD.

$500 What is 30 degrees? A change in wind speed to over/under 12 knots and more than X degrees direction change requires amendment. $500 What is a Wind.

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver.

Validation of Satellite Precipitation Estimates for Weather and Hydrological Applications Beth Ebert BMRC, Melbourne, Australia 3 rd IPWG Workshop / 3.

Dan Shoemaker Aviation Curmudgeon, NWS FWD From a 2005 study done with: Rick Curtis, Chief Meteorologist, SWA Paul Witsaman, Southern Region RAM.

Categorical Amendment Criteria (CAC) FAQ Session Aviation Services Branch November, 2009.

March 17, 2011 Severe Weather Workshop Mike York (Forecaster / Winter Weather Program Leader)

Aviation Climatology Andrew Rorke WFO LOX Los Angeles / Oxnard.

2005 Pacific Aviation Directors Workshop April 5-7 National Weather Service Pacific Region.

Lead Time Aviation Verification Onset and Cessation of Ceiling and Visibility Flight Category Conditions (IFR, MVFR, VFR) at FAA Core Airports NWS Aviation.

GreenCig/Vis Categories match Pale Green Situational awareness Orange 2 categories off, Multiple impacts Yellow 1 category off, Singular impact Red 3 categories.

Forecasting Thunderstorms in Terminal Aerodrome Forecasts (TAFs) Some new insights Steven Thompson National Weather Service (NWS) La Crosse, WI.

Paul Fajman NOAA/NWS/MDL September 7,  NDFD ugly string  NDFD Forecasts and encoding  Observations  Assumptions  Output, Scores and Display.

Reliability Trends of the Global Forecast System Model Output Statistical Guidance in the Northeastern U.S. A Statistical Analysis with Operational Forecasting.

ASSESSING THE IMPACT OF COLLABORATIVE RESEARCH PROJECTS ON NWS PERFORMANCE Jeff Waldstreicher Scientific Services Division – Eastern Region Northeast Regional.

The 10th annual Northeast Regional Operational Workshop, Albany, NY Verification of SREF Aviation Forecasts at Binghamton, NY Justin Arnott NOAA / NWS.

Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.

Aviation Verification and Convection Chris Leonardi WFO RLX August 31, 2005.

Aviation Cloud Forecasts – A True Challenge for Forecasters v       Jeffrey S. Tongue NOAA/National Weather Service - Upton, NY Wheee !

COSMO General Meeting Zurich, 2005 Institute of Meteorology and Water Management Warsaw, Poland- 1 - Verification of the LM at IMGW Katarzyna Starosta,

Writing Better Aviation AFDs (or) Do you know who your customer really is ? National Weather Service, Jackson, KY Dustin Harbage And Brian Schoettmer.

Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using forecasts and observations at all 205 UK civil and defence aerodromes.

PRACTICAL TAF WRITING Karen Oudeman NWS – Jackson, KY October 16, 2003.

AVIATION VERIFICATION NWS KEY WEST 2005 Bill South Aviation Program Leader.

1 How Are We Doing? A Verification Briefing for the SAWS III Workshop April 23, 2010 Chuck Kluepfel National Weather Service Headquarters Silver Spring,

Great Basin Verification Task 2008 Increased Variability Review of 2008 April through July Period Forecast for 4 Selected Basins Determine what verification.

© Crown copyright Met Office Operational OpenRoad verification Presented by Robert Coulson.

4IWVM - Tutorial Session - June 2009 Verification of categorical predictands Anna Ghelli ECMWF.

Ryan Kardell WFO Springfield.  Purpose of the Database  Data Sources  User Guide  Formulas Used for Scoring.

1 What’s New in Verification? A Verification Briefing for the SAWS IV Workshop October 26, 2011 Chuck Kluepfel National Weather Service Headquarters Silver.

Summer WAS*IS 2006 National Weather Service Verification Program Overview Brenton MacAloney II National Weather Service Headquarters Silver Spring, MD.

Quality Assessment - National Ceiling and Visibility (NCV) Analysis (now, not forecast) Product Tressa L. Fowler, Matthew J. Pocernich, Jamie T. Braid,

Event-based Verification and Evaluation of NWS Gridded Products: The EVENT Tool Missy Petty Forecast Impact and Quality Assessment Section NOAA/ESRL/GSD.

A Preliminary Verification of the National Hurricane Center’s Tropical Cyclone Wind Probability Forecast Product Jackie Shafer Scitor Corporation Florida.

Enhanced Lightning Products and Services for Incident Support Operations through Improved Short Term Forecast Techniques Ben Herzog 1, Matthew Volkmer.

Time Series Analysis and Forecasting

Evolving to Polygon Warnings: An Effective False Alarm Solution for the National Weather Service Pete Wolf, SOO NWS Jacksonville FL.

Latest results in verification over Poland Katarzyna Starosta, Joanna Linkowska Institute of Meteorology and Water Management, Warsaw 9th COSMO General.

Experiments in 1-6 h Forecasting of Convective Storms Using Radar Extrapolation and Numerical Weather Prediction Acknowledgements Mei Xu - MM5 Morris Weisman.

L. Mayoraz (1), J. Ambühl (1), R. Voisard (2), C. Voisard (1), M. Züger (2), H. Romang (1) (1) MeteoSwiss, Zurich, Switzerland, (2) University of Zurich,

HEMS Weather Summit – 21 March The Outlook for National-Scale Ceiling and Visibility Products Paul Herzegh Lead, FAA/AWRP National C&V Team.

Presented by: Roger Sultan, FAA, AFS-400 Date: August 8, 2012 Federal Aviation Administration FPAW Summer CFR Part 135 Automated Surface Weather.

Gregg Waller. Our Involvement… Volunteered to be an test site – installed under ATAN… Ran Apr 2007 national statistics with old verification system Installed.

Characteristics of Fog/Low Stratus Clouds are composed mainly of liquid water with a low cloud base Cloud layers are highly spatially uniform in both temperature.

Traditional Verification Scores Fake forecasts  5 geometric  7 perturbed subjective evaluation  expert scores from last year’s workshop  9 cases x.

American Meteorological Society - January 15, 2009 National Weather Service River Flood Warning Verification Mary Mullusky, Ernest Wells, Timothy Helble,

Eastern Region Aviation Overview Fred McMullen Regional Aviation Meteorologist Fred McMullen Regional.

LOW CLOUDS AND IFR FORECASTING NATIONAL WEATHER SERVICE KEN WIDELSKI October 11, 2005.

Fire Weather Customer Meeting 2004 Sponsored by NWS-ABQ.

1 Validation for CRR (PGE05) NWC SAF PAR Workshop October 2005 Madrid, Spain A. Rodríguez.

Aviation Products Derived from GOES Satellite Data Gary P. Ellrod Office of Research and Applications (NOAA/NESDIS) Camp Springs, MD.

Short Range Ensemble Prediction System Verification over Greece Petroula Louka, Flora Gofa Hellenic National Meteorological Service.

Operational verification system Rodica Dumitrache National Metorogical Administration ROMANIA.

Gridded warning verification Harold E. Brooks NOAA/National Severe Storms Laboratory Norman, Oklahoma

National Weather Service Eastern Region Activities Fred McMullen Regional Aviation Meteorologist Eastern Region Headquarters.

Verification of C&V Forecasts Jennifer Mahoney and Barbara Brown 19 April 2001.

2004 Developments in Aviation Forecast Guidance from the RUC Stan Benjamin Steve Weygandt NOAA / Forecast Systems Lab NY Courtesy:

Data Analysis of GPM Constellation Satellites-IMERG and ERA-Interim Precipitation Products over West of Iran Ehsan Sharifi 1, Reinhold Steinacker 1, and.

User-Focused Verification Barbara Brown* NCAR July 2006

The GOES-R/JPSS Approach for Identifying Hazardous Low Clouds: Overview and Operational Impacts Corey Calvert (UW-CIMSS) Mike Pavolonis (NOAA/NESDIS/STAR)

TAIWIN model verification task

COSMO Priority Project ”Quantitative Precipitation Forecasts”

Binary Forecasts and Observations

Warm Season Flash Flood Detection Performance at WFO Binghamton

Mark Andrews NOAA Aviation Weather Program Manager October 8th, 2003

Quantitative verification of cloud fraction forecasts

What forecast users might expect: an issue of forecast performance

Drivers Influencing Weather-related NAS Metrics

Presentation transcript:

NWS TAF Verification Brandi Richardson NWS Shreveport, LA

Home Previous Next Help Do we care how our forecasts verify? NO!

Home Previous Next Help Do we care how our forecasts verify? Yes! The NWS measures verification by many means –Probability of Detection (POD) –False Alarm Ratio (FAR) –Critical Success Index (CSI) –Percent Improvement Set goals for verification Local offices add own flavor Total IFR (IFR, LIFR, VLIFR)

Home Previous Next Help Why is verification important? Need to know what to improve –Lose credibility if too many forecasts are wrong Lose customers Lose jobs –Additional training –New techniques –Improved model guidance Need to know what we are doing well

Home Previous Next Help NWS TAF Verification TAFs evaluated 12 times per hour (every five minutes), or 288 times per 24-hour period TAFs compared to ASOS five-minute observations –ASOS = Automated Surface Observing System, located at TAF airports Stats calculated by flight category –i.e., VFR, MVFR, IFR, LIFR, VLIFR

Home Previous Next Help Probability of Detection How often did we correctly forecast a particular flight category to occur? –Also known as “Accuracy” POD = V/(V+M) –V = forecasted and verified events Ex: IFR conditions forecasted…IFR conditions occurred –M = missed events Ex: VFR conditions forecasted…IFR conditions occurred Ranges from 0 – 1, 1 being perfect

Home Previous Next Help False Alarm Ratio How often did we forecast a particular flight category to occur that did not occur? –i.e., how often did we “cry wolf”? FAR = U/(U+V) –U = forecasted and unverified Ex: IFR forecasted…VRF occurred –V = forecasted and verified events Ex: IFR conditions forecasted…IFR conditions occurred Ranges from 0 – 1, 0 being perfect

Home Previous Next Help Critical Success Index CSI = V/(V+M+U) –V = forecasted and verified events Ex: IFR conditions forecasted…IFR conditions occurred –M = missed events Ex: VFR conditions forecasted…IFR conditions occurred –U = forecasted and unverified Ex: IFR forecasted…VRF occurred Ranges from 0 – 1, 1 being perfect Incorporates both POD and FAR Overall score of performance

Home Previous Next Help Percent Improvement Forecaster CSI vs. Model Guidance CSI –Did we beat the model? IFR will prevail… IFR?! It’s July and dew points are in the 20s! Take that! Forecaster

Home Previous Next Help 2009 NWS Goals The NWS has set goals for TAF forecasts –For total IFR (includes IFR, LIFR, and VLIFR) POD ≥ (64%) FAR ≤ (43%) How do we measure up?...

Home Previous Next Help Examples of Local TAF Verification

Home Previous Next Help Examples of Local TAF Verification

Home Previous Next Help Examples of Local TAF Verification

Home Previous Next Help The Bottom Line Sometimes we do get the forecast wrong. Examination of TAF verification statistics helps to find our weaknesses and allows us to find ways to improve our forecasts. The NWS strives to provide quality products and services to our aviation customers and partners.