1. Meteorological Development Laboratory / OST / National Weather Service  1200 and 0600 UTC OZONE 48-h experimental, 8-h (daily max) 48-h experimental, 8-h (daily max)  0600 UTC AEROSOLS 48-h developmental, 1-h, 24-h 48-h developmental, 1-h, 24-h Performance evaluation of NOAA-EPA air quality predictions Jerry Gorline and Pius Lee NAM drives the Community Multi-scale Air Quality (CMAQ) model

2. Meteorological Development Laboratory / OST / National Weather Service  National Centers for Environmental Prediction (NCEP) Model data (1-h, 8-h avg ozone, 1-h avg aerosols): Pius Lee, Jeff McQueen  Environmental Protection Agency (EPA) Point observations: (ozone: ~1300, aerosols: ~600) Scott Jackson, Brad Johns, Phil Dickerson  Meteorological Development Laboratory (MDL) Performance evaluation: Wilson Shaffer, Jerry Gorline, Michael Schenk, Valery Dagostaro, Arthur Taylor National Air Quality Forecast Capability (NAQFC) Program Manager: Paula Davidson

3. Meteorological Development Laboratory / OST / National Weather Service  Provide performance metrics to aid model development, CONUS and by region.  Use spatial maps to complement performance measures. performance measures.  Conduct case studies as needed.  Performance metrics include: FC, TS, POD, FAR, MAE, Bias. MDL’s role in air quality performance evaluation effort (ozone and aerosols)

4. Meteorological Development Laboratory / OST / National Weather Service 2x2 contingency definitions  FC = (a + d)/(a + b + c + d)  TS = a /(a + b + c) Thresholds Used:  POD = a/(a + c) Ozone: 76 ppb  FAR = b/(a + b) Aerosols: 40 ug/m 3

5. Meteorological Development Laboratory / OST / National Weather Service 2007 ozone activity a + c a, b, c More active in 2007 a, b, c 2008 ozone activity a + c

6. Meteorological Development Laboratory / OST / National Weather Service Mean 8-h ozone predictions and observations w/ bias July 2008 ozone, CONUS, 1200 UTC cycle ~3 ppb higher bias in 2008, at peak ozone

7. Meteorological Development Laboratory / OST / National Weather Service Fraction Correct (FC), Threat Score (TS) 85 vs. 76 ppb, 1200 UTC cycle, 2008 Ozone: higher TS with 76 ppb

8. Meteorological Development Laboratory / OST / National Weather Service Ozone, Threat Score (TS) vs. number of observed exceedances, 2007 vs. 2008 slightly less under-prediction in 2008

9. Meteorological Development Laboratory / OST / National Weather Service Ozone, Threat Score (TS), 2008 1200 vs. 0600 UTC Cycle 0600 slightly better

10. Meteorological Development Laboratory / OST / National Weather Service Daily max, 8-h ozone, July 17, 2008 1200 UTC experimental FC=0.800 TS=0.360 POD=0.677 FAR=0.566 Predicted in dark blue Observed as red dots

11. Meteorological Development Laboratory / OST / National Weather Service Daily max, 8-h ozone, July 18, 2008 1200 UTC experimental FC=0.773 TS=0.362 POD=0.766 FAR=0.593 Predicted in dark blue Observed as red dots

12. Meteorological Development Laboratory / OST / National Weather Service Daily max, 8-h ozone, July 19, 2008 1200 UTC experimental FC=0.822 TS=0.261 POD=0.714 FAR=0.708 Predicted in dark blue Observed as red dots

13. Meteorological Development Laboratory / OST / National Weather Service Six U.S. regions used in AQ verification: PC, RM, LM, UM, NE, SE

14. Meteorological Development Laboratory / OST / National Weather Service Threat Score (TS), 2008 1200 UTC experimental Ozone, by region, 76 ppb Highest TS in Pacific Coast region

15. Meteorological Development Laboratory / OST / National Weather Service Daily max, 8-h ozone, 2007 1200 UTC experimental Pacific Coast region, 76 ppb

16. Meteorological Development Laboratory / OST / National Weather Service Daily max, 8-h ozone, 2008 1200 UTC experimental Pacific Coast region, 76 ppb More activity in 2008 better performance

17. Meteorological Development Laboratory / OST / National Weather Service Summary: experimental ozone prediction  2008 performance similar to 2007, slightly more under-prediction in June 2007. more under-prediction in June 2007.  0600 UTC (morning update), improved guidance compared to 1200 UTC cycle. guidance compared to 1200 UTC cycle.  The new lower threshold of 76 ppb resulted in higher Threat Score, lower resulted in higher Threat Score, lower Fraction Correct compared to 85 ppb. Fraction Correct compared to 85 ppb.  Better 2008 performance in Pacific Coast region compared to 2007. region compared to 2007.

18. Meteorological Development Laboratory / OST / National Weather Service 1-h aerosol prediction April – June, 2008, CONUS transition to under-prediction Seasonal change in May 2008

19. Meteorological Development Laboratory / OST / National Weather Service 1-h aerosol prediction July - September, 2008, CONUS transition to over-prediction Seasonal change in Sept 2008 July 4 spike in observed aerosols

20. Meteorological Development Laboratory / OST / National Weather Service Regional performance, aerosols Fraction Correct (FC), 40 ug/m 3 2007 – 2008, developmental Higher FC in the summer 2008

21. Meteorological Development Laboratory / OST / National Weather Service Regional performance, aerosols Threat Score (TS), 40 ug/m 3 2007 – 2008, developmental Lower TS in the summer 2008

22. Meteorological Development Laboratory / OST / National Weather Service Fraction Correct (FC), Threat Score (TS) 40 vs. 35 ug/m 3, 0600 UTC cycle, 2008 Aerosols: Higher TS with 35 ug/m 3

23. Meteorological Development Laboratory / OST / National Weather Service Summary: Developmental Aerosol Prediction  Over-prediction beginning In Nov 2007, ending in late Apr 2008. ending in late Apr 2008.  Under-prediction beginning in May 2008, over-prediction returning Sept 2008. over-prediction returning Sept 2008.  The new lower threshold of 35 ug/m 3 resulted in higher Threat Score (TS), lower Fraction Correct (FC), compared to 40 ug/m 3.  Improved performance in Fall 2008.