1. CMAS Conference, October 16 – 18, 2006 The work presented here was performed by the New York State Department of Environmental Conservation with partial support from the U.S. EPA under cooperative agreement CR83228001. The views expressed in this paper do not necessarily reflect the views or policies of the New York State Department of Environmental Conservation or those of the U.S. EPA. Exploring Approaches to Integrate Observations and CMAQ Simulations for Improved Air Quality Forecasts C. Hogrefe 1,2, W. Hao 1, K. Civerolo 1, J.-Y. Ku 1, and G. Sistla 1 1 New York State Department of Environmental Conservation 2 Atmospheric Sciences Research Center, State University of New York at Albany

2. Introduction Overview of NYSDEC/EPA/NOAA air quality pilot study and past model performance Overview of NYSDEC/EPA/NOAA air quality pilot study and past model performance Description of potential approaches to improve air quality predictions Description of potential approaches to improve air quality predictions Results Results Summary and Outlook Summary and Outlook

3. Establish partnership between NYSDEC, NOAA and EPA in the area of numerical air-quality predictionEstablish partnership between NYSDEC, NOAA and EPA in the area of numerical air-quality prediction Complements the NOAA/EPA national air quality forecasting activitiesComplements the NOAA/EPA national air quality forecasting activities Project period: January 2005 – December 2007Project period: January 2005 – December 2007 Apply and evaluate a state-of-science photochemical modeling system on an ongoing basis with special emphasis on PM 2.5 predictions over New York StateApply and evaluate a state-of-science photochemical modeling system on an ongoing basis with special emphasis on PM 2.5 predictions over New York State Assess the potential usefulness of grid-based photochemical models to provide O 3 and PM 2.5 forecasts across New York StateAssess the potential usefulness of grid-based photochemical models to provide O 3 and PM 2.5 forecasts across New York State Archive daily concentrations fields for potential use in air quality / health studiesArchive daily concentrations fields for potential use in air quality / health studies Overview of Pilot Study

4. Meteorology/Emissions: Based on NCEP/NWS 48-hr ETA (prior to June 2006) or WRF (since June 2006) forecasts initialized at 12:00 UTC and 2002/2004/2005 emission inventory processed with PREMAQ Photochemical Modeling: CMAQ (version 4.5.1) Horizontal resolution: 12 km Vertical resolution: 22 layers, lowest layer ~40 m Simulation Periods: July – September, 2004 January – March, 2005 June 2005 – present Simulation Setup

5. Fractional Bias (FB) as defined in Morris et al. (2005) for 24-hr average total PM 2.5 predictions at all FRM monitors located in New York State calculated for 2004 - 2005 Model Performance over NYS: The Not-So-Good … Observed and Predicted 24-hr Average Speciated PM 2.5 at Eight STN Monitors in NYS (Upstate Rural, Upstate Urban, NYC Metro), July – September 2004 Hogrefe et al., 2006, JAM, in press

6. Correlation Coefficients for Time Series of Daily Maximum 8-hr O 3 … and the promising Correlation Coefficients for Time Series of Daily Average PM 2.5 Comparison of Predicted Air Quality Index Tendencies for CMAQ (left), Routine Expert- Based Forecasts (center), and Trend Persistence Hogrefe et al., 2006, JAM, in press

7. CMAQ forecasts often have significant biases, especially for PM 2.5 CMAQ forecasts often have significant biases, especially for PM 2.5 On the other hand, CMAQ simulations show skill in capturing temporal trends in air quality On the other hand, CMAQ simulations show skill in capturing temporal trends in air quality --> Explore approaches to integrate observations and CMAQ predictions for improved air quality forecasts

8. Potential Approaches for Combining CMAQ Simulations and Observations for Air Quality Forecasts Simple Bias-Correction Prediction – “Adjustment 1” Binned Bias-Correction Prediction – “Adjustment 2” “Bias- Adjustments” Simple CMAQ-Tendency Prediction – “Adjustment 3” Variability-Adjusted CMAQ-Tendency Prediction – “Adjustment 4” Slope-Adjusted CMAQ-Tendency Prediction – “Adjustment 5” “Tendency- Adjustments”

9. Time Period and Domain of Analysis June – September, 2005 June – September, 2005 Focus on monitors in New York State Focus on monitors in New York State Note: Methods 1-2 and 4-5 rely on incorporating observations not just for the current day but for an extended time period: Note: Methods 1-2 and 4-5 rely on incorporating observations not just for the current day but for an extended time period: In a routine forecast setting, this extended time period could be the past week, month, or season In a routine forecast setting, this extended time period could be the past week, month, or season In this study, we utilized the fixed time period from June 1 – September 30 also used for evaluating these methods, i.e. for any given day, both past and future observations were included In this study, we utilized the fixed time period from June 1 – September 30 also used for evaluating these methods, i.e. for any given day, both past and future observations were included Future analysis will consider the impact of the choice of the “calibration” or “learning” period over which the adjustment parameters in methods 1-2 and 4-5 are calculated on the performance of these approaches Future analysis will consider the impact of the choice of the “calibration” or “learning” period over which the adjustment parameters in methods 1-2 and 4-5 are calculated on the performance of these approaches

10. Methods of Comparison Focus on daily maximum 8-hr O 3 and daily average PM 2.5 Focus on daily maximum 8-hr O 3 and daily average PM 2.5 Comparison of observed and simulated pollutant concentration distributions Comparison of observed and simulated pollutant concentration distributions RMSE (total, systematic, and unsystematic) RMSE (total, systematic, and unsystematic) Categorical metrics as defined in Kang et al. (2005) for thresholds of 84 ppb (O 3 ) and 40 ug/m3 (PM 2.5 ) Categorical metrics as defined in Kang et al. (2005) for thresholds of 84 ppb (O 3 ) and 40 ug/m3 (PM 2.5 ) False Alarm Ratio (FAR) False Alarm Ratio (FAR) Probability of Detection (POD) Probability of Detection (POD) Critical Success Index (CSI) Critical Success Index (CSI)

11. Illustration of Total, Systematic, and Unsystematic RMSE Unsystematic RMSE is determined by the distance between the datapoints and the linear regression best-fit line Total RMSE is determined by the distance of the data points from the 1:1 line Systematic RMSE is determined by the distance between the the linear regression best-fit line and the 1:1 line

12. Methods 3-5 yield closer agreement with observed distributions than either the unadjusted CMAQ simulations or methods 1-2 Distributions of Daily Max. 8-hr O 3 (left) and 24-hr Av. PM 2.5 (right) from Observations, CMAQ Predictions, and Adjustment Methods 1 – 5

13. Total, Systematic, and Unsystematic RMSE of Daily Max. 8-hr O 3 Methods 1 and 2 (the “bias-adjustment” methods) significantly reduce total RMSE, mostly by reducing the systematic RMSE Methods 3 -5 (the “tendency-adjustment” methods) generally show little improvement in terms of overall RMSE. While they strongly reduce the systematic RMSE, they increase the unsystematic RMSE

14. Total, Systematic, and Unsystematic RMSE of Daily Average PM 2.5 For PM 2.5, the “binned-bias-adjustment” yields the largest reduction of total RMSE Similar to daily maximum 8-hr ozone, the “tendency” adjustment approaches reduce the systematic RMSE but tend to increase the unsystematic RMSE

15. False Alarm Ratio (FAR), Probability of Detection (POD), and Critical Success Index (CSI) For Daily Average PM 2.5 Above a Threshold of 40 ug/m3 Consistent with the narrower density functions shown before, the “bias- adjustment” methods reduce both FAR and POD, while the “tendency- adjustment” methods increase both FAR and POD As a result, the overall improvement in CSI over the original CMAQ simulations is relatively small for all methods

16. Percentage of Stations at Which a Given Adjustment Method Performed Best for a Given Metric and Pollutant The “bias-correction” approaches 1 and especially 2 work best for reducing the total RMSE at most sites for both O 3 and PM 2.5The “bias-correction” approaches 1 and especially 2 work best for reducing the total RMSE at most sites for both O 3 and PM 2.5 The “tendency-correction” approaches often work best for improving the CSI, especially for O 3The “tendency-correction” approaches often work best for improving the CSI, especially for O 3

17. Summary and Outlook Motivated by past CMAQ forecast evaluation results, we tested five potential approaches for providing improved air quality forecasts based on both observations and CMAQ simulations. While the “bias-correction” approaches 1 or 2 work best for reducing the total RMSE at most sites, the approaches that combine today’s observations with unadjusted or adjusted CMAQ- predicted temporal changes often work best for improving the CSI, especially for O 3 Moreover, the best adjustment method to improve the CSI, which measures the quality of categorical forecasts, needs to be chosen on a pollutant-by-pollutant and station-by-station basis Other studies explored more advanced techniques. For example, Delle Monache et al. (2006) and Kang et al. (2006) describe the application of a Kalman filter to generate improved air quality forecasts and report good success. Additional methods might aim at including spatial correlation structures into the model adjustment algorithm rather than relying solely on temporal structures at individual monitors. Such analyses will be performed in the future.