1. Office of Research and Development National Exposure Research Laboratory, Atmospheric Modeling Division, Applied Modeling Research Branch October 8, 2008 The Atmospheric Model Evaluation Tool K. Wyat Appel and Robert C. Gilliam 7 th Annual CMAS Conference, Chapel Hill, NC October 9, 2008

2. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Acknowledgements EPA –Steve Howard –Rob Gilliam –Jenise Swall –Alice Gilliland –Sharon Phillips UNC Institute for the Environment –Alexis Zubrow –Saravanan Arunachalam

3. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory What is AMET? Atmospheric Model Evaluation Tool Two modules –Meteorology (Rob Gilliam) –Air Quality (Wyat Appel) Combination of several open source software packages –MYSQL –R –Perl AMET specifically designed to compare observations against meteorological (e.g. MM5, WRF) and air quality model (e.g. CMAQ, CAMx) predictions –Does not export all gridded data to database

4. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Model Evaluation Database MySQL server database that stores all model-observation pairs in tables for access by analysis programs. Surface-based Wind Profiler Precipitation Upper-air Match obs. with model values in time and space Connect to database and insert records Generate database records Observation-Model Synchronization Other, User-developed tools The MySQL database is a standard, widely used and easily connectable database that allows users to easily connect and extract data using other software (Excel, Matlab, Perl, SAS, etc.) Analyses Model Performance Plot (scatter, range, histogram and table) Diurnal Statistics Time series Spatial Statistics Time-height statistics Text Upper-air statistics Observations NOAA ESRL MADIS Model Output MM5 (NetCDF) WRF (NetCDF) AMET-MET Evaluation Flow Chart

5. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Generate database records (FORTRAN) Populate Database (PERL) Model Evaluation Database MySQL server database that stores all model- observation pairs in tables for access by analysis programs. Uses either web interface or existing PERL scripts to create required MySQL tables. Other, User-developed tools The MySQL database is a standard, widely used and easily connectable database that allows users to easily connect and extract data using other software (Excel, Matlab, Perl, SAS, etc.) Analyses Model Performance Plots Diurnal Statistics Time series Spatial Statistics Box Plots Scatter Plots Bar Plots “Soccer Goal” Plots Bugle Plots Observations STN, IMPROVE, CASTNet, NADP, AQS, SEARCH, MDN Model Output CMAQ (IOAPI) MCIP (IOAPI) Uses Combine program Match obs. with model values in time and space using site compare and compare airs programs Observation-Model Synchronization AMET-AQ Evaluation Flow Chart

6. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Advantages of the AMET System Somewhat automated/interactive system Data stored in relational database – Data from multiple simulations stored in a single location – Allows data queries based on many factors Time period, geographic location, time of day, etc. Pre-generated analysis scripts – Same analysis for multiple simulations – Common analyses between different groups Open Source – Easy to create new scripts (if you know R)

7. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Available AMET-MET Analyses Model Performance Summary – includes various plots (scatter, box, etc.) along with various statistics Timeseries Spatial Plots – various statistical values (e.g. NMB, NME, etc.) Bar Plots – error, bias, etc. Rawindsonde Wind Profiler Aircraft Profiler

8. Model Performance Summary TemperatureWind Direction

9. 2-m Mixing Ratio 2-m Temperature 2-m Wind Speed 2-m Wind Dir. Time Series

10. Spatial Statistics

11. Wind Profiler – Model Comparisons

12. Aircraft Mean WRF Mean Aircraft Profile Comparisons (Potential Temp) Mean Absolute Error Distribution by level

13. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Available AMET-AQ Analyses Scatter Plots – model to observation – model to model (at observation points) Summary Statistics (as csv text file) Spatial Plots – various statistical values (e.g. NMB, NME, etc.) – concentrations (predicted, observed, and difference) Box Plots Stacked Bar Plot Bugle Plot Soccer Goal Plot

14. Scatter plots Model vs. Obs Model vs. Model Multiple Networks –select statistics Single Network –additional statistics Ozone Specific Temporal Averaging –monthly, seasonal, annual

15. Spatial Plots Statistics –NMB, NME, Correlation, etc. Concentrations –model, observation, difference Sub-regions

16. Time Series Plots

17. Box Plots Diurnal Box PlotMonthly Box Plot

18. Stacked Bar Plots

20. “Bugle” Plot“Soccer Goal” Plot Other Plots

21. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Public Availability of AMET AMET is available for download on the CMAS website – Both Met and AQ versions available – Script based version – Extensive users guide included Contains most of the functionality shown here Met and AQ versions can be installed together or individually Includes tutorial data and example output plots Bugzilla available for AMET

22. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Future Improvements to AMET Java interface –Currently under development –Runs AMET locally and accesses remote database –Interactive tabs –User friendly –Portable (to a degree) Additional analysis scripts –Developed internally –Developed externally (user community) More query options