Presentation is loading. Please wait.

Presentation is loading. Please wait.

A PERFORMANCE EVALUATION OF THE ETA - CMAQ AIR QUALITY FORECAST MODEL FOR THE SUMMER OF 2004 CMAS Workshop Chapel Hill, NC 20 October, 2004.

Similar presentations


Presentation on theme: "A PERFORMANCE EVALUATION OF THE ETA - CMAQ AIR QUALITY FORECAST MODEL FOR THE SUMMER OF 2004 CMAS Workshop Chapel Hill, NC 20 October, 2004."— Presentation transcript:

1 A PERFORMANCE EVALUATION OF THE ETA - CMAQ AIR QUALITY FORECAST MODEL FOR THE SUMMER OF 2004 CMAS Workshop Chapel Hill, NC 20 October, 2004

2 Contributors Brian Eder Daiwen Kang Tanya Otte Hsiu-mu Lin Jon Pleim Shaocai Yu George Pouliot STC Ken Schere Jeff Young ASMD ARL, NOAA All: On assignment to NERL, U.S. Environmental Protection Agency Disclaimer: “The research presented here was performed under the Memorandum of Understanding between the U.S. Environmental Protection Agency (EPA) and the U.S. Department of Commerce's National Oceanic and Atmospheric Administration (NOAA) and under agreement number DW Although it has been reviewed by EPA and NOAA and approved for publication, it does not necessarily reflect their policies or views.

3 Forecast Configuration - Eta Meteorology - CBIV Mechanism - SMOKE Emissions (Offline) - 12 km grid resolution - 22 Vertical Layers - 48 Hr. Forecast (12Z Init.) Simulation Period - 1 May – 30 September June – 30 September 2004 (presented here) Domain Models-3 CMAQ Lambert Conformal Projection

4 This performance evaluation used: Hourly O 3 (ppb) from EPA’s AIRNOW network: - over 600 stations (mostly urban) - for a four month period (J, J, A, S) A suite of statistical metrics for: - discrete forecasts (MB, NMB, RMSE, NME) - categorical forecasts (A, B, POD, FAR, CSI) of hourly, max. 1-hr and max. 8-hr O 3 (presented here).

5 Discrete Forecast / Evaluation [Observed] versus [Forecast] Statistics - Summary - Biases MB NMB - Errors

6 Category Forecast / Evaluation Observed Exceedances, Non-Exceedances versus versus Forecast Exceedances, Non-Exceedances Forecast Exceedances, Non-Exceedances ab cd Forecast Exceedance No Yes Observed Exceedance a b c d

7 Monthly summaries: -June, July, August, September -Both Discrete and Categorical evaluation Daily summaries: -August 4 -12, Discrete evaluation -Examine performance over a full synoptic cycle -Illustrating both good and poor model performance Forecast / Evaluation

8 Meteorological Conditions for the Summer (J, J, A) 2004

9 Meteorological Conditions for the Summer (J, J, A) 2004 Temperature Precipitation

10 Monthly Summaries

11 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) ABPODFARCSIa=40b= c=18274d=56 a b c d June Summary

12 Correlation Mean Bias (ppb) NMB (%) NME (%) June Summary

13

14 June Summary Bias Error

15 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) ABPODFARCSIa=272b= c=18258d=93 July Summary

16 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) ABPODFARCSIa=232b= c=19000d=29 August Summary

17 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) ABPODFARCSIa=71b= c=18018d=15 September Summary

18 Daily Summaries

19 Synoptic Scale Meteorology 4 August :15

20 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 44.1 (58.5) 55.8 (62.1) 0.41 (0.27) 11.7 (3.5) 26.5 (6.0) 17.9 (13.1) 32.8 (17.3) 48 Hour Max. 8-Hour Ozone Forecast Init: 3 Aug (12 Z) Valid: 4 Aug TP (04 Z to 03 Z) Model performance is poorer along the cold front where cloud cover and heavy rain dominate

21 Synoptic Scale Meteorology 5 August :15

22 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 43.8 (58.2) 51.6 (63.0) 0.73 (0.66) 7.8 (4.9) 17.9 (8.4) 12.3 (10.4) 22.3 (14.5) 48 Hour Max. 8-Hour Ozone Forecast Init: 4 Aug (12 Z) Valid: 5 Aug TP (04 Z to 03 Z) Model performance is still poor along the cold front where cloud cover and heavy rain dominate

23 Synoptic Scale Meteorology 6 August :15

24 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 37.8 (52.6) 44.3 (55.3) 0.82 (0.67) 6.4 (2.6) 17.0 (5.0) 8.7 (6.9) 18.9 (10.1) 48 Hour Max. 8-Hour Ozone Forecast Init: 5 Aug (12 Z) Valid: 6 Aug TP (04 Z to 03 Z) Model performance is very good throughout the domain, except for sections of New England and southeast Ontario, where cloud cover is greatest.

25 Synoptic Scale Meteorology 7 August :15

26 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 41.2 (50.1) 45.9 (50.7) 0.79 (0.71) 4.7 (0.6) 11.5 (1.3) 8.6 (5.3) 16.2 (8.2) 48 Hour Max. 8-Hour Ozone Forecast Init: 6 Aug (12 Z) Valid: 7 Aug TP (04 Z to 03 Z) Model performance is again very good, except for sections of New England, southeast Ontario and Wisconsin, where cloud cover is greatest.

27 Synoptic Scale Meteorology 8 August :15

28 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 48.7 (54.8) 52.0 (55.2) 0.75 (0.62) 3.3 (0.4) 6.8 (0.7) 9.6 (6.6) 14.5 (9.5) 48 Hour Max. 8-Hour Ozone Forecast Init: 7 Aug (12 Z) Valid: 8 Aug TP (04 Z to 03 Z) Model continues to perform well as the cP airmass undergoes modification. Warmer temperatures allow for higher ozone values (mean observed up to 48.7). New England still has more cloud cover.

29 Synoptic Scale Meteorology 9 August :15

30 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 52.9 (57.1) 55.2 () (0.61) 2.3 (1.0) 4.4 (1.8) 8.0 (7.4) 11.6 (9.8) 48 Hour Max. 8-Hour Ozone Forecast Init: 8 Aug (12 Z) Valid: 9 Aug TP (04 Z to 03 Z) Good model performance continues, as the cP airmass continues to warm. Highest ozone values (52.9) observed on this day. Cloud cover in New England dissipates resulting in better model performance there.

31 Synoptic Scale Meteorology 10 August :15

32 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 49.4 (58.4) 57.7 (60.8) 0.69 (0.59) 8.6 (2.4) 17.4 (4.2) 15.6 (9.4) 24.5 (12.6) 48 Hour Max. 8-Hour Ozone Forecast Init: 9 Aug (12 Z) Valid: 10 Aug TP (04 Z to 03 Z) Model performance degrades somewhat as cloud cover and precipitation increases over the domain, most notably with a cold front approaching from the Great Lakes and a low forming over south

33 Synoptic Scale Meteorology 11 August :15

34 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 42.2 (51.8) 54.9 (60.7) 0.71 (0.33) 12.7 (9.0) 30.2 (17.3) 15.7 (12.7) 31.7 (19.3) 48 Hour Max. 8-Hour Ozone Forecast Init: 10 Aug (12 Z) Valid: 11 Aug TP (04 Z to 03 Z) Model performance continues to suffer over the Great Lakes area as cloud cover, precipitation and very cool temperatures dominate this region.

35 Synoptic Scale Meteorology 12 August :15

36 Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) 32.5 (44.9) 51.1 (56.6) 0.38 (0.18) 18.6 (11.7) 57.4 (26.1) 21.0 (14.1) 58.0 (28.1) 48 Hour Max. 8-Hour Ozone Forecast Init: 11 Aug (12 Z) Valid: 12 Aug TP (04 Z to 03 Z) Model performance at its worst as extensive cloud cover, precipitation and very cool temperatures dominate the entire domain.

37 Monthly Summary Obs Mean Model Mean r MB (ppb) NMB (%) RMSE (ppb) NME (%) June July August September Summary Model performed reasonably well over the four month period, despite anomalously cool and wet conditions. Performance was closely tied to meteorology: M very good performance when skies were generally clear and void of precipitation, M poorer when cloud cover and precipitation occurred.


Download ppt "A PERFORMANCE EVALUATION OF THE ETA - CMAQ AIR QUALITY FORECAST MODEL FOR THE SUMMER OF 2004 CMAS Workshop Chapel Hill, NC 20 October, 2004."

Similar presentations


Ads by Google