1. 15 / 05 / 2008 Model ensembles for the simulation of air quality over Europe Robert Vautard Laboratoire des Sciences du Climat et de l’Environnement And many colleagues from IPSL, LISA, INERIS, EURODELTA and TRANSCOM projects

2. 15 / 05 / 2008 Why air quality modelling?  Short-term forecasts (0-3 days)  Long-term predictions of emission scenarios (climate?):  2010 or 2020 or more  Increase knowledge on processes together with observations…

3. 15 / 05 / 2008 Air Quality forecasting Prevention 10 Years ago: statistical models, actions based on observations Now many deterministic forecasting systems Data assimilation in some cases In France, PREV’AIR system European GEMS/MACC projects (GMES)

4. 15 / 05 / 2008 What are regional AQ models? Transport Chemistry CTM Weather Boundary Conditions Emissions Landuse Concentrations Many many uncertainties…

5. 15 / 05 / 2008 Regional air quality forecast not really an initial value problem Timmermans et al 2007 Assimilation experiments without assimilation with assimilation Blond et al 2004

6. 15 / 05 / 2008 What are the skill of regional AQ forecasts? PREV’AIR Operational AQ forecasts (3 Summers): Average skill over >200 stations in Europe Honoré et al. 2008

7. 15 / 05 / 2008 Ensembles with perturbed meteorology (ARPEGE), chemistry Carvalho et al., in preparation

8. 15 / 05 / 2008 Emissions control Action Loss in life expectancy attributable to PM2.5, and 2020 simulation with current legislation, Amann et al 2005

9. 15 / 05 / 2008 But some species are very poorly simulated PM Episode intercomparison Stern et al. 2008

10. 15 / 05 / 2008 Hopes from ensembles Represent the « unpredictable part » of the system Meteorological/emission « noise », knowledge gaps  Provide better deterministic predictions by « error cancelation » Delle Monache and Stull 2003; Galmarini et al., 2004; McKeen et al., 2005  Predict the uncertainty (in forecasts, in scenarios), using the range Using one perturbed mode Hanna et al., 2001; Mallet and Sportisse 2006, Deguillaume et al., 2008, … or a model ensemble ; Vautard et al., 2006; How to evaluate ?  Easy for deterministic predictions  More difficult for uncertainty: tools borrowed from ensemble weather forecasting

11. 15 / 05 / 2008 EuroDelta Experiment Regional, european scale evaluation of emission scenarios for 2010 or 2020 Control experiment: simulation of Year 2001 7 models: CHIMERE, DEHM, EMEP, LOTOS-EUROS, MATCH, RCG, TM5, Comparison with rural stations (EMEP or AIRBASE) Results in – Van Loon et al., 2007 (Atmos. Env.) – Schaap et al., 2008 (in revision…) – Vautard et al., 2006 (Geophys. Res. Lett.) – Vautard et al., 2008 (AE, submitted)

12. 15 / 05 / 2008 Example of improvement by ensemble averaging: Mean diurnal cycles OzoneOx=O3+NO2

13. 15 / 05 / 2008 Seasonal skill scores for ozone Table 5: Correlation coefficients for daily average and daily maximum O 3. daily averagedaily maximum yearDJFMAMJJASONyearDJFMAMJJASON EMEP 0.720.670.550.500.550.750.600.590.610.53 LOTOS 0.700.490.540.490.430.760.470.700.660.48 MATCH 0.800.680.660.600.0.810.580.680.70.61 CHIMERE 0.760.620.580.640.600.840.620.710.770.62 RCG 0.710.580.590.520.360.760.560.700.610.44 DEHM 0.640.450.410.560.310.750.450.600.680.45 TM5 0.670.690.440.350.620.720.630.470.510.58 Ensemble 0.790.740.660.680.580.840.690.760.780.59

14. 15 / 05 / 2008 The skill of the ensemble mean Perfect ensemble: Assume that the ensemble of K values x k is drawn from a distribution of physically possible states:  Then the observation x a has the same statistical properties than any member of the ensemble, and the RMSE of the ensemble average can be written: b is the ensemble bias, s is the ensemble spread (standard deviation)  The RMSE is a decreasing function of the number of members K  The RMSE (ensemble skill) is linearly linked to the ensemble spread,

15. 15 / 05 / 2008 Evaluation of uncertainty Concepts and tools borrowed from ensemble weather forecasting Reliability: observation could be one of the members – Observation compatible with predicted distribution  Rank histogram: count the times the rank is 1, 2, …, n: frequencies should be equal But predicted distributions can have no information content (random or climatological) Resolution: the smaller the ensemble spread, the higher the resolution

16. 15 / 05 / 2008 Examples: time series Too large spread Too small spread

17. 15 / 05 / 2008 Mean Rank Histograms Stability of the ensemble

18. 15 / 05 / 2008 Reliability and Resolution Resolution index: Normalized spread = spread/stdev Reliability index: (extreme counts – central counts) / total counts

19. 15 / 05 / 2008 Spread - Skill relation

20. 15 / 05 / 2008 CO2 Modelling : TRANSCOM Work in progress CO2 modeling important for understanding and inverting fluxes TRANSCOM ensemble (Law et al., 2008) : Evaluation of model ability to simulate CO2 at regional scale 2 Simulation Years: 2002 and 2003 17 atmospheric models/model versions differing by resolution, input biospheric fluxes (2), anthropogenic CO2 fluxes (2) 6 monitoring sites from CARBOEUROPE-IP

21. 15 / 05 / 2008 Lack of spread Model or/and data representativeness problems?

22. 15 / 05 / 2008 Origin of ensemble spread and skill

23. 15 / 05 / 2008 Conclusions Develop methods to evaluate uncertainty prediction European ensemble displays relatively complementary aspects For ozone, poor resolution in Atlantic areas, poor reliability in complex terrain, balanced ensemble in Northern Europe. For NO2, poor reliability, for secondary inorganic aerosols reliable ensemble. For nitrate, poor reliability in gaz/solid balance. For CO2: model ensemble mean spread too small. Analysis coming soon.

24. 15 / 05 / 2008 European papers on evaluation and AQ model ensembles (several missing, most probably!)  Many individual model evaluations (to be reviewed)  EUROTRAC reports…  Tilmes et al., 2002: Forecasts over 1 month of ozone in Germany  Galmarini et al., 2004a,b; 2007 (ENSEMBLE project, dispersion models, ETEX)  EMEP review report Van Loon et al., 2004  Vautard et al., 2007, AE (CityDelta project): City-Scale (5 EU cities, 1 year), eulerian approach  Thunis et al., 2007, AE (CityDelta): Scenario ensembles at city scale  Van Loon et al., 2007, AE (EuroDelta project): Regional scale, Eulerian, ozone, 1 year  Vautard et al., 2006, GRL (EuroDelta, ozone): Ensemble uncertainty  Schaap et al., 2008, AE (EuroDelta): PM10 and components evaluation  Stern et al., 2008 (UBA exercise): PM10 extreme case in Germany