10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE1 IES - Institute for Environment and Sustainability Ispra - Italy Joint Research Centre (JRC) WG2 Scoping Paper on Quality Assurance... Scoping Paper on Quality Assurance of Models in Relation to the new EU Air Quality Directive (WG2 - FAIRMODE) E.Georgieva, P.Dilara

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE2 WG2 Scoping Paper on Quality Assurance... Purpose of the Scoping Paper: Provide background information on compliance related aspects of the models quality assurance procedures Define the aims of WG2 activities Propose methodologies and relevant issues to be discussed at the kick-off meeting At present the document has a “open questions” character

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE3 WG2 Scoping Paper on Quality Assurance... Modelling in the context of the new EU AQD: 1. Assessment of ambient air quality 2. Planning and mitigation strategies (e.g. emission scenarios) 3. Assessment of the contribution of natural sources, road dust and winter sanding/salting (source apportionment) 4. Short term forecast for exceedances of information and /or alert thresholds Modelling: implicitely referred as most like methodology 1.- In all zones and agglomerations, depending on pollution levels with respect to LT, UT and LV

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE4 WG2 Scoping Paper on Quality Assurance... Assessment (AQD) : modelling shall be sufficient : combination of modelling and measurements : measurements may be supplemented by modelling LAT UAT LV Limit value, Upper and Lower assess. Threshold SO2, CO, NO2, NOx, PM, lead, benzene

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE5 WG2 Scoping Paper on Quality Assurance... Models in support of the AQD should be reliable and trustful Model Quality Assurance is a crucial stage and has to be tailored to different type of models use (explicitly or implicitly mentioned in the Directive)

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE6 WG2 Scoping Paper on Quality Assurance... Model Quality Objectives (MQO) Relevant to AQ assessment Defined as modelling uncertainty with regard to hourly, 8h avg., daily or annual values WG1: Relative Directive Error PollutantQuality IndicatorModelling uncertainty SO 2, NO 2, NO x, CO Hourly 50 % 8-hour averages 50 % Daily averages 50 % Annual averages 30 % PM 10, PM 2.5, Pb Daily averages Not yet defined Annual averages 50 % Ozone and related NO and NO 2 Hourly 50% 8-hour averages 50% Benzene Annual averages 50%

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE7 WG2 Scoping Paper on Quality Assurance... Modelling uncertainty, WG1 interpretation Relative Directive Error O LV is the closest observed concentration to the limit value (LV) and M LV is the correspondingly ranked modelled concentration.

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE8 WG2 Scoping Paper on Quality Assurance... The uncertainty concept ( …remind) e.g. Britter,94; Borrego, Miranda et al.,2008 essential for models supporting political decision Can be determined by statistical evaluation comparing observations with model results TOTAL uncertainty MODEL uncertainty INPUT DATA uncertainty Variability = + + Emissions, meteorology, chemistry Stochastic atm. + anthropog. processes Parameterizations, numerical, grid

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE9 WG2 Scoping Paper on Quality Assurance... Uncertainty and model quality assurance AQD criteria for acceptability of modelling results: Modelling uncertainty Model evaluation (fitness- for- purpose): scientific evaluation verification validation model intercomparison sensitivity analysis uncertainty analysis operational evaluation If generally accepted Model Evaluation Protocol is missing, model validation and intercomparison usually address the model assessment

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE10 WG2 Scoping Paper on Quality Assurance... Open questions 1/2: stations Representativeness of stations – compound specific? data Quality of monitored data (data coverage, comparability of PM instruments, treatments of small concentrations) single modules Uncertainties of the single modules of the modelling system (emission, meteorology, chemical transport model) nested models Uncertainty of nested models (multiscale model cascades) (different spatial scales in emission inventory top/down and bottom/up, nesting in the meteorological and chemical transport model)

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE11 WG2 Scoping Paper on Quality Assurance... Open questions 2/2: combination of monitoring and modelling Uncertainty for combination of monitoring and modelling (data assimilation), uptake of Kopernikus (GMES) Atmospheric Service statistical indexes Identify complimentary quality indicators (statistical indexes) for different type of model applications, compound specific ? Communication Communication of model uncertainty to decision makers (uncertainty spatial mapping, reporting for further use in health and environmental applications) model’s The model’s user as a source of uncertainty – how to evaluate and how to overcome it

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE12 WG2 Scoping Paper on Quality Assurance... AIMS of WG2 activities: Overall goal: To develop methodology for quality assurance of models in support of the new EU – AQD (i) Develop and recommend an EU wide standard procedure for uncertainty evaluation and model performance indicators in support of AQ assessment according the AQD

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE13 WG2 Scoping Paper on Quality Assurance... AIMS of WG2 activities - continued: (ii) Create a European Framework for Model Validation for different type of models (from hotspot to regional scale) and different purposes (assessment, scenario calculations, forecast of exceedances, contribution from natural sources and winter sanding/salting) (iii) Steer any potential modifications of the AQD

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE14 WG2 Scoping Paper on Quality Assurance... WG2 participation: All EU national experts (bring their needs and expectations, practical test cases) DG-ENV (collaborate with respect implementation of the AQD) JRC- Ispra: chair for the beginning (meetings, workshops) EEA-ETC/ACC: contribute External experts if needed Others?

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE15 WG2 Scoping Paper on Quality Assurance... Activities of WG2 (tentative list): a) Organisation of intercomparison exercises for the purposes of the AQD. b) Uncertainty analysis c) Characterisation of monitoring stations

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE16 WG2 Scoping Paper on Quality Assurance... a) Organisation of intercomparison exercises for the purposes of the AQD. Models will be evaluated in two ways: How well are the models able to reproduce current air pollution ? (AQ assessment) How well do the models respond to emission changes? (scenario calculations) JRC experience with CityDelta, EuroDelta

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE17 WG2 Scoping Paper on Quality Assurance... ….Intercomparison exercise – possible objectives/activities Study the impact of different modules of an AQ system on hourly, daily and annual concentrations Study and develop appropriate statistical parameters (indexes) in addition to the AQD parameter Propose and test benchmarks Develop/adopt appropriate graphical representation (compound specific?)

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE18 WG2 Scoping Paper on Quality Assurance... b) Uncertainty analysis: Develop/adopt and regularly update datasets (make use of databases by ACCENT, COST728/732, JRC- DAM, EU – INSPIRE Initiative) Uncertainty/sensitivity due to individual physical and chemical processes Sensitivity analysis to input parameters (IC/BC, meteo, land use etc.) How to define uncertainty for Ensemble Means? Address contribution of stochastic variability

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE19 WG2 Scoping Paper on Quality Assurance... c) Characterisation of monitoring stations: Representativeness, both for AQ and Meteo Number and types Analyse if compound specific or not

10 October 2008, Cavtat (CROATIA) – First Planery Meeting FAIRMODE20 WG2 Scoping Paper on Quality Assurance... Concluding remarks: The Scoping paper is very general We invite participants to share their needs and expectations SIGN UP in FAIRMODE web site for WG2 Questionnaire Meeting in January 09 on definition of the framework, topics and first exercises