Presentation is loading. Please wait.

Presentation is loading. Please wait.

A New Method for Evaluating Regional Air Quality Forecasts

Published byAmanda Preston Modified over 5 years ago

Similar presentations

Presentation on theme: "A New Method for Evaluating Regional Air Quality Forecasts"— Presentation transcript:

1 A New Method for Evaluating Regional Air Quality Forecasts
Helen Dacre Suzanne Gray, Stephen Belcher Heini Wernli

2 Talk Outline Studying pollution transport using the UM
Frontal flows Convection and coastal processes Qualitative model evaluation Quantitative model evaluation European Tracer EXperiment SAL evaluation method Future work MACC AQ model inter-comparison

3 Pollution Transport Boundary layer ~1km Advection Convection Mixing

4 Surface pressure analysis 00UTC on 15/11/94
Frontal Flows Surface pressure analysis 00UTC on 15/11/94 Release site

5 Frontal Flows Dacre (2009) T+12 T+24 ngm-3 T+12 T+24 Height (km)
1.00 ngm-3 0.01 0.10 10.0 T+12 T+24 Height (km) Height (km) 1.00 ngm-3 0.01 0.10 10.0 Dacre (2009) Cold front

6 Convection and Coastal Processes
UK Met Office surface pressure analysis 00UTC on 9/5/05

7 Convection and Coastal Processes
Tracer in free troposphere integrated over height 17 UTC wind direction land coast sea 17 UTC kg/m2

8 AMPEP Observations AMPEP flight path AMPEP flight height
(Aircraft Measurements of chemical Processing and Export fluxes of Pollutants) AMPEP flight path AMPEP flight height wind direction I D

9 Boundary layer profile in central England
Model tracer profile AMPEP CO profile 17:24 UTC boundary layer top background concentration Dacre et al. (2007)

10 Mass conservation

11 Mass Conservation Mass/total input mass (%) 200 100 Constant emission
Time evolution of the total amount of tracer in the UM. Release over 1 grid box (red), UM conserved tracer (blue)

12 European tracer experiment (ETEX)

13 ETEX Aim: To evaluate the ability of a variety of long-range dispersion models to predict pollution concentrations across Europe Inert and non-depositing tracer released from NW France Tracer perfluoromethylcyclopentane (PMCP) 168 surface station measurements

14 ETEX Observational Network
Release site

15 Synoptic Analysis at 00UTC 24/10/94
ETEX 1 Overview AVHRR infrared UTC 23/10/94 Synoptic Analysis at 00UTC 24/10/94

16 Observed Tracer Observed non-zero tracer concentrations 24, 36 and 48 hours after the start of the tracer release.

17 Traditional Evaluation
50km UM concentration ng/m3 12km UM concentration ng/m3 NMSE Observed concentration ng/m3 Observed concentration ng/m3 Scatter plot of observed vs modelled tracer concentration (a) 50km, (b) 12km. Zero concentrations are set to 0.01 ng m-3, dotted line shows factor of 2. Time since start of release (hours)

18 SAL Evaluation Method (Wernli et al 2008)
Takes into account the spatial correlation existing within the tracer field. Considers aspects of the Structure, Amplitude and Location of the tracer field independently. Interpolate observations and modelled concentrations onto same grid. Define threshold to identify features

19 Observed and Modelled Tracer
Observed (top) and 12km modelled (bottom) tracer concentrations interpolated onto a 1o1o grid, 12, 36 and 60 hours after the start of the tracer release.

20 SAL : Amplitude Component
The amplitude component quantifies errors in the over- or under-prediction of the tracer field. Where N is the number of gridcells in the domain, Qi,j is the concentration in each gridcell. -2 ≤ A ≤ +2 and 0 denotes a perfect amplitude forecast.

21 Time since start of release (hours)
SAL Diagnostics Amplitude Time since start of release (hours) Observed UM NAME

22 SAL: Location Component
The location component quantifies errors in the position of the tracer field. L1 measures the normalised distance between the centres of mass of the predicted and observed field. L2 measures the averaged distance between the centre of mass of the total field and individual objects. 0 ≤ L ≤ 2 but 0 does not necessarily denote a perfect location forecast.

23 Time since start of release (hours)
SAL Diagnostics ~422 km d=2816 km ~282 km Location ~141 km Time since start of release (hours) Observed UM NAME

24 SAL: Structure Component
The structure component quantifies errors in the predicted shape and size of the tracer field. -2 ≤ S ≤ +2 and 0 denotes a perfect structure forecast. + S → model predicts widespread tracer in a situation of small tracer features. - S → model predicts a small peaked feature in a situation of a large flat feature.

25 Idealised Example of Structure

26 Time since start of release (hours)
SAL Diagnostics Structure Time since start of release (hours) Observed UM NAME

27 Time since start of release (hours)
Model Physics Amplitude Time since start of release (hours) NAME UM Height (m) Dacre (2010)

28 Quantitative Evaluation Summary
SAL quantitative results are close to a subjective visual inspection of the predictions. SAL can be used to; Identify differences in tracer transport between models. Compare forecasts from different resolution simulations. Evaluate model performance over longer time periods and hence identify systematic errors. Evaluate ensemble predictions with varying meteorology or emissions

29 MACC Ensemble 11 air quality model forecasts of hourly and daily mean SO2, PM10, O3, NO2 and CO. 9 offline CTM’s, 2 ‘online’ Met-CTM’s and ensemble mean. 

30 AirBase Observation Network
European AirBase network. >7000 surface stations recording hourly and daily mean SO2, PM10, O3, NO2, CO

31 Future Work Do AQ models with common met input data show similar SAL performance? Online vs offline models Meteorological resolution How is the forecast skill of air quality models related to the forecast skill of the driving met model? Low-level wind skill and SAL performance for long-lived pollutants SAL precipitation performance and SAL performance for soluble species Dependence on synoptic situation

32 Extra figures

33 AMPEP Observations (Aircraft Measurements of chemical Processing and Export fluxes of Pollutants) Direct measurements of the mass budgets of pollutants in the boundary layer over the UK Upwind air measurements (background concentrations) Z Downwind air measurements (mass flux of pollution coming off UK)

34

35 UM Tracer UM tracer concentration interpolated onto 1o ×1o lat/lon grid 12, 24, 36, 48 and 60 hours after the start of the tracer release.

36 ‘Double Penalty’ Problem
Observed 50km UM 12km UM NMSE Time since start of release (hours)

37 New Evaluation Methods
Neighbourhood based Consider neighbouring observation in space and time thus relaxing the requirements for perfect matching (Roberts, 2007). Need dense network of observations such as radar. Feature based Identify features in predicted and observed fields and assess different attributes associated with each pair of forecast-observed features. Need observations on a regular grid but do not need such a dense network of observations.

38 Frontal Flows Schematic
Release site

39

40 Future Work – Multi-model evaluation
SAL: aLMo 24hr precipitation SAL: ECMWF 24hr precipitation 2 2 Amplitude Amplitude -2 -2 -2 2 -2 2 Structure Structure SAL diagrams for summer precipitation forecasts in Elbe region. Every dot shows S, A and L values for a particular day (Wernli et al. 2008).

41 Model Physics Amplitude Time series of amplitude component for UM (solid), UM with no convection (dotted), NAME (dashed) and NAME with no convection (dash-dot) simulations.

Download ppt "A New Method for Evaluating Regional Air Quality Forecasts"

Similar presentations

The University of Reading Helen Dacre The Prediction And Observation Of Volcanic Ash Clouds During The Eyjafjallajökull Eruption Helen Dacre and Alan Grant.

The University of Reading Helen Dacre The Prediction And Observation Of Volcanic Ash Clouds During The Eyjafjallajökull Eruption Helen Dacre and Alan Grant.
The University of Reading Helen Dacre AGU Dec 2008 Boundary Layer Ventilation by Convection and Coastal Processes Helen Dacre, Sue Gray, Stephen Belcher.

The University of Reading Helen Dacre AGU Dec 2008 Boundary Layer Ventilation by Convection and Coastal Processes Helen Dacre, Sue Gray, Stephen Belcher.
Vertical distribution of ash at source Time-height plots of mass concentration at Chilbolton. The height above the summit into which ash particles are.

Vertical distribution of ash at source Time-height plots of mass concentration at Chilbolton. The height above the summit into which ash particles are.
The University of Reading Helen Dacre AMS 2010 Air Quality Forecasting using a Numerical Weather Prediction Model ETEX Surface Measurement Sites.

The University of Reading Helen Dacre AMS 2010 Air Quality Forecasting using a Numerical Weather Prediction Model ETEX Surface Measurement Sites.
The University of Reading Helen Dacre UM user 2009 Forecasting the transport of pollution using a NWP model ETEX Surface Measurement Sites.

The University of Reading Helen Dacre UM user 2009 Forecasting the transport of pollution using a NWP model ETEX Surface Measurement Sites.
Www.met.rdg.ac.uk/~swr06iab Moisture Transport in Baroclinic Waves Ian Boutle a, Stephen Belcher a, Bob Plant a Bob Beare b, Andy Brown c 24 April 2014.

Moisture Transport in Baroclinic Waves Ian Boutle a, Stephen Belcher a, Bob Plant a Bob Beare b, Andy Brown c 24 April 2014.
Some recent studies using Models-3 Ian Rodgers Presentation to APRIL meeting London 4 th March 2003.

Some recent studies using Models-3 Ian Rodgers Presentation to APRIL meeting London 4 th March 2003.
Quantification of the sensitivity of NASA CMS-Flux inversions to uncertainty in atmospheric transport Thomas Lauvaux, NASA JPL Martha Butler, Kenneth Davis,

Quantification of the sensitivity of NASA CMS-Flux inversions to uncertainty in atmospheric transport Thomas Lauvaux, NASA JPL Martha Butler, Kenneth Davis,
Using old numerical techniques with a new climate-science approach, a paper from UC Irvine has created a solid, quantitative link between measurements.

Using old numerical techniques with a new climate-science approach, a paper from UC Irvine has created a solid, quantitative link between measurements.
Cold Fronts and their relationship to density currents: A case study and idealised modelling experiments Victoria Sinclair University of HelsinkI David.

Cold Fronts and their relationship to density currents: A case study and idealised modelling experiments Victoria Sinclair University of HelsinkI David.
A statistical method for calculating the impact of climate change on future air quality over the Northeast United States. Collaborators: Cynthia Lin, Katharine.

A statistical method for calculating the impact of climate change on future air quality over the Northeast United States. Collaborators: Cynthia Lin, Katharine.
PREV ’AIR : An operational system for large scale air quality monitoring and forecasting over Europe

PREV ’AIR : An operational system for large scale air quality monitoring and forecasting over Europe
Title EMEP Unified model Importance of observations for model evaluation Svetlana Tsyro MSC-W / EMEP TFMM workshop, Lillestrøm, 19 October 2010.

Title EMEP Unified model Importance of observations for model evaluation Svetlana Tsyro MSC-W / EMEP TFMM workshop, Lillestrøm, 19 October 2010.
Data assimilation of trace gases in a regional chemical transport model: the impact on model forecasts E. Emili 1, O. Pannekoucke 1,2, E. Jaumouillé 2,

Data assimilation of trace gases in a regional chemical transport model: the impact on model forecasts E. Emili 1, O. Pannekoucke 1,2, E. Jaumouillé 2,
The comparison of TransCom continuous experimental results at upper troposphere Takashi MAKI, Hidekazu MATSUEDA and TransCom Continuous modelers.

The comparison of TransCom continuous experimental results at upper troposphere Takashi MAKI, Hidekazu MATSUEDA and TransCom Continuous modelers.
ACCENT/GLOREAM 2006, 11-13 October, Paris Photo-oxidants formation and transport over Europe during the heat wave period in July 2006 Joanna Struzewska.

ACCENT/GLOREAM 2006, October, Paris Photo-oxidants formation and transport over Europe during the heat wave period in July 2006 Joanna Struzewska.
1 COPS Workshop 25 Sept 2006 UK COPS: Convection and Transport in Complex Terrain (CATICT) Alan Blyth and Stephen Mobbs.

1 COPS Workshop 25 Sept 2006 UK COPS: Convection and Transport in Complex Terrain (CATICT) Alan Blyth and Stephen Mobbs.
Impact of chemistry scheme complexity on UK air quality modelling Met Office FitzRoy Road, Exeter, Devon, EX1 3PB United Kingdom Tel: 01392 886136 Fax:

Impact of chemistry scheme complexity on UK air quality modelling Met Office FitzRoy Road, Exeter, Devon, EX1 3PB United Kingdom Tel: Fax:
© University of Reading 2007www.reading.ac.uk RMetS Student Conference, Manchester September 2008 Boundary layer ventilation by mid-latitude cyclones Victoria.

© University of Reading 2007www.reading.ac.uk RMetS Student Conference, Manchester September 2008 Boundary layer ventilation by mid-latitude cyclones Victoria.
Helen DacreDepartment of MeteorologyUniversity of Reading 1 Helen Dacre 1, Alan Grant 1, Natalie Harvey 1, Helen Webster 2, Ben Johnson 2, David Thomson.

Helen DacreDepartment of MeteorologyUniversity of Reading 1 Helen Dacre 1, Alan Grant 1, Natalie Harvey 1, Helen Webster 2, Ben Johnson 2, David Thomson.

Similar presentations

Ads by Google