GEMS Kick- off MPI -Hamburg CTM - IFS interfaces GEMS- GRG Review of meeting in January and more recent thoughts Johannes Flemming

GEMS Kick- off MPI -Hamburg Background Assimilating remotely sensed observation about atmospheric composition GEMS production is hosted at ECMWF Integrated Forecast System (IFS) provides background –Transport, emissions, deposition and chemical conversion GRG-specific problem: –Chemical mechanism with > 40 species can hardly be incorporated in IFS

GEMS Kick- off MPI -Hamburg GRG - approach Include observable species in IFS (transport and assimilation) O 3, NO 2, SO 2, CO and HCHO can be observed from space (?) Assimilate species in IFS with ECMWF 4D-VAR Simulated source and sinks by coupling Chemical Transport Models (CTM) CTM: MOZART (MPI-H), TM5 (KNMI), Mocage (meteo-france)

GEMS Kick- off MPI -Hamburg What do we have to consider? Fields to be exchanged Choice of coupling method Choice of model grid Frequency and timing of exchange in forecast and assimilation runs Common computing environment for IFS and CTMs Harmonised interfaces

GEMS Kick- off MPI -Hamburg Coupling method Coupler OASIS4 Is being developed in PRISM (work in progress) Can exchange 3D fields between different grids –Mass conservation atmospheric balances (?) –Support of reduced Gaussian grid (?) –Spectral and grid point presentation Coupling on the level of the MPI processes –CTM should be MPI parallel File output possible Coding interfaces and control by XML scripts GEMS needs will be considered in development

GEMS Kick- off MPI -Hamburg CTM & IFS Horizontal resolution –IFS: reduced Gaussian grid, T159 –CTM: Regular or Gaussian grid, T63/ T42 Vertical resolution –IFS: L90 Levels in sigma hybrid coordinate –CTM: L20-30 Levels in sigma hybrid coordinate (same coefficients) Orography & surface pressure –CTMs should use an orography consistent with the spectral truncation of the IFS output Advection scheme, chemical mechanism MPI parallelisation (?)

GEMS Kick- off MPI -Hamburg IFS fields to CTM Wind components – mass fluxes Temperature Humidity Clouds Convective mass fluxes Profiles of precipitation Vertical diffusion coefficient IFS concentrations (data assimilation mode) Surface pressure ………?

GEMS Kick- off MPI -Hamburg CTM fields to IFS Production P Loss rate L – L C and P C due to chemistry (3D) – P E due to emissions (2D) – L D due to deposition (2D/3D cloud) –Not standard output – L C, P C,L D and P E are not independent –Totals only (?) Concentrations (Initial conditions) (?)

GEMS Kick- off MPI -Hamburg Dislocation problem

GEMS Kick- off MPI -Hamburg Dislocation problem IFS concentration fields differ from CTM fields because of different transport and data assimilation –CTMs P&L is not consistent with IFS concentration fields –IFS chemical analysis is not chemically consistent with CTM fields (difficult to merge) –Is this an issue ?

GEMS Kick- off MPI -Hamburg Forecast mode 2-way coupling requires synchronous run –Exchanged met-data and P&L have to be assumed constant till next coupling –P&L IC required –How to consistently merge IFS chemical analysis in CTM IC 1-way coupling (CTM gets met-data) –would allow lagged coupling (temporal interpolation of met-data)

GEMS Kick- off MPI -Hamburg Coupling in Forecast mode

GEMS Kick- off MPI -Hamburg ECMWF 4D-VAR Data assimilation T,u,v,q,O 3 O 3 transport + chemistry O 3 advection only O 3 transport + chemistry

GEMS Kick- off MPI -Hamburg Coupling in Data assimilation mode Tangent linear and adjoint of coupled system (IFS- OASIS4-CTM) would be needed (not feasible) Problems: –Including P&L chemistry in inner loops (Do we need it? What are the consequences?) –How to consistently transfer assimilated concentration to CTM IC –Impact of analysed met data on CTM (P&L) 1. Option: –No chemistry in 4D-VAR inner loops (as currently for Ozone) 2. Option: –Fixed L and P terms from CTM forecast –no impact on CTM except for met data in out loop Does coupler work in 4DVAR (?)

GEMS Kick- off MPI -Hamburg CTM-implementation GEMS partners access to ECMWF HPCF –Member state or special project access Source code management for CTM and Coupler –Perforce projects for CTMS and OASIS (license?) Build and scheduling system for CTMs and Coupler –ECMWF build-system for the CTMs Code sharing CTM and IFS –no restriction to the CTM code (?) CTM parallelisation –The CTMs need to be MPI-parallel for efficient coupling OASIS MPI1 mode only on IBM –Model have to run with communicator provided by coupler –Seems to be no problem

GEMS Kick- off MPI -Hamburg Status and (short-term) Plans at ECMWF IFS code –CY29r2 based version with 5 GRG prognostic and Aerosol –Identify best position for OASIS4 interfaces (met and chemistry) –Routine for global budget calculation OASIS4 coupler –Playing with code and toy models (local linux environment) –No support of reduced Gaussian grid –MPI2 mode (spawning) not possible –Graphical Interface ready for editing of XML configuration files –Build CTM toy model and couple it to IFS CTM – Implementation –Meeting on technical issues –TM5 and Mozart run at ECMWF HPCF in special project status –Implement CTMs in GEMS environment as soon as code is available

GEMS Kick- off MPI -Hamburg Issues CTM-IFS coupling I Technical implementation of coupler –How long will it take to get OASIS4 running with IFS and CTM –CTM efficiency, MPI communication, Coupler efficiency Data to be exchanged –List complete –Grids support (vertical and horizontally) –Spectral or Grid point –Different resolution and orography –Mass conservative interpolation Coupling frequency –High temporal coupling frequency reduces problems (requires efficient coupler)

GEMS Kick- off MPI -Hamburg Issues CTM-IFS coupling II –Constant L&P and met-data assumption –Careful analysis of temporal scales of P and L Coupling in data assimilation mode –chemistry modelling in inner loops –Analysed met-data for CTM –Different resolution and orography Different concentration patterns in CTM and IFS (Dislocation problem) –P&L fields dislocated –Chemically consistent CTM analysis –More chemistry in IFS (?) –Using more balanced tracers (NOx, Ox) in IFS

GEMS Kick- off MPI -Hamburg END Thank You!