1. INFSO-RI-031688 Enabling Grids for E-sciencE www.eu-egee.org ES applications in EGEEII – M. Petitdidier –11 February 2008 Earth Science Application Overview in EGEE Infrasctructure M. Petitdidier (IPSL/CETP), monique.petitdidier@cetp.ipsl.fr In collaboration with EGEE and DEGREE EU- project

2. SSA- IST 2005-034619 ES applications in EGEEII –– M. Petitdidier –11 February 2007 Topography & Motion Land cover & vegetation Marine SST, SSH& colour Currents, bathymetry & ice Planet Earth : a complex system Stratospheric Ozone Industrial Emissions Atmospheric profiles 10 y displacement of Etna 1992-01 GOME total ozone assimilation

3. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Acute Questions for Earth Science Forecasting of meteorological events –Extreme events: storm, tornado, hurricane, earthquake, tsunami… –Water management: precipitations, flood, aquifer.. –Pollution,,… => To provide real time information: real time data access, data assimilation and modelling Long term prediction : Climate Change In October 2007, Albert Arnold (Al) Gore Jr. and the IPCC (Intergovernmental Panel on Climate Change): Nobel Peace Prize 2007 "for their efforts to build up and disseminate greater knowledge about man-made climate change, and to lay the foundations for the measures that are needed to counteract such change”. Climatology i.e. trend of parameter variations like temperature, precipitations…. –Polar ozone hole –Impact on agriculture, weather… => Long series of multiple data sets, Intense processing, Discoveries and Dissemination of the knowledge –How it works, Why… –These are Questions for Operational and Science organisations

4. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Earth Science Challenges To answer the questions, needs to address the following situations: Global, regional, local applications –Alternative use of the data at different time and spatial resolutions Large historical distributed archives –Long term data archives to be exploited for climatology Near real-time access to data and processing –For processing, value adding and dissemination –For now-casting and alert Models to provide long term trends and forecast –Processing-intensive, data-intensive and complex applications Integrate different data sources –Standardisation, Data policy, … –Data fusion, data assimilation, data mining, modelling … Link data to technical information and scientific results Data are at the centre of ES applications

5. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 GRID Technology GRID infrastructure –Considered a comfortable “open platform” for handling computing resources, data, tools… –Impressive number of shared resources –A collaborative possible platform among teams and/or countries –Secure and restricted access to resources, data, tools Use of Grid: application requirements of the proposed application, data, numerical approach, type of computing resources, languages of code EGEE well adapted to : –Sharing data and algorithm –Processing and/or production of large data sets of data –Complex workflow –Statistical approach –Other approaches: MPI, Modelling, simulation –Grid opens new fields of investigation in Earth Science

6. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Earth Science Applications in EGEEII ESA, UTV(IT), KNMI(NL), IPSL(FR)- Production and validation of 7 years of Ozone profiles from GOME Rapid Earthquake analysis (mechanism and epicenter) 50- 100CPUs IPGP(FR) Modelling seawater intrusion in costal aquifer (SWIMED) CRS4(IT),INAT(TU), Univ.Neuchâtel(CH)- Geocluster for Academy and industry CGG(FR)- Flood of a Danube river- Cascade of models (meteorology,hydraulic,hydrodynamic….) UISAV(SK)- Specfem3D: Seismic application. Benchmark for MPI (2 to 2000 CPUs) (IPGP,FR) DKRZ(DE)- Data access studies, climate impacts on agriculture Data mining Meteorology & Space Weather (GCRAS, RU) Air Pollution model- BAS(BG) Mars atmosphere CETP( FR):

7. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Sharing data and algorithm GOME/ERS – Production of ozone profiles and validation by Lidar data –ESA, UTV, KNMI, IPSL –2 papers JQSRT GEOCLUSTER – seismic plateform developed by CGGVeritas –Available for academic end-users ELMER –available for ESR VO partners CODESA3D - Hydrology

8. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 GOME ESA(IT) – KNMI(NL) Processing of raw GOME data to ozone profiles. 2 alternative algorithms ~28000 profiles/day IPSL(FR) Validate some of the GOME ozone profiles (~10 6 /y) Coincident in space and time with Ground-Based measurements Visualization & Analyze EGEE environment Level 2 (example of 1 day total O 3 ) Level 1 Raw satellite data from the GOME instrument (~75 GB - ~5000 orbits/y) Meta Database server PosgreSQL – geospatial search

9. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 GOME 7 years of data, 14,5 orbits/day Algorithms: –Neural network, NNO, (ESA, UTV) using IDL - 2 versions –Inversion Algorithm (KNMI) –data, O3 climatology, ECMWF.. Lidar data (NSDC) –7 stations maximum (IPSL) Number of filed: 70000 for both 2 versions of NNO Common development –Metadata base on a server with security and restricted access –Query by Geolocalisation in time and space of orbits passing over a lidar site by using PosgreSQL) Results –Unique case of validation of a whole satellite data set with all the data taken by other instruments –Once the application was ported the validation of another algorithm or version is very fast

10. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Geocluster – CGG Veritas Seismic platform software developed by CGG (400 modules) ported on EGEE for research (R&D Industry + Academy) including tools le for data processing, simulation and visualisation License server for authorized users Available for academic french users of Geocluster – last Version available no need to download – Processing at a larger scale by using Grid resources

11. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 ELMER - CSC Physical models of fluid dynamics, structural mechanics, electromagnetics, heat transfer and acoustics. Available on Grid to ESR VO http://www.csc.fi/elmer Pictures describe the flow of the glacier inside volcano Uskovsky, Kamchatka, Russia See the poster

12. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Processing and/or production of large data sets GOME Geoscope: –Noise analysis of Geoscope database 25 years of seismic data Evolution of pesticides into the soil –BRGM –European project Footprint –On going

13. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Geoscope 25 years of data 28 seismological stations and data center. Data access: query by mail, and made available on the geoscope ftp server A daily data file for all the stations SEED (~200MB) Job download the data (wget) and compute the noise DSP (~1h CPU). IPGP:E. Stutzmann, G. Patau, IPSL/EGEE: D. Weissenbach Code Fortran, octave (~Matlab), shell and perl scripts Impact on the design of other seismic data server See the poster

14. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Footprint (http://www.eu-footprint.eu) European project coordinator BRGM (Bureau de Recherches Géologiques et Minières) Risk evaluation of contamination of water resources by pesticides at different time and spatial scales. Weather scenarios (data base by Meteo France), soil scenarios and 100 pesticides  12 millions de run 1-2h each  10 TBytes  Interest to use EGEE:  24/24 et 7 days/7  Possibility to run simultaneously hundreds of jobs

15. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Complex Workflow Forecasting of river flood –Development of a platform on EGEE by IISAS (SK) –Collaboration IISAS in the framework of INTAS (CNES, Ukraine and Russia) to be used for Ukrainian rivers –Collaboration IISAS with l’Ecole des Mines d’Alès for French rivers –Interest of European project CYCLOPS (Cyber-infrastructure for CiviL protection Operative Procedures)

16. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Forecasting of flood Danube river Data :météorology, river network rivières, landscape Meteorology model ALADIN (MPI-parallel), MM5 (MPI- parallel) Hydrology HSPF (sequential- parametric), NLC (sequential- parametric), Hydraulic. DaveF (MPI- parallel), FESWMS (MPI- parallel) output: weather, précipitations, hydrography, water level and flood speed forecast Cascade of jobs managed by dynamical workflow Data sources Hydrological simulation Hydraulic simulation Portal Meteorological simulation Cascade de simulations L. Hluchy, Viet Tran, M. Ciglan …. (II-SAS, Bratislava Slovaquie)

17. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Statistical approach Intrusion of seawater into coastal aquifer –CRS4 (Italie), Univ. Neuchâtel (Suisse), INAT (Tunisie), France –Large interest by mediterranean countries –SWIMED, EUMedGrid –Monte Carlo method to get probability map with good accuracy Radiation Transfer –OPGC/LaMP, France –Simulation of the back-scattered signal of the CALIOP experiment aboard the satellite CALIPSO using Monte Carlo approach. –high-resolution vertical profiles of aerosols and clouds

18. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Hydrology Management of water resources in Mediterranean area (SWIMED): G. Lecca (CRS4 Italy), P. Renard (Unine, CH), J. Kerrou (INAT, Tunisia), R. Ababou (IMFT, Fr) CODESA-3D: Density-dependent 3D coupled groundwater flow and transport simulations Data requirement: Geology, Topography, Meteorology, Water extraction by the farmer, Aquifer properties (Soil maps, Land use) GRID Impacts: Enhancing the data sharing among the field geologists, modelers, and water managers Allowing the water managers in Tunisia to investigate the potential impacts of management decision through a remote use of the grid : web interface - EUMEDGrid

19. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 OTHER Approaches Control and optimisation of execution time of a bunch of simultaneous jobs –Earthquakes Near –Real time job –Flood -Cyclops –RISICO - Cyclops parametric Jobs –Geomorphologiy Chain of jobs –Polar Ozone –Jobs longs (>24h) MPI –SPECFEM3D –3DSEM_UNSTRUCT Meteorological Modelling –MM5, WRF, RAMS –Danish Pollution model

20. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Earthquake Characteristics Fast Determination of mechanisms of important earthquakes (IPGP: E. Clévédé, G. Patau; IPSL: D. Weissenbach) Application to run on alert Collect data of 30 seismic stations from GEOSCOPE worldwide network Select stations and data Define a spatial 3D grid +time based on the assumed earthquake location Run for each grid point or group of grid points a job => ~ 50-100jobs Results obtained ~6hr after the earthquake data arrival Important for emergency action and other related researches All major earthquakes so treated: 21/24 in 2006 => catalogue

21. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Geomorphology Examples : erosion of the mountains, dynamic of dunes... Algorithm : 3D cellular automaton for geomorphological research very simple transition rules between cells of different states (transport, deposition...). Objective : understanding landscapes formation and evolution IPGP: C. Narteaux and O. Rozier

22. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Modelling study of polar stratospheric ozone loss (IPSL) Advection time step: 1 h interpolation vertical scheme every 6 h ECMWF Winds (2.5°/2.5°) Diabatic heating rates calculation chemical scheme REPROBU S (2°/2°) MIMOSA- CHIM resolution: (1 pt/ °) regular gridadvected grid S. Godin-Beekmann, J. Gazeaux A. Pazmiño, C. Boonne, O.P. Tripathi, D.Weissenbach.Long Job Complex workflow

23. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 SpecFEM3D (moguilny@ipgp.jussieu.fr) Resolution of regional scale seismic wave propagation problems to model wave propagation at high frequencies and for complex geological structures, with use of the spectral-element method. Application first written by D. Komatitsch (Université de Pau), used in more than 75 laboratories in the world,especially for the study of earthquakes. Very scalable application using F90 + MPI needing NFS mounted homes and requiring the successive launch of two mpirun on the SAME nodes, allocated in the SAME order. Project: to be distributed on EGEEII to authorized users

24. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Meteorological modelling MM5, WRF or Rams Very common software in Earth Science Large variety of applications: Dependent on the input data for initialization and added modules weather forecasting, climate, time evolution of phenomena, sensitivity test Rams already ported MM5, WRF on going MPI based Skiron/Eta weather forecasting system for short time periods Limited Area Weather and Environmental Forecasting on Grid Platform (LAWEF-GRID)

25. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Final remarks In EGEEIII improvement and/or implementation of functionalities more adapted to Earth Science –License server –Data management –Workflow –Portal New tools needed to use the whole Grid potential –Due to Change in scale of computing power –Need of Exploration of huge data sets –Creation of Platform integrating web services, computing power, information systems…. New conceptual approach of Earth Science –Role of Scientist §Interactive collaboration -> less duplication of development and/or adaptation §More time for new ideas, new research §Confidentiality of the research –Application development ( access to several large data sets, more CPUs…)

26. Dissemination and Exploitation of GRids in Earth sciencE SSA- IST 2005-034619 ES applications –AAAS’07 – M. Petitdidier –February 2007 Strategic objectives –Bridge the ES and GRID communities throughout Europe –Ensure that ES requirements are satisfied in next Grid generation –Ensure the integration of emerging technologies for managing ES knowledge The DEGREE team: IISAS, Slovakia (Coordinator) CNRS, France KNMI, The Netherlands UNINE, Switzerland CRS4, Italy SCAI, Germany GCRAS, Russia ESA-ESRIN, Italy CGG, France Dutch Space, The Netherlands Project Vision Build a bridge linking the ES and Grid communities http://www.eu-degree.eu

27. ES applications in EGEEII – M. Petitdidier – 11 February 2008 Enabling Grids for E-sciencE INFSO-RI-031688 Earth Science Expectations Pushing frontiers of scientific discovery by exploiting advanced computational methods. EGU (European Geophysical Union), Vienna, Austria – 13-18 April. 2008 EGEE booth & 2 Grid sessions ESSI8 and ESSI9