1. EU 2nd Year Review – 04-05 Jan. 2003 – WP9 WP9 Earth Observation Applications Demonstration Pedro Goncalves : pedro.goncalves@esa.intpedro.goncalves@esa.int Christine Leroy: clipsl@ipsl.jussieu.fr http://documents.cern.ch/AGE/current/fullAgenda.php?ida=a021814

2. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 2 The EO WebMap Portal  The scientist interacts with the GOME information system using the WebMap portal  This allows the user to choose a geographic area and time interval to be investigated  The WebMap portal retrieves and displays the available GOME products on the map  It does this by communicating with several background services :  Web Map Server – provides map images  Web Service Broker – provides GRID web services  EO Grid Engine – integrates the Data Grid services  Products which were recently requested will be already available  Other products will be processed “on the fly” using the GRID resources

3. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 3 EO Grid Engine Web Portal Application Web Service Interface Earth Observation GRID Engine Data Grid Services Fabric & Resources JDL Composition Job Execution Data Transfer & Replication Metadata Management Archive Management Interface Application GRID Web Mapping Technology Web Services Technology GRID + other Technology GRID Surfer (Desktop application)

4. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 4 Global Ozone (GOME) Data Processing and Validation  The DataGrid testbed provides a collaborative processing environment for 3 geographically distributed EO sites (Holland, France, Italy)  Level-1 data (raw satellite measurements) are analysed to retrieve the physical quantities : Ozone Profiles (Level-2 data)  Ozone Profiles contain measurements of OZONE within a vertical column of atmosphere at a given lat/lon location above the Earth’s surface  Two different Grid processing techniques are being investigated  OPERA (Holland) - Tightly coupled - based on MPI  NOPREGO (Italy) - Loosely coupled - based on Neural Networks  Selected Profiles are validated by comparing them with ground-based LIDAR measurements in a coincident area and time

5. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 5 ESA(IT) – KNMI(NL) Processing of raw GOME data to ozone profiles. 2 alternative algorithms ~28000 profiles/day IPSL(FR) Validate some of the GOME ozone profiles (~10 6 /y) Coincident in space and time with Ground-Based measurements Visualization & Analyze EDG EO challenge: Processing / validation of 1y of GOME data LIDAR data (7 stations, 2.5MB per month) DataGrid environment Level 2 (example of 1 day total O 3 ) Level 1 Raw satellite data from the GOME instrument (~75 GB - ~5000 orbits/y)

6. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 6 EO Use Case File Numbers DatasetNumber of filesSize Level 14,72415 Mb Level 2 NNO (ESA) 9,448,00010 kb Level 2 Opera (KNMI) 9,448,00012 kb Validation Lidar (IPSL) 122.5 Mb Total: 18,900,736 files 267 Gb (just one part of a 5-year dataset) 1 Year of GOME data :

7. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 7 Overview  Demo Part 1  Ozone Profile Processing  Demo Part 2  Ozone Profile Validation

8. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 8 Part 1 - GOME Ozone Profile Processing  Select geographic area and time to be investigated  Retrieve list of available Level-1 data files from the Product Catalogue  Transfer files to the GRID  Perform Level-1 Processing, retrieve Ozone Profiles  Visualize results

9. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 9 Web Portal EO Product Catalogue EDG Storage Element EDG User Interface EDG Resource Broker EDG Computing Element EO Replica Catalogue Processing Sequence EO Grid Engine EO Product Archive 1. Search Level-1 catalogue 2. Retrieve Level-2 products 3. Level-2 Products already registered in RC? 8. Submit jobs to process Level-1 data 7. Register Level-1 data 11. Register level-2 data 9. Process Level-1 data 10. Transfer Level-2 data to SE 12. Return new Level-2 products Yes? 4. Return available Level-2 products No? 5. Perform GRID processing on-the-fly 6. Transfer Level-1 data from Archive to the Grid

10. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 10 Validation Processing Sequence Level 2 Catalogue Lidar data catalogue Queries and data information retrieval from the Lidar metadata catalogue GRID Computing Element Storage Elements with Lidar data Queries and data information retrieval from the Gome Level 2 orbit or pixel metadata catalogues When completed comparison between lidar and satellite ozone profiles Satellite data validation Lidar site Level 2 Catalogue (5/02/2003) GRID Portal Storage Elements with Gome L2 data Submission of the Job in the GRID 1 2 3 4

11. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 11 Part 2 - GOME Ozone Profile Validation Goals of the DataGrid application to point out : the capability to validate satellite data with all ground based data available in an easy way:  Comparison of ozone profiles provided by satellite with lidar data in differents locations and times (see the web portal)  Statistical comparison and analysis in order to improve algorithms. OZONE LAYER 50 km 10 km ERS/GOME satellite Lidar at the Haute Provence Observatory (5/02/2003)

12. EU 2nd Year Review – 04-05 Feb. 2003 – WP9 – n° 12 Summary  The Web Portal Integrates several components of the EO Infrastructure  The EO GRID Engine provides a standard way for Web Clients to request GRID operations  The Web Portal and the EO Grid Engine allow the complexity of the underlying components to be hidden from the user  This Framework can be adapted to provide new tools for the satellite end-users