1. GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories http://www.genesi-dr.eu Grant agreement no: 212073 EGEE 4 th User Forum – Catania, 4 Mar 2009 EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA

2. EC Grant Agreement no. 212073 24/02/20092EGEE 4th UF - Catania - March 4th, 2009 Outline Introduction to GENESI-DR –The Earth Science data deluge –The needs for data discovery and access The deployed GENESI-DR infrastructure EGEE – GENESI-DR collaboration The GOME processing and validation scenario

3. EC Grant Agreement no. 212073 24/02/20093EGEE 4th UF - Catania - March 4th, 2009 The EO data deluge… Data (Petabytes) 1980 1985 1990 1995 2000 2005 2010 2015 2020 2 4 6 8 10 12 14 Petabytes of already acquired data are presently underexploited, because to get the results in a reasonable time not enough computing resources, tools and algorithms are available.

4. EC Grant Agreement no. 212073 24/02/20094EGEE 4th UF - Catania - March 4th, 2009 The ES data deluge… Similar situation with other ES sensors airborne, on ground and in the sea – e.g. sensor webs

5. EC Grant Agreement no. 212073 24/02/20095EGEE 4th UF - Catania - March 4th, 2009 Several Digital Repositories Difficulties in discovering and accessing data and products Differences in data policies, formats and quality…

6. EC Grant Agreement no. 212073 24/02/20096EGEE 4th UF - Catania - March 4th, 2009 What is unique in GENESI-DR (1) GENESI-DR A centralized discovery service to seamlessly access heterogeneous sources of data… …in situ, satellite, airborne…

7. EC Grant Agreement no. 212073 24/02/20097EGEE 4th UF - Catania - March 4th, 2009 How it works… User can discover via the GENESI-DR Web portal (or web services) data and services on the base of free text string and metadata… such as geographical area, temporal range.. …and other specific parameters as applicable

8. EC Grant Agreement no. 212073 24/02/20098EGEE 4th UF - Catania - March 4th, 2009 about=rdf:about="http://earth.esa.int/object/index.c fm?fobjectid=1454" title=MERIS Full Resolution Geophysical Product (MER_FR__2P) abstract=MERIS Full Resolution Geophysical Product (MER_FR__2P) MERIS MER_FR__2P is a Full-Resolution Geophysical Product for Ocean, Land and Atmosphere. The product is also specific to Water Leaving Radiance (Reflectance). Each MERIS Level 2 geophysical product is derived from a MERIS Level 1b product and auxiliary parameter files specific to the MERIS Level 2 processing and distributed following the floating scene concept. NRT version is available 1 day after acquisition, OFL (fully consolidated version) 2 weeks after acquisition. The file size is 187 Mbytes per scene and auxilary data include: Surface identification flags included in the level 1b product, Orbit state vector, Time correlation parameters, Latitude, Longitude, altitude and topographic corrections, Sun azimuth, Sun elevation, view azimuth, view elevation, Mean Sea Level pressure, Total column ozone, Total column water vapour, Wind speed. datasetname=MER_FR__2P processingLevel=Level 2 satellite=ENVISAT sensor=MERIS subject1=Atmosphere (Clouds) subject2=Ocean and Coast (Sea Surface Temperature,Ocean Colour/Biology) subject3=Atmosphere (Air Quality (Ozone),Atmospheric Winds,Clouds,Precipitation) subject4=Land (Soil) rights=Free Usage with Rigths projection=pseudo-satellite projection along track resolution=265 spatial=POLYGON((-180.0 -90.0, -180 90, 180 90, 180 -90, -180 -90)) fileFormat=N1 (ENVISAT) contact=rdf:resource="http://eohelp.esa.int/" How it works… GENESI-DR central site owns a description of the series of data archived at the federated DRs

9. EC Grant Agreement no. 212073 24/02/20099EGEE 4th UF - Catania - March 4th, 2009 How it works… DRs reply with all data and services matching the query CNR ENEA DR

10. EC Grant Agreement no. 212073 24/02/200910EGEE 4th UF - Catania - March 4th, 2009 How it works… User is provided with the direct access to the DR catalogue, data and services. (Data are not copied to the central site). ENEA CNR ENEA ESA Individual DR data policies are considered and respected

11. EC Grant Agreement no. 212073 24/02/200911EGEE 4th UF - Catania - March 4th, 2009 How it works… User can access and manipulate data … …buoys archived at ENEA …SST maps previously archived at CNR …SST generated on the fly from satellite data at ESA using Grid resources CNR …to perform new science (and collaborate ) …to generate new products … to populate new repositories GENESI-DR

12. EC Grant Agreement no. 212073 24/02/200912EGEE 4th UF - Catania - March 4th, 2009 The deployed operational infrastructure –Currently: 8 DRs have been GENESI-fied 42 different dataset series –Data discovery, data access, processing –“GENESI-DR Operations Manual” and “GENESI- DR User Manual” satellite Oceanographic ships Buoys Airplanes Lidar Model results

13. EC Grant Agreement no. 212073 24/02/200913EGEE 4th UF - Catania - March 4th, 2009 Which services/features we demonstrate GOME Processing & Validation In DRs Data access/ download Processing Data discovery Web portal GlobModel Visualization and Validation Service SeaDataNet SST and VOS AlgalBloom SST generation Hetero- genous data API In the central cite Multiple data download Results registration in GENESI- DR Multiple access points

14. EC Grant Agreement no. 212073 24/02/200914EGEE 4th UF - Catania - March 4th, 2009 EGEE – GENESI-DR collaboration A.1 : Enable EGEE ES community access to GENESI-DR data. A.2 : To enable the EGEE ES community and GENESI-DR users to share applications across the two infrastructures

15. EC Grant Agreement no. 212073 24/02/200915EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Satellite-borne GOME sensor is able to measure the total column of Ozone in the atmosphere. Algorithms allow the scientists to estimate the vertical profiles of Ozone starting from the total column Lidar stations measure the vertical profile of Ozone While satellites provide global coverage, Lidar stations can provide measures only in a narrow area of around them Scientists are interested in comparing/validating the Ozone profiles obtained processing GOME acquisitions with Lidar measures

16. EC Grant Agreement no. 212073 24/02/200916EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Aim: to discover, access, massive processing and validation of Ozone data and products Data: –ERS-2 GOME data @ESA –LIDAR profiles @ IPSL/CETP Processing: –The algorithms for the computing the vertical profiles of Ozone and for validating these results with Lidar profiles are integrated in GENESI-DR

17. EC Grant Agreement no. 212073 24/02/200917EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Webporta l Central Site ESA GOME data IPSL Lidar in-situ data Ozone processing Profiles comparision GENESI-DR “External” world

18. EC Grant Agreement no. 212073 24/02/200918EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation

19. EC Grant Agreement no. 212073 24/02/200919EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation ESA IPSL User Portal Central Site Digital repositories Computing Elements Log in, service selection, parameter definition Data discovery Available data Service triggering Send processing Results Selected data Task status Query Dataset Series available

20. EC Grant Agreement no. 212073 24/02/200920EGEE 4th UF - Catania - March 4th, 2009 Conclusions GENESI-DR is an ambitious project with large implications on present operations in ES data providers Getting the right set of demonstration and validation scenarios are of fundamental importance to convince both Data Providers and Data Users to join the GENESI-DR federation of Earth Science Digital Repositories EGEE & GENESI-DR collaboration has already allowed the deployment of a “validation application” The application is accessed via web portal. More in general GENESI-DR exposes a discovery web service Future work within the collaboration will enable the two communities to share data and resources

21. GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories http://www.genesi-dr.eu Grant agreement no: 212073 EGEE 4 th User Forum – Catania, 4 Mar 2009 EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA