GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant agreement no: EGEE 4 th User Forum – Catania, 4 Mar 2009 EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA

EC Grant Agreement no /02/20092EGEE 4th UF - Catania - March 4th, 2009 Outline Introduction to GENESI-DR –The Earth Science data deluge –The needs for data discovery and access The deployed GENESI-DR infrastructure EGEE – GENESI-DR collaboration The GOME processing and validation scenario

EC Grant Agreement no /02/20093EGEE 4th UF - Catania - March 4th, 2009 The EO data deluge… Data (Petabytes) Petabytes of already acquired data are presently underexploited, because to get the results in a reasonable time not enough computing resources, tools and algorithms are available.

EC Grant Agreement no /02/20094EGEE 4th UF - Catania - March 4th, 2009 The ES data deluge… Similar situation with other ES sensors airborne, on ground and in the sea – e.g. sensor webs

EC Grant Agreement no /02/20095EGEE 4th UF - Catania - March 4th, 2009 Several Digital Repositories Difficulties in discovering and accessing data and products Differences in data policies, formats and quality…

EC Grant Agreement no /02/20096EGEE 4th UF - Catania - March 4th, 2009 What is unique in GENESI-DR (1) GENESI-DR A centralized discovery service to seamlessly access heterogeneous sources of data… …in situ, satellite, airborne…

EC Grant Agreement no /02/20097EGEE 4th UF - Catania - March 4th, 2009 How it works… User can discover via the GENESI-DR Web portal (or web services) data and services on the base of free text string and metadata… such as geographical area, temporal range.. …and other specific parameters as applicable

EC Grant Agreement no /02/20098EGEE 4th UF - Catania - March 4th, 2009 about=rdf:about=" fm?fobjectid=1454" title=MERIS Full Resolution Geophysical Product (MER_FR__2P) abstract=MERIS Full Resolution Geophysical Product (MER_FR__2P) MERIS MER_FR__2P is a Full-Resolution Geophysical Product for Ocean, Land and Atmosphere. The product is also specific to Water Leaving Radiance (Reflectance). Each MERIS Level 2 geophysical product is derived from a MERIS Level 1b product and auxiliary parameter files specific to the MERIS Level 2 processing and distributed following the floating scene concept. NRT version is available 1 day after acquisition, OFL (fully consolidated version) 2 weeks after acquisition. The file size is 187 Mbytes per scene and auxilary data include: Surface identification flags included in the level 1b product, Orbit state vector, Time correlation parameters, Latitude, Longitude, altitude and topographic corrections, Sun azimuth, Sun elevation, view azimuth, view elevation, Mean Sea Level pressure, Total column ozone, Total column water vapour, Wind speed. datasetname=MER_FR__2P processingLevel=Level 2 satellite=ENVISAT sensor=MERIS subject1=Atmosphere (Clouds) subject2=Ocean and Coast (Sea Surface Temperature,Ocean Colour/Biology) subject3=Atmosphere (Air Quality (Ozone),Atmospheric Winds,Clouds,Precipitation) subject4=Land (Soil) rights=Free Usage with Rigths projection=pseudo-satellite projection along track resolution=265 spatial=POLYGON(( , , , , )) fileFormat=N1 (ENVISAT) contact=rdf:resource=" How it works… GENESI-DR central site owns a description of the series of data archived at the federated DRs

EC Grant Agreement no /02/20099EGEE 4th UF - Catania - March 4th, 2009 How it works… DRs reply with all data and services matching the query CNR ENEA DR

EC Grant Agreement no /02/200910EGEE 4th UF - Catania - March 4th, 2009 How it works… User is provided with the direct access to the DR catalogue, data and services. (Data are not copied to the central site). ENEA CNR ENEA ESA Individual DR data policies are considered and respected

EC Grant Agreement no /02/200911EGEE 4th UF - Catania - March 4th, 2009 How it works… User can access and manipulate data … …buoys archived at ENEA …SST maps previously archived at CNR …SST generated on the fly from satellite data at ESA using Grid resources CNR …to perform new science (and collaborate ) …to generate new products … to populate new repositories GENESI-DR

EC Grant Agreement no /02/200912EGEE 4th UF - Catania - March 4th, 2009 The deployed operational infrastructure –Currently: 8 DRs have been GENESI-fied 42 different dataset series –Data discovery, data access, processing –“GENESI-DR Operations Manual” and “GENESI- DR User Manual” satellite Oceanographic ships Buoys Airplanes Lidar Model results

EC Grant Agreement no /02/200913EGEE 4th UF - Catania - March 4th, 2009 Which services/features we demonstrate GOME Processing & Validation In DRs Data access/ download Processing Data discovery Web portal GlobModel Visualization and Validation Service SeaDataNet SST and VOS AlgalBloom SST generation Hetero- genous data API In the central cite Multiple data download Results registration in GENESI- DR Multiple access points

EC Grant Agreement no /02/200914EGEE 4th UF - Catania - March 4th, 2009 EGEE – GENESI-DR collaboration A.1 : Enable EGEE ES community access to GENESI-DR data. A.2 : To enable the EGEE ES community and GENESI-DR users to share applications across the two infrastructures

EC Grant Agreement no /02/200915EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Satellite-borne GOME sensor is able to measure the total column of Ozone in the atmosphere. Algorithms allow the scientists to estimate the vertical profiles of Ozone starting from the total column Lidar stations measure the vertical profile of Ozone While satellites provide global coverage, Lidar stations can provide measures only in a narrow area of around them Scientists are interested in comparing/validating the Ozone profiles obtained processing GOME acquisitions with Lidar measures

EC Grant Agreement no /02/200916EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Aim: to discover, access, massive processing and validation of Ozone data and products Data: –ERS-2 GOME –LIDAR IPSL/CETP Processing: –The algorithms for the computing the vertical profiles of Ozone and for validating these results with Lidar profiles are integrated in GENESI-DR

EC Grant Agreement no /02/200917EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation Webporta l Central Site ESA GOME data IPSL Lidar in-situ data Ozone processing Profiles comparision GENESI-DR “External” world

EC Grant Agreement no /02/200918EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation

EC Grant Agreement no /02/200919EGEE 4th UF - Catania - March 4th, 2009 Ozone processing and validation ESA IPSL User Portal Central Site Digital repositories Computing Elements Log in, service selection, parameter definition Data discovery Available data Service triggering Send processing Results Selected data Task status Query Dataset Series available

EC Grant Agreement no /02/200920EGEE 4th UF - Catania - March 4th, 2009 Conclusions GENESI-DR is an ambitious project with large implications on present operations in ES data providers Getting the right set of demonstration and validation scenarios are of fundamental importance to convince both Data Providers and Data Users to join the GENESI-DR federation of Earth Science Digital Repositories EGEE & GENESI-DR collaboration has already allowed the deployment of a “validation application” The application is accessed via web portal. More in general GENESI-DR exposes a discovery web service Future work within the collaboration will enable the two communities to share data and resources

GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant agreement no: EGEE 4 th User Forum – Catania, 4 Mar 2009 EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA EGEE and GENESI-DR: an interface for accessing and processing Earth Science data Roberto Cossu – ESA