1. EGEE-II INFSO-RI-031688 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks Implementation of Web Coverage Service functionalities over the EGEE Platform V.Angelini (CNR-IMAA) P.Mazzetti (CNR-IMAA) M.Mazzucato (INFN) S.Nativi (CNR-IMAA) M.Verlato (INFN) OGF – EGEE-II User Forum Manchester, 11 May 2007

2. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 2 Outline Introduction The CYCLOPS project –ID card –activities –architectural framework Geospatial grid service prototyping –OGC Web Services (OWS) –Web Coverage Service (WCS) grid implementation Conclusions

3. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 3 GMES, CP, GIS and OGC GMES is EU and ESA initiative with the aim of “establishing by 2008 a European capacity for Global Monitoring of Environment and Security” –GMES is a set of services for European citizens helping to improve their quality of life regarding environment and security The Report for the GMES Initial Period recognises the European Civil Protection (CP) as one of the GMES Service Categories –Services for risk management including prevention, monitoring and assessment at EU level of:  Natural hazards (floods, forest fires, landslides, earthquakes, etc…)  Technological hazards (industrial activities)  Maritime transport and security (oil spills, ice monitoring) GMES services based on Geographic Information System (GIS) –observation data from Earth Observation satellites and ground based information, but big fragmentation, lack of harmonisation, gaps in avail. –recent Geospatial Science developments provides the basis for GIS interoperability  Open Geospatial Consortium (OGC) as stdz. body

4. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 4 Why GRID for CP? CP deals with emergency situations: forecasting, monitoring/warning, disaster management, assessment –Many different actors involved:  At administrative level: European, National, Regional, Provincial, Local  At mission level: emergency & rescue, health, police, environmental monitoring, meteorology, scientific observatories –They need to use and share heterogeneous and distributed resources (computing, data, services, knowledge, expertise, etc) in a coordinate and effective way –Many CP applications are high demanding (high performance, high throughput, intensive data access)  GRID is ideally suited to match CP needs

5. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 5 Communities involved GMES & CP Communities Geospatial Community GRID Community Applications GI Services Provider GI Services Provider Resources Provision Resources Provision

6. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 6 CYCLOPS project ID card CYber-infrastructure for CiviL protection Operative ProcedureS Specific Support Action of EGEE Goal: to bridge the gap between Grid and GMES communities Start/end date: 01/06/2006 – 31/05/2008 Budget: 825 k€ Contact persons: –roberto.sorani@protezionecivile.it (Project Manager) –nativi@imaa.cnr.it (Technical Manager) Web site: www.cyclops-project.euwww.cyclops-project.eu Collaboration with: –PREVIEW, Risk EOS, RISK-AWARE, BOSS4GMES, INSPIRE, EGEE, DEGREE

7. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 7 8 Partners from 4 EU countries Civil Protection Agencies –CP-CH (Civil Protection of Chania Prefecture) –DDSC (Direction de la Défense et de la Sécurité Civiles) –DPC (Dipartimento della Protezione Civile) –SNBPC (Serviço Nacional de Bombeiros e Protecção Civil) Scientific/Technological partners –EMA (Ecole des Mines d' Ales ) –IMAA-CNR (Istituto di Metodologie per l’Analisi Ambientale del Consiglio Nazionale delle Ricerche) –INFN (Istituto Nazionale di Fisica Nucleare) –TEI-CR (Technological Educational Institute of Crete)

8. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 8 CYCLOPS main activities Definition of research and innovation strategies –Analysis of CP systems –EGEE Request for Enhancements –Research strategies for enabling CP applications on Grid infrastructure –Guidelines for CP innovation towards the adoption of Grid technologies Cross-dissemination between GRID (EGEE) community and GMES (Civil Protection) community. –Seminars, workshops, tutorials directed to CP personnel –Reports to EGEE Working Groups

9. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 9 CP application requirements Civil Protection and GMES applications/systems have specific requirements: –to access infrastructure, run models and search information in real-time (RT) or near-real-time (NRT):  privileging time of response instead of accuracy –to control sensors networks and acquisition systems and modify their acquisition strategy and processing chain –to share geospatial information that has complex characteristics:  huge amounts of remotely-sensed observations, which are multidimensional and frequently updated –to formalise the knowledge required to analyse data and provide decision-makers with effective information –to implement the strict data policy and the security requirements typical of dual systems (civil/military)

10. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 10 Selected use cases: Flood Forecast Involves heavily meteorology and hydrological modelling Involves many sensors (floods gauges, rain radars, meteo sats) Linked to GMES flash flood anticipation service

11. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 11 Large forest fire propagation Propagation models used on the field are simplistic due to computational limitations More sophisticated models require heavy computations, large number of field and meteo parameters French research project PAREFEU, link with GMES service PREVIEW After Sero-Guillaume et al.

12. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 12 The CYCLOPS Platform Conceive a complete Grid-based platform supporting Civil Protection/GMES applications –The CYCLOPS platform Open issues to address: –Integration & interoperability with EGEE platform  Implement Spatial Data Infrastructures (SDI) services in the EGEE platform  Integrate sensor networks and acquisition systems in the EGEE platform –Define specific services for GMES/CP applications  Advanced services like knowledge-based services, QoS management, etc.  Data Policy and Security services  Generic Business Logic, Presentation and Fruition services

13. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 13 CYCLOPS Platform CYCLOPS Infrastructure Spatial Data Infrastructure Services CYCLOPS Architectural Framework Processing Systems Infrastructure Data Systems GRID Platform (EGEE) Security Infrastructure Real Time and Near Real Time Applications for Civil Protection (Data integration, high-performance computing and distributed environment for simulations) Real Time and Near Real Time Applications for Civil Protection (Data integration, high-performance computing and distributed environment for simulations) Interoperability Platform Sensor Element Services Advanced Grid Services Business logic Services Presentation and Fruition Services Environmental Monitoring Resource Infrastructure Geospatial Resources Services

14. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 14 OGC Web Services (OWS) CYCLOPS must work on two scenarios: –The present EGEE infrastructure –The future EGEE infrastructure Roadmap: to implement OGC Web Services (OWS) on top of the Grid –Discovery/Catalog services –Access services –Processing services –Presentation services –Web Sensor services OWS are the standard de-facto e de-iure (i.e. ISO 19100) for Geospatial Community (Land Management & Earth Sciences Communities) Most mature: –Web Map Service (WMS) –Web Feature Service (WFS) –Web Coverage Service (WCS) –Catalog Service for Web (CS-W) Each service exposes several functions (operations)

15. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 15 OWS Grid implementation Web over Grid Grid-enabling OWS –Build OWS implementations which make use of Grid capabilities Grid over Web Gridifying OWS –Expose OWS as Grid computational elements (CE) which run native calls to external WS

16. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 16 Grid-enabling OWS OWS request Service register OWS URL netCDF HDF GRIB Others... OWS HTTP-GET OWS.function request netCDF HDF GRIB Others... netCDF HDF GRIB Others... netCDF HD F GRIB Others... LFC JDL translation, job submission WMProxy OWS WMS

17. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 17 Web Coverage Service (WCS) The Web Coverage Service (WCS) supports electronic retrieval of geospatial data as "coverages“ –digital geospatial information representing space-varying phenomena that relate a spatio-temporal domain to a (possibly multidimensional) range of properties A WCS provides access to potentially detailed and rich sets of geospatial information, in forms that are useful for client-side rendering, and input into scientific models and other clients A WCS provides three operations: –GetCapabilities –DescribeCoverage –GetCoverage  returns a coverage encoded in a well-known format

18. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 18 Traditional WCS WCS interface Data Subsetting Resampling DataEncoding Data Encoding RequestResponse

19. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 19 EGEE User Forum, Manchester, 11/5/2007 19 The WCS implementation Prototype running at CNR-IMAA ESSI laboratory in Prato cyclops VO registered with CIC-portal (cic.gridops.org) gLite 3.0 services made available by INFN-GRID: –VOMS server for cyclops VO at INFN-CNAF –LFC server for cyclops VO at INFN-CNAF –“production” WMS/LB at INFN-CNAF & Padova –“production” CE and SE at INFN-Padova Geospatial data used –Unidata NetCDF files (NetCDF-CF dialect). Software deployed at CE –Java TM Runtime Environment 6. –Some software libraries (NetCDF Java, …)

20. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 20 GRID WCS WCS interface Data Subsetting Resampling DataEncoding Data Encoding EGEE Grid Interface Request CE Async. response SE EGEE Grid Interface CE EGEE Grid Interface CE EGEE Grid Interface Workload Manager S S RRR D R R R D WCS Interface Result Result URL

21. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 21 GRID WCS : Technologies EGEE Grid Interface CE SE EGEE Grid Interface CE EGEE Grid Interface CE EGEE Grid Interface Workload Manager WCS Interface WEB Application Layer (Java) EGEE GRID Computing Layer (Java) Request Async. response ResultHTTP/SOAP S R R R D JDL Java Serialization S RRR D

22. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 22 EGEE User Forum, Manchester, 11/5/2007 22 Summary A prototype of Grid-enabled WCS on top of EGEE Platform has been implemented –After a typical getCoverage request we could create a workflow to efficiently run different tasks, like subsetting, resampling and data encoding in a parallel environment –The WCS server prototype is entirely written in Java TM, and does not need to be run on an UI to connect to the Grid –Some software libraries are stored in the VO software repository on the CE to speed up the job submission process Potential benefit of grid approach –Grid support of file replicas –Use of mobile code instead of mobile data –High performances working on large datasets –Scalability

23. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 23 Conclusion CYCLOPS is a project with a lot of potential  many expectations from both CP/GMES and GRID communities The project is in the phase of CP agencies system analysis, use-case analysis and system requirements analysis Some prototyping to test the present gLite potential and identify the possible required enhancements are started

24. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 24

25. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 25 BACKUP SLIDES

26. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 26 Standardization Communities GRID community –test and validate, the middleware re-engineered by EGEE and EGEE-2 in compliance with the current open standards (OGF, OMII Europe) Geospatial Information community –INSPIRE (The INfrastructure for SPatial InfoRmation in Europe ) –GEOSS (Global Earth Observation System of Systems) –OGC/ISO (Open Geospatial Consortium)  Risk and Crisis Management  Sensor Web Enablement  Earth Observation  OWS: WCS, WFS, CS/W, CTS, WPS, … Civil Protection community –GMES (Global Monitoring for Environment and Security)

27. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 27 Spatial Data Infrastructure tiers WFS WCS WMS Access and Download services Discovery services based on ISO 19115 profiles CS-W WMS Thematic Portals View services View services Presentation services Resources Processing Services

28. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 28 Geospatial Information GRID platform ---SE HostFile Block DatasetFile Coverage/Layer/Map--- Logical Aggregation Level Host name  LFN for a file block Data Aggregation levels (semantic levels)

29. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 29 Run application software which calls OWS external services –Implement a Grid application which makes use of OWS service chaining –Possible use of all Grid job types: collection, DAGs, Parametrics, MPI, etc. Minimal case: Run an existing OWS function as a Grid job netCDF HDF GRIB + + Grid Job Gridifying OWS

30. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 30 Gridifying OWS SE CE Resource Broker/ WM proxy WSDL netCDF HDF GRIB Others... Native call OWS

31. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE User Forum, Manchester, 11/5/2007 31 Virtualize Sensors Two approaches: –A new Grid sensor element  Instrument Element (like GridCC?) –Make use of OGC Sensor Web Enablement services CYCLOPS Infrastructure Spatial Data Infrastructure Services Processing Systems Infrastructure Data Systems GRID Platform (EGEE) Sensor Element Services Advanced Grid Services Environmental Monitoring Resource Infrastructure Geospatial Resources Services