1. 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision of large data sets to multiple user communities 14-16 July 2014, Boulder CO MONITORING WEATHER AND CLIMATE FROM SPACE

2. 2 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Mission Statement We are the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT). EUMETSAT is an intergovernmental organisation and was founded in 1986. Our purpose is to supply weather and climate- related satellite data, images and products 24 hours a day, 365 days a year to the National Meteorological Services of our Member and Cooperating States in Europe, and other users worldwide. www.eumetsat.int

3. 3 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETSAT Headquarters (Darmstadt, Germany)

4. 4 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETSAT Data Services - High Level Overview

5. 5 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETSAT Data Services - Data Flows EUMETSAT GROUND INFRASTRUCTURE

6. 6 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int  Single Access Point  One stop shop  Data discovery  User registration (>10.000)  Subscription for data services:  Near-real time data  Online Data Access  Long Term Archive  Additional  Image Gallery (WMS/ WCS)  User Notifications  Operational Real-Time Services Status Indicator EUMETSAT Earth Observation Portal

7. 7 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETCast Dissemination Near Real Time data delivery to users in 3 continents via Satellite Users (2/2014) 4 3 2-10 11-50 51-200 201-500 1 User 500+ 0 User 3,280 users worldwide, 4168 stations 5 new users per month EUMETCast Americas SES 6, C-BAND EUMETCast Africa EUTELSAT 5 WEST A, C-BAND EUMETCast Europe EUTELSAT 9 A, KU-BAND

8. 8 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETCast Dissemination: Integrating terrestrial dissemination

9. 9 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int MULTICAST ACCROSS GÉANT and NREN INFRASTRUCTURE Exploring the multicast usage on GÉANT infrastructure in Europe and their world wide connectivity;  2011 Proof of concept on terrestrial multicast on GÉANT test bed  2012 3 month feasibility study on terrestrial multicast via GÉANT  2013 Multicast test to KMA Korea  2014 Multicast test to NOAA (Boulder)  2014 Start of 2 year prototype emulating operational scenarios of multicast across Europe and our overseas partners using GÉANT, NREN, I2 and Teint infrastructure.

10. 10 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETCast Terrestrial real-time monitoring -Multiple Multicast channels -SSM mode -20-30Mbit/s -FEC ¾ -Closed loop monitoring based on successful file receipt -Packet/channel based ack/nack retransmission possible but not yet enabled Acknowledgements: N-WAVE&NOAA Alex Hsia, David Hartzell

11. 11 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETCast Dissemination: Low cost, off the shelf Receiving Station DVB PCI Card EKU Antenna Laptop/ PC Software DVB USB DVB Router Typical total cost of a station < 2000 Euros

12. 12 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int EUMETSAT Long Term Archive  Archived data starting 1981  All EUMETSAT Satellite data including reprocessed data  > 1 Petabytes in the archive for MTP, MSG, Metop  Retrieval/Ingestion: 10 to 1  Long-Term Data Preservation in practice since 1995;  Disaster resilient - 3 rd Copy  Access via EUMETSAT EO Portal

13. 13 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int What we are working on…

14. 14 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Outlook – Enhance Future abilities in terms of Scalability Flexibility, Responsiveness to emerging needs, Resource efficiency While aiming for: Better sharing of resources, Avoidance of data islands, Reducing costs Fit the organisational structure

15. EUM/TSS/VWG/14/762641, 7 July 201415 Many aspects to consider => Challenge Viewing Angle

16. 16 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Outlook - Product delivery and management Generation of user tailored products Efficient for write once, read many times Suitable products for user needs and dissemination infrastructure (bandwidth) Avoid/ minimise processing where possible Efficient for slicing Usage of Format evolutions Secure the raw data (golden copy) Initially smaller products which increase over time Reprocessing to re-build formats

17. 17 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Outlook - Bringing data to Users Adaptive and flexible services to deliver data to users/ communities (e.g. EUMETCast) Community driven formats optimised for dissemination mean Serving large user groups from different communities Managing data policies in accordance with data provider requirements via a data policy and legal framework support by the related technical mechanisms

18. 18 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Outlook - Bringing Users to data  Standardisation activities in relation to Discovery, Access and Retrieval (DAR) of data  Strong focus on interoperability and ease of user experience  One stop shop experience for discovering services and data  Facilitating the Interoperability of portals by exploiting standards (OGC, INSPIRE)  Enhance Long Term Archive, Online Data Access  Outreach activities  Exploiting capabilities to execute user algorithms on data  Post-Processing, Hosted Processing

19. 19 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Outlook - Information to Users Delivering Information to users rather than data Processing where the data is located Service Interfaces instead Facility interfaces Application of (private) cloud principles

20. 20 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Thank you for your attention