Big Data in Earth Observation Cristiano Lopes 21/03/2017

Diversity of EO Data at ESA ESA Missions (active + legacy) ERS, ENVISAT, GOCE, Cryosat, SMOS, Swarm, Proba-1, Proba-V Copernicus Missions (active) Sentinel 1, Sentinel 2, Sentinel 3 Third Party Missions (active + legacy) Landsat, SPOT, ALOS, OceanSat, Aura, OMI, DMC, IRS, KOMPSAT, etc, etc, etc High-level Summary: 46+ Missions 68+ Instruments (with countless different modes)

Quantity of EO Data at ESA ESA + TPM Missions (active + legacy) Archive: ~3-5PB, ~5Million items Distribution: ~3 PB, ~6 Million items Copernicus Missions (active) Archive: ~8 PB, ~4 Million items Distribution: ~2 Million items Rate of publication (last 3 months): ~470TB, ~520K items Rate of download (last 3 months): ~6PB, ~8M items ~ 1CD (820MB) / second ~ 3 items / second (Expected to increase even more once both A&B units are fully active in the 3 initial Sentinels.)

EO Data Different Perspectives Different types of Sensor and Resolutions Radar (X, L, C) Optical (multispectral and panchromatic) Altimetry Atmospheric LIDAR Magnetometers Application Domains / Earth Science Topics: Agriculture Atmosphere Solid Earth Water Land Oceans and Coasts Snow and Ice Natural Disasters

EO Data is only the start… More is needed. Single EO image Multiple EO images, processed Plus population density… Urban growth Credits: Urban Thematic Exploitation Platform

How to fit it all? Ensuring interoperability The challenge is to fit “Big”, “Linked” and “Geo” data in an interoperable manner so that it solves the problems of the end-user… For the “Linked Data” domain we are still at a very early phase  learning and understanding Progress made in the “Big Data” domain, leasing to an on-going shift in paradigm

“Bring the user to the data” The old approach of getting the user to download a copy of satellite data to his environment is rapidly becoming un-manageable The new paradigm is to have the user perform the analysis “on-line”, i.e. get the user to upload the algorithm instead of downloading the data This involves not only Earth Observation data but also links with in-situ data, social-economic elements, etc On-going initiatives like the Thematic-Exploitation Platforms, address these topic by providing a framework were different data co-exist for example, EPOS (sysmologogic data, near fault observatories,

Big Linked Data and EO Exploitation Platform Interoperability in this domain is mandatory, examples: data discovery and access (considering Linked Data, not only Spatial) processing, algorithm executing this must follow a write-once, run anywhere approach User AuthN & AuthZ must be shared. How can this all fit together? What can ESA offer to make EO data more easily available to the Linked Data community?

Linked data – possibilities ? Collection / Dataset discovery based on similarities, linking: data from same type of sensors (e.g. SAR ERS,ENVISAT) data with same topology (e.g. Optical Multispectral Medium Resolution, Landsat and S2) complementary data for application (e.g. Land Use, S2 and Population) Provenance linking: Results to input data (imagery, social economics, etc) Results to scientific background (and back)

Ground Segment System Engineer Earth Observation Programmes Cristiano Lopes Ground Segment System Engineer Earth Observation Programmes Phone: +39 06 941 80735 Email: cristiano.lopes@esa.int