The role of the ISIC facility for Climate and Environmental Monitoring from Space (CEMS) in the development of Quality Assured Datasets and Downstream Services from Earth Observation Data Sam Almond, Logica & Kevin Talsall, Telespazio Vega

Contents 1.What is CEMS? 2.Why is CEMS needed? 3.What CEMS offers? 4.Overall benefit to the UK 5.How CEMS enables Quality Assurance (QA) 6.CEMS QA / Data Integrity projects 1.TruDAT 2.Data Integrity Pilot Project 7.Looking forward 8.Conclusions 212 September 2012RSPSoc 2012

1. Overview of CEMS 312 September 2012RSPSoc 2012 CEMS is an ISIC facility to enable innovative climate and environmental services to be developed, hosted and served to users - These services will be for widespread use by public and private sector organisations in UK and internationally, supporting excellent science and generating commercial revenues - CEMS will consolidate the UK’s leadership position in climate science, calibration/validation & downstream applications, and will position the UK for key opportunities e.g. ESA CCI, GMES climate services What is unique about CEMS? - It will make the data from satellites, ground and airborne sources accessible in native & common formats - It will provide tools and information to ensure the integrity of the data – science and commercial users will be able to rely on the information - It will provide a scalable model by hosting services and applications close to high performance computing facilities and very large quantities of source data

2. Why do we need CEMS? 412 September 2012RSPSoc 2012 Climate and global environmental change monitoring from space is an increasingly exciting yet challenging task, leading to potential benefits in science understanding of our planet, as well as commercial opportunities for UK companies. The available volume of high-quality Earth observation (EO) data is increasing exponentially, as new satellites are being developed globally from both public and private funds. These observations must be processed, quality controlled and managed prior to visualisation and analysis, both independently and in conjunction with state-of-the-art model predictions. Leading-edge data processing, management and dissemination capability at the ISIC Facility in Harwell will enable the UK to maximise the value of these data and create economic benefit across a wide spectrum of UK interests.

3. What CEMS offers At its heart CEMS is a facility that provides: High-powered processing and data storage: Processing: 240 real cores (480 virtual cores) & 2 Terabytes (2000 Gigabytes) of memory Storage: 1.7 Petabytes (1.7 Million Gigabytes) Accessible through Virtual Machines allowing access from anywhere Data processing: Users can process large amounts of data efficiently & cheaply Application hosting: organisations can take advantage of CEMS processing and storage capacity and use CEMS to reach a wider audience for their applications Data Integrity Facility: provides key QA information to help users determine if their data is fit for purpose Consultancy: access to the wide-ranging skills of the ISIC partners Visualisation and Application Centre: utilise existing ISIC facilities such as the Applications Innovations Centre and the Visualisation Suite 512 September 2012RSPSoc 2012

3. CEMS Technical Implementation Public and Commercial Cloud Infrastructure Hardware – data storage [NCEO and Commercial Data] and processing App 2 App 3 App N App 1 … Business and research user communities Data Access Quality Services Core Services Applications Cloud Management Services Data Processing

3. The CEMS Overview

4. Overall benefits to the UK The implementation of CEMS through ISIC enables the infrastructure to act as a platform for science and commerce to develop further activities which will bring economic benefit to the UK. In science, CEMS will support evidence on how Earth system components are changing, through the provision of long-term, quality-controlled data sets for climate and geo-hazards research. - In commerce, the CEMS core infrastructure will provide the means to develop commercial applications with a focus on UK sectoral strengths: - Financial markets (carbon trading) - informed use of climate and carbon data - potential revenues/benefits throughout the value chain, rising from £1-2M in 2013/14 to more than £100 million p.a. by Insurance/risk markets – modelling/services using improved knowledge of environmental data; risk/hazard forecasting, etc. - Environment/energy markets – use of UK and global datasets to inform decision making and forecasting - Services relevant for many public sector users e.g. Defra, DECC, etc. 8

5. How CEMS enables QA in EO QA is important to enable users to make judgements on the quality, provenance and suitability of any potential datasets The Data Integrity study recommends that CEMS provides: Adopt a DI framework based on QA4EO principles A DI Helpdesk and Consultancy Service to assist users in optimally utilising EO and climate data A Certification Service to allow for quality certification of data The ability to create bespoke tools to enable the completion of ‘missing’ data integrity information not provided by data providers. User access to such tools to enable users to perform their own comparative validation exercises of data within CEMS 912 September 2012RSPSoc 2012

5. The CEMS QA / DI process 1012 September 2012RSPSoc 2012

6. CEMS QA / Data Integrity projects 1112 September 2012RSPSoc 2012 CEMS has already successfully bid for QA / DIF projects :  TruDat – led by NPL: Detailed data integrity procedures will be developed using optical & SAR  Data Integrity Pilot Project – led by Vega: Use SST data as a basis to produce a prototype added value product and pilot data integrity service.  HIVACROM – led by Logica: Crop monitoring service using EO data to accurately estimate crop yields (initially potato & sugar beet)  CHARM (Characterisation of metadata) (FP7) - Enable climate- relevant data to be consistently characterised as to their provenance, errors, biases and consistency; develop tools to enable users to access information to determine dataset fitness of purpose.

6. TruDAT TRUsted DATa for Carbon, Science and Finance TSB project lead by NPL, partnered by University of Sheffield & Logica Objective is to provide a quality develop strategy / methodology to establish rigorous Quality Indicators (QI based on QA4EO) on CEMS delivered EO data/products 1212 September 2012RSPSoc 2012 Forest Carbon Case study QI used to enable harmonious, consistent and seamless choice of data sources (Optical and SAR), processing steps and validation data Prototype Automation of process within CEMS and also provide visualisation tools Applications include carbon trading, validating climate data and low carbon technologies

6. Data Integrity Pilot Project TSB project lead by Telespazio Vega, partnered by NCEO & Logica Objective is to create a pilot demonstrator of a Data Integrity Facility within CEMS using a sample dataset of ESA’s Envisat Advanced Along Track Scanning Radiometer (AATSR) data. It will address 3 key Data integrity recommendations: 1312 September 2012RSPSoc Establish a Helpdesk with capability to answer user questions related to the DI data provided with the dataset 2.Establish a DI Certification process through an appropriate metadata schema, by which the sample dataset may be assessed 3.Develop an environment and associated Tools specifically developed for assessing the AATSR data and the output information fed into the DI certification process

7. Looking forward: CEMS strategic goals 1412 September 2012RSPSoc 2012 Integrated Delivery of Climate Data Major processing & delivery node of timely, robust and synthesised satellite information products from ESA’s CCI CEMS platform/infrastructure to help enable the UK to become the world leader in climate science, validation, adaptation, mitigation and commercial climate services GMES Service Provision CEMS to be a leading European node for Sentinel data and services, initially ensuring the GMES Sentinel 1 & 2 data supports a unique set of tools and services. Stimulating Commercial Applications ISIC & CEMS to provide the open innovation environment and infrastructure to stimulate the development of a vast array of commercially hosted applications Next Generation Visualisation Facilities Enable users to visualise a wealth of data sets using state-of-the-art extraction and animation tools to monitor the health of our oceans, validate carbon assets and inspire our next generation of young scientists.

8. Conclusions CEMS provides a unique facility to enable innovative climate and environmental services to be developed, hosted and served to users It positions the UK to take a leadership role in key UK and international opportunities in the climate and environmental services domain CEMS will help foster growth in the downstream applications that are predicted to increase with availability of vast quantities of EO data A key principle of CEMS is focused on the Quality Assurance and Data Integrity of EO and climate data, allowing users to make informed choices about their data CEMS has already begun providing additional QA data, as can be seen by the QA / DIF projects started 1512 September 2012RSPSoc 2012

Questions 16