1. Earth Observation Science CEOS Guidelines for the validation of satellite SST Observations David T Llewellyn-Jones &Gary K Corlett Space Research Centre University of Leicester

2. Need for Standards & Traceability “It is a truth, universally acknowledged, that a young man, in possession of a good fortune, must be in want of a wife.” Jane Austen: “Pride and Prejudice” Opening sentence quantitative geophysical measurements for Climate Monitoring should use reproducible and accepted procedures as well as traceable standards for accuracy.”

3. CEOS Objectives  To optimise the benefits of spaceborne Earth observation - through cooperation of its Members in mission planning and in the development of compatible data products, formats, services, applications and policies;  To aid both its Members and the international user community by inter alia, serving as the focal point for international coordination of space-related Earth observation activities, - including the Group on Earth Observations and entities related to global change;  To exchange policy and technical information to encourage complementarity and compatibility among spaceborne Earth observation systems currently in service or development, - and the data received from them; issues of common interest across the spectrum of Earth observation satellite missions will be addressed.

4. WG on Cal/Val Objectives  To enhance coordination and complementarity,  To promote international cooperation, and  To focus activities in the calibration and validation of Earth observations for the benefit of the CEOS members and the international user community. Specific objectives include: Sensor-specific calibration and validation - To document and establish forums for the assessment and recommendation of current techniques and standards for pre- and post-launch characterizations and calibration. Geophysical validation - To document and establish forums for the assessment and recommendation of techniques for validation of geophysical parameters derived from Earth observation satellite systems.

5. CEOS WGCV/GEO Workshop Held at NIST, Gaithersburg Md., May 7-9, 20008 GEO/CEOS Workshop on Quality Assurance of Calibration and Validation Processes: Establishing an Operational Framework. The ultimate goal of this workshop was to establish an operational framework based on a set of 'key guidelines' to govern / facilitate data interoperability and in particular the Quality Assurance of Cal/Val processes.

6. Available from the Cal/Val portal at: http://calvalportal.ceos.org/ through the sections marked “Resources” → “Workshops”.

7. What Does the Document Say?  The Guidelines “must of course address the needs and / or guidelines established within the other themes. The key themes are: 1) Communication and Education, 2) Data Policy, and 3) Data Quality.  Each theme has a guiding principle that must be met to achieve interoperability and a set of key guidelines to support this operationally.” Note: These are intended as guidelines for best practice – nobody is proposing any form of certification

8. Communication and Education The guiding principles for this theme are as follows: Guideline 1: Use and encourage the use of internationally agreed terminology where possible, which can be found in the endorsed dictionaries listed on the GEO/CEOS Cal/Val Portal, or at least provide a clear definition. H Guideline 2: All appropriately quality assured (CEOS endorsed) Cal/Val data and associated supporting information needed to support interoperability should be made fully accessible through a single web portal. Guideline 3: To facilitate “search and find” of documents, guidelines etc the CEOS document cataloguing/numbering system needs to be used for all Cal/Val related documents. DP Guideline 4: Seek to encourage the uptake of “best practise” methods and facilitate rapid entry and contribution of new groups to support global Cal/Val needs when developing or carrying out Cal/Val related activities Guideline 5: …..

9. Data Policy The guiding principles for this theme adf as follows: Guideline 1: Establish common format for Cal/Val data exchange. Guideline 2: Code of practise for Cal/Val data providers and users to ensure free access and unencumbered access whilst providing recognition for the efforts in its generation Guideline 3: A common set of content and means of association of metadata to Cal/Val data sets. Guideline 4: …..

10. Data Quality - Guiing Principle All data products must have associated with them a “Quality indicator” (QI) based on documented quantitative assessment of its traceability to internationally agreed reference standards. This requires all steps in the data product delivery chain (collection, processing and dissemination) to be documented with evidence of their traceability. This can be achieved by following the guidance document: CEOS GEN/DA/001 (Annex 1). To facilitate the above, guidelines are required to support the following generic (non-sensor specific) processes/activities: Establish QI for a sensor delivered data product Organisation and analysis of “comparison of measurements” Establishing a reference standard Evaluating Uncertainty of Measurement Writing and Validating models and algorithms Writing and Validating software Establishing and assessing “quantitative evidence of traceability” … Isn’t this basically, the SSES??

11. Data Quality – the Guidelines Guideline 1. A guide to content of a documentary procedure to meet the Quality Assurance requirements of CEOS Guideline 2: To establish a QI for satellite sensor delivered data products (further) guidelines are recommended: Guideline 3: organisation and analysis of a measurement comparison H Guideline 4: Establishing a reference standard – suitability and acceptance H Guideline 5: Evaluating and expressing uncertainty of measurement H Guideline 6: Writing and validating models and algorithms H Guideline 7: Writing and validating software H Guideline 8: Establishing and assessing quantitative evidence of traceability H Guideline 9: ….

12. Data Quality – that Guideline 2 1. Pre-flight: All Data producing aspects of a sensor should, where possible, be traceably calibrated/evaluated against international standards (ideally SI) at system level and as integrated “end-to- end”, and where appropriate, witness samples of key calibration sensitive components should be stored and maintained for later evaluation. 2. Post-Launch: Sensors should seek to demonstrate their performance for the following three aspects:  Characteristics compared to pre-flight specifications  Biases to similar in-flight sensors  Stability of delivered data products during mission life and its correlation with data from historical and future missions.  This can best be achieved through comparison to a CEOS endorsed reference standard using an endorsed methodology.

13. What should GHRSST Science Team do? GHRSST is already doing much of what CEOS is seeking to establish: –Emphasis on Product rather than specific sensors –Error analyses specific to each sensor –Agreed reference standards GHRSST is a potential Pilot Scheme for achieving standardisation and traceability Suggestion – this idea should be promoted within CEOS (via IVOS and WGCV with objective of ensuring that CEOS guidelines are acceptable and implementable –Also - support and help steer the next radiometer inter-comparison Does the GHRSST Science Team agree?

14. Preferred Approach Top-down? or Bottom-up?

15. Available from the Cal/Val portal at: http://calvalportal.ceos.org/ through the sections marked “Resources” → “Workshops”.

16. , launched March 2002 Sun-synchronous orbit, 1030 local time daylight descending AATSR ENVISAT Europe’s largest satellite

17. Group on Earth Observations / Committee on Earth Observation Satellites Quality Control / Assurance and Best practice Guidelines on Calibration & Validation Processes “A Documentary Framework” April 2008 Version 0.9 This draft document was prepared and compiled by: Pascal Lecomte (ESA)Gregory Stensaas (USGS)Nigel Fox (NPL / BNSC)Gyanesh Chander (USGS / SGT)Marie-Claire Greening (Greening Consulting / ESA)Bojan Bojkov (NASA)Giuseppe Ottavianelli (ESA)