1. Assessing the Maturity of Climate Data Records
John J Bates
National Climatic Data Center, NOAA/NESDIS

2. Overview
Climate variability and change are having profound effects on society
Society deserves full and open access to the data and methods used to produce climate products
Scientists often use nomenclature and methods that are difficult for non-specialists, and sometimes even specialists, to understand
Scientists must do a better job in standardizing their nomenclature and methods, and more clearly communicate with the public

3. Motivation What is at stake?
History shows that weather observations did not become useful for society until a lexicon was agreed to
The Beaufort scale did this for wind climatology and maritime commerce in the 19th century
For climate services to benefit society, they must adopt a lexicon that sets expectations for openness, process and transparency that are accessible to the public
How might we define a climate record lexicon useful to the public in the 21st century?

4. Motivation
Climate Change is a highly applied scientific field with major aspects related to regulation and societal wellbeing
Society is, understandably, demanding scientific process for generating CDRs to be open and transparent
Increasingly complex observing systems require more process control to ensure quality, access, and preservation
For example the NASA/NOAA Suomi NPP satellite is generating over 5 TB/day of data
Software Engineering is also increasingly complex and process management is required to optimize cost, schedule, productivity and quality
These benefits are greatest for larger organizations that perform tasks over many years

5. Climate Observation Best Practices
Steps to long-term monitoring
Over the last years many investigators have developed methods for seaming together observations with evolving coverage and accuracies
From these experiences, common elements are emerging on best practices for Climate Data Records
How do we capture and make available these best practices and pass them to new generations?
60S-60N ERSST Annual Anomaly ( )
60S-60N ERSST Annual Anomaly ( )
60S-60N ERSST Annual Anomaly ( )

6. Climate Observation Best Practices – Needs for Long-term Preservation
What common preservation information do we need and how to capture communicate this?

7. A Maturity Matrix for Assessing the Completeness of CDRs
The Maturity Matrix assess Software Readiness, Metadata, Documentation, Product Validation, Public Access, and Utility on a Maturity on a scale of 1-6 in each area
Details can be found in Bates and Privette 2012
DOI : /2012EO440006
These attributes monitor process, documentation, and access to CDRs.
These are essential in ensuring openness and transparency
The Maturity Matrix attempts to ensure these attributes are available for a scientific assessment of a CDR
This is not itself a scientific assessment
The climate community was invited to apply the Matrix and comment

8. Community Input on the Maturity Matrix
The Maturity Matrix has been evaluated by a GCOS Workshop, EUMETSAT and the EU FP7 project CORE-CLIMAX , the ESA CCI, and the CEOS SST Virtual Constellation –
EUMETSAT CORE-CLIMAX Workshop in January 2014 performed a self assessment of 30 CDRs (23 satellite, 6 in situ, one re-analysis)
They updated the maturity matrix based on feedback and use and called it a system maturity matrix (in order to separate this from an assessment of the CDR science)

9. Community Input on the Maturity Matrix
CORE-CLIMAX workshop results:
Evaluation and Quality Control needs to consider both scientific and process quality expressed as maturity
Because the Maturity Matrix always needs interpretation it shall not be used for a beauty contest by adding up or averaging scores and ranking
Metrics for scientific and process oriented assessments need further development
Process maturity should periodically be assessed for ECV Climate Data Records entering climate (change) services

10. CDR Maturity Matrix – Next Steps
The CEOS-CGMS Joint Working Group on Climate is asking members to provide further evaluation of the CORE-CLIMAX System Maturity Matrix prior to recommending adoption (by later this year; NASA is a member of both organizations)
The WCRP Data Advisory Council is considering guidelines for scientific assessments and such assessments should be considered separate from system or process maturity
Software assessment tools (called static analysis tools) should be used to evaluate code to minimize complexity and remove excess code

11. CDR Maturity Matrix – Final Thoughts
The stakes in climate change are too high to assume a standard research approach to the creation of CDRs. Society is demanding more documentation, openness, and transparency.
It is imperative that we, the climate community, responds with quantifiable metrics that inform society of both the scientific quality and process maturity of CDRs.