1. CF Metadata Conventions: Governance, Support, and Future Karl E. Taylor Program for Climate Model Diagnosis and Intercomparison Lawrence Livermore National Laboratory Presented to the GO-ESSP Community Workshop Paris, France 11 June 2007

2. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Intro to CF metadata conventions Climate and Weather Forecast Metadata that makes data files self-describing Examples of CF metadata à Coordinate information needed to locate data in space and time à Further grid information (e.g., grid cell bounds, area) à Standard names – in conjunction with other attributes helps users determine whether data from different sources are the same physical quantity and helps distinguish variables within archives à Some processing info. (e.g., zonal mean, climatological processing)

3. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Brief history of CF Started with “COARDS” standard, and developed through efforts of a handful of volunteers 2003: CF 1.0 released 2005: CF white paper discussing future governance circulated 2006: White paper revised and presented to WCRP WGCM 2006: Current governance structure established 2003-present: increasing acceptance of CF and adoption by several community coordinated projects: à IPCC AR4 archive conforms à PMIP, HTAP, regoinal groups, seasonal groups, ENSEMBLES …

4. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Governance structure Original CF “authors” turned over control to two working committees: à Conventions Committee à Standard Name Committee Committee work done via email and web discussion (“Trac”) Support provided by: à BADC: Alison Pamment (50%) – standard names à PCMDI: Kyle Halliday (20%) – web site support WCRP/WGCM has been approached to appoint a CF Governance Panel charged with responsibility for stewardship of CF.

5. PCMDI BASC / CRC 17 May 2007 K. E. Taylor What happened at last GO-ESSP meeting? Discussed CF governance & procedure for making changes Some pressing CF issues were identified and in some cases individuals volunteered to try to make progress à Augmented grid information, especially for less structured grids. à Relationship between CF and GIS grids & metadata à Standard name issues (including dialects, profiles, hierarchy structures) à Relationship between CF and netCDF4 à CF handling of in situ observations à How to handle discovery information à Sample datasets and reference implementations

6. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Agenda for today Governance issues – what needs fixing? Underlying principles – any adjustments needed? Technical issues – what’s been done? Can we make progress on any of the discussion topics? Supportive software – status reports Any additions? Other concerns?

7. Conventions Committee: Support and Governance Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007

8. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Current committee members Karl Taylor (chair) Kyle Halliday (secretary) Balaji John Caron Jonathan Gregory Tom Gross Steve Hankin Jamie Kettleborough Russ Rew Rich Signell

9. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Terms of reference Further develop (revise) CF conventions Consider implementation of CF metadata conventions in other file formats (besides netCDF) Determine what is meant by CF conformance

10. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Current procedure for modifying/extending CF Anyone can propose a change Discussion opens & a member of the committee volunteers to moderate A provisional resolution is reached by consensus and summarized by moderator Reference files are produced illustrating the new feature If not a trivial change, trial implementations are carried out. If problems emerge iterate When consensus reached, the proposed change is provisionally accepted. (advanced implementers may begin adoption) Collection of changes evaluated as a package leading to new CF versions released by consensus Test reference files become part of the “test data”

11. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Problems encountered. Progress on some proposals, but few have been carried through to implementation. Contrast with standard names committee à Few actions have taken place à With a couple of exceptions, the volunteer moderation of discussion topics has floundered Procedure for introducing changes outlined in white paper: à Email à Web-based Trac system Jonathan’s thoughts

12. Issues Concerning Underlying Principles of CF Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007

13. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Underlying principles Special needs of observations Should the scope of metadata accommodated by CF include: à Description of experiment à Description of model à Description of higher-order grid-characteristics (e.g., relationship between u & v grids, which grid cells are connected to which) Competing needs of data producers & application developers à CF extensions can be implemented faster by producers who sometimes are impatient for changes to be formally adopted Should requirement be relaxed that all CF metadata reside in the same file as the data itself? Bundling of common definitions of a quantity into I.D.’s (defined by a combination of attributes including the standard name + others) Aggregation layers CF profiles

14. Miscellaneous Technical Issues Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007

15. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Miscellaneous technical issues Ensembles (including realization_weight and other issues): Alison Time axis “issues” for forecasts: Jonathan Definition of the “geoid”: Alison Use of “anomaly”: Alison or Jonathan Subgrid variation description issues (use of “where”): Jonathan Time and calendar issues for paleoclimate simulations: Jean-Yves? Balaji?

16. The Future of CMOR Karl E. Taylor Program for Climate Model Diagnosis and Intercomparison Lawrence Livermore National Laboratory Presented to the GO-ESSP Community Workshop Paris, France 11 June 2007

17. PCMDI BASC / CRC 17 May 2007 K. E. Taylor The Climate Model Output Rewriter (CMOR) Provides a uniform interface for contributing CF-compliant data (adhering to “good practices”) to the growing number of MIP’s (e.g., CMIP, AMIP, CFMIP, PMIP, APE, HTAP) For IPCC AR4, nearly all model output in the archive was written through CMOR. Assures CF-compliance and all MIP-established output requirements are met. Traps many metadata errors when files are produced. See http://www-pcmdi.llnl.gov/software/about_software.php

18. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Project-specific CMOR tables facilitate and ensure consistency of model output from contributing groups Proper specification of several coordinate attributes, including: à Correct standard name à “axis”, “positive”, and “formula_terms” attributes, as appropriate Proper specification of several variable attributes, including: à Correct standard name à Required dimensions à “cell_methods” attribute A capability to à Reorder axis order à Reverse axis direction (or translate longitude dimension) à Convert units (through udunits)

19. PCMDI BASC / CRC 17 May 2007 K. E. Taylor CMOR flagged common errors, including Pointing out when required metadata are omitted. Rejecting incorrect metadata (wrong units, inadmissible attribute values, etc.) Rejecting inconsistent coordinate dimensions passed by user to CMOR. Rejecting non-monotonic coordinate values or inconsistent boundary values, as passed by user. Rejecting values that are clearly unrealistic (likely indicating improper units conversion or incorrect sign).

20. PCMDI BASC / CRC 17 May 2007 K. E. Taylor What changes will be implemented in the next 6 months? Station data (needed, for example, by the HTAP project), including à Station names à Station locations Output from regional models à With logically rectangular grids à But with non-cartesian longitudes and latitudes à i.e., data(i,j), lon(i.j), and lat(i,j)

21. PCMDI BASC / CRC 17 May 2007 K. E. Taylor Longer term prospects for CMOR Difficult to see extending CMOR to write the grid- description files mandated by the GridSpec proposal. Probably possible, however, to write the actual data files through CMOR (and leave it to other software to produce the special GridSpec files).