NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, March 29, 2005 Presentation for NOAA’s Climate Working Group Tom Karl, Observations and Analysis Climate Program Manager John Bates, SDS Co-Lead Mitch Goldberg, SDS Co-Lead 1 NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures and Linkages

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Outline Background Defining Scientific Data Stewardship and CDRs NOAA Scientific Data Stewardship Program Governance and Management Program Drivers and Performance Measures Linkages to Other Programs Questions?

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Climate Goal/Climate Program Element 1 – Observations and Analysis Scientific Data Stewardship (SDS) Funds to support SDS Reprogrammed from EDSM (ESDIM) President’s FY06 Total EDSM Budget: $9.384M Funding shared among several programs including SDS SDS from EDSM ~$2.5M OGP’s Applied Research Center for Data Set Development ~ $1.4M (Maybe more or less, depending on FY06 distribution) FY06 PBA 100% Requirement for SDS is $3.5M FY07 - FY12 100% requirement substantially more Background

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, NOAA’s Scientific Data Stewardship rooted in NRC dialogue and reports NOAA/NRC SDS leads –Bates –Goldberg Background

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Data Stewardship is a subset of Data Management and consists of the application of rigorous analyses and oversight to ensure that data sets meet the needs of users (NOSC definition). For environmental satellite observations, SDS priorities include: A.Observing System Performance Monitoring i.Documenting measurement practices and processing practices (metadata) ii.Providing feedback on observing system performance, including recommending corrective action for errant or non-optimal operations. B.Climate Data Records i.Reprocessing (incorporate new data, apply new algorithms, perform bias corrections, integrate/blend data sets from different sources or observing systems) ii.Inter-comparison of data sets for validation Defining Scientific Data Stewardship

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Climate Data Records Notional Functions of Scientific Data Stewardship for Climate Network Performance Monitoring Observations & Metadata Analyses and Quality Control Feedbacks Stewardship Teams Archives Climate Analyses Reference Data Sets (Reprocessing) Climate Quality Products Scientific Data Stewardship Real time and retrospective management of climate data Model Re- analyses Defining Scientific Data Stewardship

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Sensor Data Records (SDRs) Data (Direct & Remotely Sensed) Fundamental Climate Data Records (FCDRs) Thematic Climate Data Records (TCDRs) Climate Data Records or Homogenized Time Series Homogenization and Calibration Time-tagged Geo-Referenced Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Converted to Bio-Geophysical Variables Defining CDRs Climate Data Records

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Climate Working Group NOAA Science Advisory Board Scientific Data Stewardship CLASS Working Group Scientific Data Stewardship Program Management Research Climate Data Science Teams FCDR Teams Observations Scoping Requirement Systems C 2 D 2 NCDC ORA FY06 EDSM $2.5M C 2 D 2 $1 to $1.5M Governance and Management Structure TCDR Teams R&D Products and Services Theme Areas NOAA (5-10%) External (1-2%) NOAA, Other Agencies External Scientific (~90%) NOAA Climate Program & Climate Board NOAA Observing System Council Operational CDR Generation & Data Mgmt. CDR Generation NOAA Other Agencies Universities Private Sector CLASS NOAA IT Infrastructure operators Research to Operations Currently existsFY05FY06 NOAA’s Scientific Data Stewardship Program NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages 8

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Program Drivers What is ….. –Changing –Causing the Change –The Impact $

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Program Drivers Climate Program Essential Global Climate Variables (from CCSP & GCOS) The following essential atmospheric variables are required over land, sea and ice: 1.1 Surface a. Air temperature b. Precipitation c. Air pressure 1.2Upper-air a. Earth radiation budget (including solar irradiance) b. Upper-air temperature (including MSU radiances) c. Wind speed and direction d. Water vapor e. Cloud properties 1.3Composition a. Carbon dioxide b. Methane c. Ozone *Including nitrous oxide (N 2 O), chloroflurorocarbons (CFCs), hydrofluorocarbons (HFCs), sulpher hexaflouride (SF 6 ), and perfluorocarbons (PFCs). d. Surface radiation budget e. Wind speed and direction f. Water vapor d. Other long-lived greenhouse gases * e. Aerosol properties 1. Atmospheric Variables g. Evaporation & evapotranspiration

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Program Drivers Essential Global Climate Variables (from CCSP & GCOS) 2.1 Surface a. Sea-surface temperature b. Sea-surface salinity c. Sea level d. Sea state e. Sea ice f. Current g. Ocean color (for biological activity) h. Carbon dioxide partial pressure i. ocean surface wind & wind stress j. Surface air temp/humidity k. Precip (fresh water/salinity flux) l. Evaporation m. Fresh water flux from rivers & ice melt n. CO 2 flux across the air sea interface o. Geothermal heat flux – ocean bottom 2. Ocean Variables 2.2Sub-surface a. Temperature b. Salinity c. Current d. Nutrients e. Carbon f. Ocean tracers g. Phytoplankton

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Program Drivers Essential Global Climate Variables (from CCSP & GCOS) a. Snow cover b. Glaciers and ice caps c. Permafrost and seasonally-frozen ground d. Albedo e. Land cover (including vegetation type) 3. Terrestrial Variables f. Fraction of absorbed photosynthetically active radiation (FAPAR) g. Leaf area index (LAI) h. Biomass i. Land surface temp

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Performance Measures for CDRs (PERFORMANCE) SOCIETAL IMPACTS Time-dependent biases SOFTWARE ENGINEERING CRITERIA SCIENTIFIC CRITERIA Reproducibility Multiple Observing Systems Multiple analysis teams Error structure Metadata Data formats and data models Data archive & access Standards, NARA, EGDC, EFA, etc. Version control New knowledge Outreach & network building New tools & techniques New products & services Assessments & decision support

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Century Multi- Century to Multi- Millennia Decadal Linking Ice Ages to Space Age Multi- year Minor Observing Changes Major Observing Changes Direct versus Indirect Observables

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Performance Measures for Climate Data Records All CDRs address time-dependent biases and random errors in the data set and are reproducible by independent analysis teams. Bronze --- Bronze --- A single time series produced by a single analysis team from a single observing system (e.g. MSU) or Silver or Gold level failures to meet Silver --- Silver --- Multiple time series: – produced by multiple analysis teams based a common observing system OR –produced from multiple independent observing systems by a single analysis team (e.g. MSU vs. Radiosondes) Gold --- Gold --- Multiple time series produced by multiple analysis teams from multiple independent observing systems. Note: Trends within data sets must be larger than the differences among data sets to reach Gold or Silver status. Scientific Criteria

Essential Climate Variables Multi-Century to Multi Millennia CenturyDecadalMulti-Year 1.1 Surface a. Air temperature b. Precipitation c. Air pressure d. Suface radiation budget e. Wind speed and direction f. Water vapor g. Evaporation & evapotranspiration 1.2 Upper Air a. Earth radiation b. Upper-air temp c. Wind speed & dir. d. Water vapor e. Cloud properties 1.3 Composition a. Carbon dioxide b. Methane c. Ozone d. Other greenhs. Gas e. Aerosol properties GoldSilverBronzeNo CDR Notional Status of Global Scale CDRs - Atmosphere 16 Scientific Criteria

Essential Climate Variables Multi-Century to Multi Millennia CenturyDecadalMulti-Year 2.1 Surface a. SS temp b. SS salinity c. Sea level d. Sea state e. Sea ice f. Current g. Ocean color h. Carbon dioxide i. Ocean surface wind & wind stress j. Surface air temp/humidity k. Precip (fresh water/salinity flux) l. Evaporation m. Fresh water flux from rivers & ice melt n. CO 2 flux across the air/sea interface o. Geothermal heat flux-ocean bottom 2.2 Sub-Surface a. Temperature b. Salinity c. Current d. Nutrients e. Carbon f. Ocean tracers g. Phytoplankton GoldSilverBronzeNo CDR Notional Status of Global Scale CDRs - Ocean 17

Essential Climate Variables Multi-Century to Multi Millennia CenturyDecadalMulti-Year a. Soil moisture b. Glaciers & ice caps c. Permafrost & seasonally frozen ground d. Albedo e. Land cover (including vegetation type) f. Fraction of absorbed photosynthetically active radiation (FAPAR) g. Leaf area index (LAI) h. Biomass i. Land surface temp GoldSilverBronzeNo CDR Notional Status of Global Scale CDRs – Terrestrial Variables 18 Scientific Criteria

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, a. fire occurrence b. volcanic effects (on surface) c. biodiversity d. chemical (fertilizer/pesticide & gas exchange) e. waste disposal & other contaminants f. earthquakes, tectonic motions g. nutrients and soil microbial activity h. coastal zones/margins i. erosion, sediment transport j. land surface structure & topography 3. Terrestrial Variables a. Organic & inorganic effluents (into ocean) 2. Ocean Variables Secondary Variables External Forcing or Feedback Variables Scientific Criteria

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Working Group Report on Implementation Plans for the CCSP Deliverable: CDR production enables: CDR production enables: Identification, reduction and removal of discontinuities in Re-analyses Reducing impact of changes in the observing system Linkage to Other Programs Re-analysis “Re-analyses of historical climate data for key atmospheric features. Implications for attribution of causes of observed change”

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, General CDR Development Quality control and bias correct satellite radiance data Develop, acquire, and quality control needed forcing data sets, e.g. SST, land use, solar variability, green house gases, aerosols Homogenize key observations, especially upper air data that have suffered from a rapidly changing observing system Linkage to Other Programs

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Modern Satellite Era (1979-present) Upper-air Era (approx present) Surface Era (approx present) Linkage to Other Programs CDR Relevant Strategies & data streams aimed at developing improved Re-analyses for:

Linkage to Other GEOSS Programs 23

International Project Management, (GEO) Regular Monitoring Reports for NOAA’s Earth Observation Partnership of the America’s (EOPA) Activities/Products Improved Data Analysis CDRs Model Reanalysis State of “…” Monitoring Reports User Requirement Document Programmatic Guidance (e.g., GEO, GCOS, GTOS, WHYCOS, IGBP, IGOS, etc. Data Management Climate Liaison Team (Stakeholder Consultation) Application of Data Products Improved Earth System Understanding and Forecasting Applications & Assessments Validation of Earth System Modeling Projections & Prediction (including Extreme Events) Atmosphere Terrestrial Oceans Cryosphere 24 IEOS Near-Term Deliverables and Potential Regional Linkages Drought (NIDIS) Air Quality Sea Level Land Use/Land Cover Natural Hazards IWGEO Near-Term Deliverables Integrated Data sets 24

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, Questions

Collection - Internet, private networks, satellite, media Ingest/Management - Collect/create/maintain metadata - Catalog and inventory Quality Control - Basic-missing data, identifying and correcting suspected errors including - Model/Data intercomparison - Observing system performance monitoring - Developing Climate Data Records Preservation/Archive - Sort/Reformat - Storage - Backup Data Management +(Analysis) { CLASS { Scientific Data Steward- ship Analysis - Monitor climate indicators - Data products (including integration) - Reprocessing in time and space (means and extremes) - Data filtering (filter high frequency noise) - Trend analysis Discovery/Access - Interactive browse on-line, near-line - Via customer services Data Delivery - On-line, Internet - Media { CLASS Overarching the above data management components are interoperable systems, effective user feedback and standards/protocols { CLASS