World Conference on Climate Change October 24-26, 2016 Valencia, Spain

Slides:

Advertisements

Similar presentations

Std-doi Publication of Climate Data at WDCC DataCite Summer Meeting 7./8. June 2010 Publication of climate data Heinke Höck World Data Center for Climate.

Advertisements

© GEO Secretariat Agenda Item 3. GEO UPDATE. © GEO Secretariat Membership 67 members and 43 Participating Organisations – New Members:Latvia, Moldova,

Preservation and Long Term Access of Data at the World Data Centre for Climate Frank Toussaint N.P. Drakenberg, H. Höck, M. Lautenschlager, H. Luthardt,

Climate Analytics on Global Data Archives Aparna Radhakrishnan 1, Venkatramani Balaji 2 1 DRC/NOAA-GFDL, 2 Princeton University/NOAA-GFDL 2. Use-case 3.

Astrophysics, Biology, Climate, Combustion, Fusion, Nanoscience Working Group on Simulation-Driven Applications 10 CS, 10 Sim, 1 VR.

CLIMATE SCIENTISTS’ BIG CHALLENGE: REPRODUCIBILITY USING BIG DATA Kyo Lee, Chris Mattmann, and RCMES team Jet Propulsion Laboratory (JPL), Caltech.

M. Stockhause et al. Martina Stockhause, Michael Lautenschlager, Frank Toussaint Deutsches Klimarechenzentrum (DKRZ) World Data Centre for Climate (WDCC)

The Earth System Grid Discovery and Semantic Web Technologies Line Pouchard Oak Ridge National Laboratory Luca Cinquini, Gary Strand National Center for.

Workshop on Climate Change Impacts, Adaptation, and Vulnerability (IAV) Community Coordination 8-9 January 2009, National Center of Atmospheric Research.

M.Lautenschlager (WDCC / MPI-M) / / 1 GO-ESSP at LLNL Livermore, June 19th – 21st, 2006 World Data Center Climate: Status and Portal Integration.

M. Lautenschlager (M&D/MPIM)1 The CERA Database Michael Lautenschlager Modelle und Daten Max-Planck-Institut für Meteorologie Workshop "Definition.

Z EGU Integration of external metadata into the Earth System Grid Federation (ESGF) K. Berger 1, G. Levavasseur 2, M. Stockhause 1, and M. Lautenschlager.

IS-ENES [ees-enes] InfraStructure for the European Network for Earth System Modelling IS-ENES will develop a virtual Earth System Modelling Resource Centre.

Working Group: Practical Policy Rainer Stotzka, Reagan Moore.

F. Toussaint (WDCC, Hamburg) / / 1 CERA : Data Structure and User Interface Frank Toussaint Michael Lautenschlager World Data Center for Climate.

NOCS, PML, STFC, BODC, BADC The NERC DataGrid = Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC.

CCSM DATA MANGEMENT POLICY The Community Climate System Model (CCSM) Data Management Policy documents the procedures for the management of model data produced.

Michael Lautenschlager World Data Center Climate Model and Data / Max-Planck-Institute for Meteorology German Climate Computing Centre (DKRZ)

Bulk Metadata Structures in CERA Frank Toussaint, Michael Lautenschlager Max-Planck-Institut für Meteorologie World Data Center for Climate.

Creating documentation and metadata: Recording provenance and context Jeff Arnfield National Climatic Data Center Version a1.0 Review Date.

Data Management in Scholarly Journals and possible Roles for Libraries – Some Insights from EDaWaX Sven Vlaeminck | Leibniz-Information Centre for Economics.

Data Publication and Quality Control Procedure for CMIP5 / IPCC-AR5 Data WDC Climate / DKRZ:

_______________________________________________________________CMAQ Libraries and Utilities ___________________________________________________Community.

Adoption of RDA-DFT Terminology and Data Model to the Description and Structuring of Atmospheric Data Aaron Addison, Rudolf Husar, Cynthia Hudson-Vitale.

EVA Workshop, 26 March 2003, Florence, Italy1 COINE Cultural Objects In Networked Environments Anthi Baliou University of Macedonia,Library Thessaloniki,

Using the Global Change Master Directory (GCMD) to Promote and Discover ESIP Data, Services, and Climate Visualizations Presented by GCMD Staff January.

DataONE: Preserving Data and Enabling Data-Intensive Biological and Environmental Research Bob Cook Environmental Sciences Division Oak Ridge National.

- Vendredi 27 mars PRODIGUER un nœud de distribution des données CMIP5 GIEC/IPCC Sébastien Denvil Pôle de Modélisation, IPSL.

IPCC TGICA and IPCC DDC for AR5 Data GO-ESSP Meeting, Seattle, Michael Lautenschlager World Data Center Climate Model and Data / Max-Planck-Institute.

The CF Conventions: Options for Sustained Support Involving Unidata Russ Rew Unidata Policy Committee May 12, 2008.

Cyberinfrastructure to promote Model - Data Integration Robert Cook, Yaxing Wei, and Suresh S. Vannan Oak Ridge National Laboratory Presented at the Model-Data.

WP6/SA2: Access to IS-ENES Data Federation SA2 is a European distributed data infrastructure providing access to data from ESM simulations produced in.

Archiving microdata Standards and good practices United Nations Statistics Commission New York, February 26, 2009 Olivier Dupriez World Bank, Development.

Mid-Decade Assessment of the United Nations 2010 World Population and Housing Census Program Arona L. Pistiner Office of the Associate Director for 2020.

M. Stockhause 1, G. Levavasseur 2, K. Berger 1 1 Deutsches Klimarechenzentrum (DKRZ) 2 Institute Pierre Simon Laplace (IPSL) ESGF-QCWT Quality Control.

SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.

1 Summary. 2 ESG-CET Purpose and Objectives Purpose  Provide climate researchers worldwide with access to data, information, models, analysis tools,

Fire Emissions Network Sept. 4, 2002 A white paper for the development of a NSF Digital Government Program proposal Stefan Falke Washington University.

17 th October 2002Data Provenance Grid Data Requirements Scoping Metadata & Provenance Dave Pearson Oracle Corporation UK.

The Data Sharing Working Group 24 th meeting of the GEO Executive Committee Geneva, Switzerland March 2012 Report of the Data Sharing Working Group.

Ed Kearns National Climatic Data Center Asheville, NC.

Providing access to your data: Determining your audience Robert R. Downs, PhD NASA Socioeconomic Data and Applications Center (SEDAC) Center for International.

Interagency Forum on Earth Data Preservation/LifeCycle /Stewardship January 8, 2009 Rob Raskin NASA/Jet Propulsion Lab.

IPCC WG II + III Requirements for AR5 Data Management GO-ESSP Meeting, Paris, Michael Lautenschlager, Hans Luthardt World Data Center Climate.

Hannes Thiemann Michael Lautenschlager Deutsches Klimarechenzentrum GmbH, Germany EGU 2010.

Support to scientific research on seasonal-to-decadal climate and air quality modelling Pierre-Antoine Bretonnière Francesco Benincasa IC3-BSC - Spain.

M. Lautenschlager (M&D/MPIM)1 WDC on Climate as Part of the CERA 1 Database System Michael Lautenschlager Modelle und Daten Max-Planck-Institut.

CAS2K11 in Annecy, France September 11 – 14, 2011 Data Infrastructures at DKRZ Michael Lautenschlager.

AOLI 2015 The NMME Experience: A Research Community Archive Lessons learned from Climate Model data archive and use AOLI Meeting 2015 Eric Nienhouse NCAR.

Weigel, Berger, Kindermann, Lautenschlager EGU Versioning for CMIP6 in the Earth System Grid Federation Data preparation Initial registration.

1 This slide indicated the continuous cycle of creating raw data or derived data based on collections of existing data. Identify components that could.

Intentions and Goals Comparison of core documents from DFIG and Publishing Workflow IG show that there is much overlap despite different starting points.

Accessing the VI-SEEM infrastructure

Approaches and Challenges in Managing Persistent Identifiers

AP7/AP8: Long-Term Archival of CMIP6 Data

Data Citation Service for CMIP6 and IPCC DDC Aspects

Data Ingestion in ENES and collaboration with RDA

Overview of GEOSS Data Sharing Principles and Implementation GEO-XIII Plenary Side Event Towards Open Earth Observation Data Policies Wenbo Chu GEO.

Persistent Identifiers Implementation in EOSDIS

DA Task Report Data Integration and Analysis System

Climate Data Analytics in a Big Data world

Institutional Framework, Resources and Management

IPET-OPSLS/CCl-17 relevant issues before EC-70

IS-ENES Cases Seven use cases are listed as data lifecycle steps A B C

WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe

how users and data producers interact on WIS

Robert Dattore and Steven Worley

Scientific Workflows Lecture 15

Presentation transcript:

World Conference on Climate Change October 24-26, 2016 Valencia, Spain The role of climate model data and long-term data archives in climate change research World Conference on Climate Change October 24-26, 2016 Valencia, Spain

Observations and Models (1) Characteristics of Observations Diverse: different instruments measuring different parameters Discrete: certain spatial-temporal coverage Continuously extended No future information Observation Data Management Big Data: many different parameters and data formats Products for different research purposes, esp. satellite data  new versions: reprocessing with improved algorithms Description on measurement conditions and provenance required ESGF Conference 2016 06.-09.12.2016

Observations and Models (2) Characteristics of Climate Model Relatively homogeneous, with standardized formats and naming conventions, e.g. CF Static (Once created and many times analyzed) 4D data for many parameters as mean values for grid cells over a time interval Future projections and scientific questions Model Data Management High Volume Data: PBytes of homogeneous data created New data versions are rare  for post-processed datasets Data access of data subsets Detailed description of data subsets required  Reanalysis Models for observation data assimilation ESGF Conference 2016 06.-09.12.2016

IPCC Data Distribution Centre (DDC) ISENES2 Workshop on ESM Workflows 2016 28.09.2016

IPCC DDC and TGICA IPCC DDC (Data Distribution Centre) – ipcc-data.org jointly managed by: British Atmospheric Data Centre (BADC): Climatologies World Data Center Climate (WDCC) at DKRZ: Reference Data Archive for climate model output Center for International Earth Science Information Network (CIESIN) at Columbia University: social-economic data archive  Certified ICSU World Data System (WDS) members FAR: NCAR comparable role to ETH Zurich in AR5 IPCC TGICA (Task Group on Data and Scenario Support for Impact and Climate Analysis) Oversees IPCC DDCs Enables research and sharing of information across the IPCC Working Groups  Mandate and structure of TGICA are currently under review by the IPCC ESGF Conference 2016 06.-09.12.2016

History of IPCC DDC 1995: IPCC SAR climate model data long-term archived 1998: IPCC DDC formally established 2008: parts of FAR data added to DDC 2013/14: IPCC DDC AR5 data long-term archival 2016: IPCC Task Force built for transformation of the organization of IPCC data and information to serve the needs of the IPCC during and beyond AR6. 2020/21: IPCC DDC AR6 long-term archival FAR: NCAR comparable role to ETH Zurich in AR5 ESGF Conference 2016 06.-09.12.2016

IPCC DDC Reference Data Archive The IPCC DDC provides data on the long-term for an interdisciplinary user community in support of the IPCC Authors. Long-term: archival with second data copy in an established data center Interdisciplinary Use: add information to the data for a creator-independent usage IPCC Author Support: provide a reliable, up-to-date and easily-accessible CMIP6 data pool ESGF Conference 2016 06.-09.12.2016

IPCC DDC Services for AR6 Data Creator IPCC Author IPCC DDC User IPCC DDC AR6 Services Earth System Grid Federation (ESGF) CMIP Data Pool at DKRZ IPCC DDC Reference Data Archive at DKRZ CMIP6 (subset) AR6 Replication Long-Term Archival CMIP5 AR5 CORDEX AR4 Derived Products AR3 Analysis Input AR2 ESGF Conference 2016 06.-09.12.2016

ISENES2 Workshop on ESM Workflows 2016 Data Management ISENES2 Workshop on ESM Workflows 2016 28.09.2016

Replicate to build CMIP Data Pool Challenges: Implementation: Timely automated update of evolving data Provide easy and script-based access to data User support Coordination nationally and internationally Synda tool used based on ESGF Search API User accounts on Linux machine provided Request tracker Task group built ESGF Conference 2016 06.-09.12.2016

Principles of Long-Term Archival (1) Gather what you can… …as long as it is available. Metadata challenges: Implementation: Ancillary metadata is diverse in respect to: Granularity, Format, Access, Stability. Registration of ancillary metadata URL in relation to the data by metadata providers. refer- ences citation ESMVal errata ESGF Conference 2016 06.-09.12.2016

Principals of Long-Term Archival (2) Automate what you can… …for a timely archival. Automated access, interpretation and mapping of metadata sorts the different pieces of metadata in the hierarchical structure of the LTA metadata schema. citation refer- ences Mapping errata ESMVal ESGF Conference 2016 06.-09.12.2016

Principals of Long-Term Archival (3) Check everything at least twice… …before archival. The automated process is interrupted at several stages in order to ensure metadata consistency. After archival such a quality assurance is unfeasible. refer- ences citation ESMVal errata ESGF Conference 2016 06.-09.12.2016

ISENES2 Workshop on ESM Workflows 2016 Data Usage ISENES2 Workshop on ESM Workflows 2016 28.09.2016

IPCC DDC Users Climate researchers like IPCC WG I Authors: familiar with data formats and tools to analyze the data skills to interpret and use data without additional services Climate impact researchers like IPCC WG III Authors: need information on data formats and analysis tools need more information on how to use the data additional at technical services requested like derived climate parameters, data regridding Policy advisors: need assistance on data formats, tools and data interpretation Personnel as well as technical services required, ideally a climate service center with trained personnel ESGF Conference 2016 06.-09.12.2016

IPCC DDC: AR5 Reference Archive The DDC Reference Archive / The IPCC WG1 Archive Experiments: 101 / 78 different experiments / scenarios Variables: 605 / 123 different variables (628 requested variables) Size: 1.6 PByte / 100 TByte (all AR data: 1.7 PByte) Models: 60 / 58 participating models Institutes: 27 / 24 participating institutes Simulations: 1145 / 952 provided simulations 818795 / 93247 provided variables ESGF Conference 2016 06.-09.12.2016

IPCC-DDC: Usage (1) Reference Archive for Climate Model Output Data 1 PByte 1 TByte ESGF Conference 2016 06.-09.12.2016

IPCC-DDC Usage (3) Reference Archive for Climate Model Output Data 702 Active DDC Users in 2015: 42% located in developing and economy-in-transition countries (Africa, Asia, South America) 15 users requested regional data on storage media Download Counts per Continent in 2015: 69 % of downloads from users in developing and economy-in-transition countries (59 % Asia) average download count per user = 2 800 Asian and African users were the most active with average download numbers per user of 5 000 and 4 600.

IPCC DDC: http://ipcc-data.org DDC at DKRZ: http://ipcc.wdc-climate.de M. Stockhause, F. Toussaint, M. Lautenschlager (2015): CMIP6 Data Citation and LTA. WIP white paper. Zenodo. doi:10.5281/zenodo.35178. ESGF Conference 2016 06.-09.12.2016