ESG Observational Data Integration Presented by Feiyi Wang Technology Integration Group National Center of Computational Sciences

Managed by UT-Battelle for the U. S. Department of Energy The Need for ESG: Climate Science Data Sharing

Managed by UT-Battelle for the U. S. Department of Energy ESG Sponsors Current ESG Sites ESG-CET Federation Team ORNL LLNL NCAR NASA

Managed by UT-Battelle for the U. S. Department of Energy ESG’s Broad Use  LLNL CMIP-3 (IPCC AR4) ESG portal – 35 TB of data at one location  83,337 files, model data from 13 countries  Generated by a modeling campaign coordinated by the Intergovernmental Panel on Climate Change (IPCC)  ORNL (C-LAMP, CCSM Data) ESG portal – 71 TB of data at one location  270,378 files  Includes all model data generated by he Carbon Land Model Intercomparison Project (C-LAMP)  CDIAC and ARM  NCAR CCSM ESG portal – 237 TB of data at four locations (NCAR, LBNL, ORNL, LANL) : 965,551 files  Includes the past 7 years of joint DOE/NSF climate modeling experiments  Broad Impact  Over 2,700 sites  20,000 users  120 countries  Over 1 PB downloaded  500 – 800 active users in a given month  Over 600 published papers Courtesy: Gary Strand - NCAR  Serving a wide range of model data  Coupled Model Intercomparison Project, Phase 3 (CMIP-3)  Community Climate System Model (CCSM)  Parallel Climate Model (PCM)  Parallel Ocean Program (POP)  The North American Regional Climate Change Assessment Program (NARCCAP)  Cloud Feedback Model Intercomparison Project (CFMIP)  Carbon-Land Model Intercomparison Project (C-LAMP)

Managed by UT-Battelle for the U. S. Department of Energy ESG: All About Federation

Managed by UT-Battelle for the U. S. Department of Energy Carbon-Land Model Inter-comparison Project (C-LAMP) – An Exemplar  Allows the scientific community to evaluate the performance of biogeochemical models normally coupled to general circulation models (GCMs).  Established metrics to evaluate model performance via observational dataset inter-comparison Courtesy: Jamison Daniel- ORNL Courtesy: Forrest Hoffman- ORNL

Managed by UT-Battelle for the U. S. Department of Energy Publishing High Impact Observational Data  CDIAC – AmeriFlux (L2) – AmeriFlux (L4) – When available – Gridded Fossil Fuels  ARM – Climate Modeling Best Estimate  Atmospheric Quantities  Cloud and Radiation Quantities ARM ArchiveCDIAC SponsorDOE BER Type of data Atmospheric processes, cloud dynamics Atmospheric gases, emissions data, Ameriflux Archive Size> 190 TB~ 400 GB Users/year~1000~350,000 Year Started

Managed by UT-Battelle for the U. S. Department of Energy ARM – Climate Modeling Best Estimate Top level project Nested collections & datasets Nested collections & datasets File collection Harvested metadata File collection Harvested metadata

Managed by UT-Battelle for the U. S. Department of Energy CDIAC – AmeriFlux Level 2

Managed by UT-Battelle for the U. S. Department of Energy ESG Integration: Next Steps  Support search of non CF-compliant variables  Spatio-temporal search support – ORNL will lead this effort in the ESG community  More flexible data model – In collaboration with NCAR and NASA JPL  Enhanced search API and web intelligence – Support both RDF triple store and Next Generation Search Technologies  Flexible data publishing workflow – Incremental, continuous, automated agents – In collaboration with LLNL  Soliciting feedback on CDIAC user community requirements

Managed by UT-Battelle for the U. S. Department of Energy Synergistic Activities – UV-CDAT  The Ultra-scale Visualization – Climate Data Analysis Tools (UV-CDAT) – An integrated framework for end-to-end data analysis and visualization of climate model and observational datasets – Provenance, Workflow and automated inter-comparison (both model/model and model/observations)

Managed by UT-Battelle for the U. S. Department of Energy Summary  We published several high impact observational datasets and made them available to a broader community  We made good progress on identifying the limitations of the current system and identifying areas of improvement  We have developed a path forward to fully integrate high-impact Observational Datasets within ESG  We developed a collaborative relationship between the ESG and observation dataset community and are playing an active role in extending ESG to support their requirements

Managed by UT-Battelle for the U. S. Department of Energy ESG Integration Contributors  NCCS Technology Integration Group – Galen Shipman (PI), Feiyi Wang, Ross Miller, John Harney  Environment Science and Data Management Group – CDIAC: Thomas Boden, Misha Krassovski – ARM: Raymond McCord, Sigurd Christensen – Bruce Wilson, Jerry Pan, Giri Palanisamy  CSM – Meili Chen  Other contributors – Bob Drach (PCMDI/LLNL) – Luca Cinquini (NASA/JPL) – Eric Neinhouse (NCAR)

Managed by UT-Battelle for the U. S. Department of Energy Thank You