NOCS, PML, STFC, BODC, BADC The NERC DataGrid + ++ + = Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC.

Slides:

Advertisements

Similar presentations

The Next Generation Network Enabled Weather (NNEW) SWIM Application Asia/Pacific AMHS/SWIM Workshop Chaing Mai, Thailand March 5-7, 2012 Tom McParland,

Advertisements

BADC Workshop 1: Data & Services from the BADC Royal Met. Soc. Conference – 12 September 2005 Kevin Marsh et al.

Environmental Information Data Centre: enabling the discovery of CEH-held data John Watkins Deputy Director EIDC.

Page 1 © Crown copyright 2005 Workshop on Metadata Beijing27-29 September WMO Interoperability – the vision Gil Ross (Met Office UK) Metadata workshop;

Information Modelling MOLES Metadata Objects for Linking Environmental Sciences S. Ventouras Rutherford Appleton Laboratory.

Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.

BADC Workshop 2: BADC Services to Data Suppliers Royal Met. Soc. Conference – 14 September 2005 Ag Stephens et al.

Crossing the Digital Divide

Development of a database of intensively monitored groundwater systems in Australia Barry Croke National Centre for Groundwater Research and Training Australian.

Bryan Lawrence on behalf of BADC, BODC, CCLRC, PML and SOC The British Atmospheric Data Centre and the NERC DataGrid (for) [ ]=

The MashMyData project Combining and comparing environmental science data on the web Alastair Gemmell 1, Jon Blower 1, Keith Haines 1, Stephen Pascoe 2,

NERC Data Grid Helen Snaith and the NDG consortium …

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

BADC, BODC, CCLRC, PML and SOC The NERC Metadata Gateway: a product of the NERC DataGrid [ ]= Bryan Lawrence (on behalf of a big team)

Serving society Stimulating innovation Supporting legislation Meeting on OGC Sensor Observation Service (SOS) for INSPIRE Mickael.

The use of standard OGC web services in integrating distributed model, satellite and in-situ datasets Alastair Gemmell Jon Blower Keith Haines Environmental.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

Interoperability ERRA System.

Crossing the Digital Divide Presented by: Fernando R. Salas David Maidment, Enrico Boldrini, Stefano Nativi, Ben Domenico OGC Technical Meeting – Met/Occean.

Research Data at NCAR 1 August, 2002 Steven Worley Scientific Computing Division Data Support Section.

GADS: A Web Service for accessing large environmental data sets Jon Blower, Keith Haines, Adit Santokhee Reading e-Science Centre University of Reading.

CIM – The Common Information Model in Climate Research

NERC DataGrid and CSML NDG Team. CSML: Context NERC DataGrid: the integration problem –multiple organisations, formats, storage mechanisms (file, relational)

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

Bryan Lawrence on behalf of BADC, BODC, CCLRC, PML and SOC An Introduction to NDG concepts [ ]=

CF Conventions Support at BADC Alison Pamment Roy Lowry (BODC)

INSPIRE, EC Water Standards and WaterML Workshop Keiran Millard Keiran Millard

2 3 ROMS/COAWST NcML file 4 5 Exploiting IOOS: A Distributed, Standards-Based Framework and Software Stack for Searching, Accessing, Analyzing and.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

THREDDS Data Server Ethan Davis GEOSS Climate Workshop 23 September 2011.

1 The NERC DataGrid DataGrid The NERC DataGrid DataGrid AHM 2003 – 2 Sept, 2003 e-Science Centre Metadata of the NERC DataGrid Kevin O’Neill CCLRC e-Science.

Mapping between SOS standard specifications and INSPIRE legislation. Relationship between SOS and D2.9 Matthes Rieke, Dr. Albert Remke (m.rieke,

Geology, mining, groundwater, landscape and soils The ‘Earth Science’ domains Bruce Simons Spatial Information Modelling Community of Practice workshop,

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

DELIVERING ENVIRONMENTAL WEB SERVICES (DEWS) Partners: UK Met Office (Lead Partner), British Atmospheric Data Centre (BADC), British Maritime Technology.

Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.

1 International Standards for Data Interoperability GALEON Geo-interface for Air, Environment, Land, Ocean NetCDF Ben Domenico Unidata Program Center*

VO Sandpit, November 2009 CEDA Metadata Steve Donegan/Sam Pepler.

BADC, BODC, CCLRC, PML and SOC Interacting with NDG [ ]= Bryan Lawrence (on behalf of a big team)

NDG Discovery Gateway ISO19139 Issues Bryan Lawrence Director of Environmental Data Archival and Associated Research, CCLRC Head of the British Atmospheric.

XIth International Congress for Mathematical Geology - September 3-8, 2006 – Liège, Belgium Contribution of GeoScienceML to the INSPIRE data harmonisation.

Integrating the Climate Science Modelling Language with geospatial software and services Dominic Lowe British Atmospheric Data

IOOS Data Services with the THREDDS Data Server Rich Signell USGS, Woods Hole IOOS DMAC Workshop Silver Spring Sep 10, 2013 Rich Signell USGS, Woods Hole.

COWS CEDA OGC Web Services Framework Stephen Pascoe.

AUKEGGSWorkshop ANU, Canberra, 29 November 2006 Implementing CSML Feature Types in applications within the NERC DataGrid Dominic Lowe, British Atmospheric.

Creating Good Documentation NOAA National Geophysical Data Center

Observations & Measurements & SWE in Inspire OGC Hydro DWG Workshop – Reading – Sylvain Grellet Office International de l’Eau.

Slide 1 SDTSSDTS FGDC CWG SDTS Revision Project ANSI INCITS L1 Project to Update SDTS FGDC CWG September 2, 2003.

Alison Pamment 1, Steve Donegan 1, Calum Byrom 2, Oliver Clements 3, Bryan Lawrence 1, Roy Lowry 3 1 NCAS/BADC, Science and Technology Facilities Council,

© University of Reading 2008www.reading.ac.uk Reading e-Science Centre 9 September 2008 Harmonization of environmental data using the Climate Science Modelling.

Dominic Lowe, British Atmospheric Data Centre, STFC OGC TC, Boulder.

GO-ESSP LLNL, June 2006 CSML – Stocktake and Forward Look Andrew Woolf Dominic CCLRC Rutherford Appleton Laboratory.

Grids and Beyond: netCDF-CF and ISO/OGC Features and Coverages Ethan Davis, John Caron, Ben Domenico UCAR/Unidata AMS IIPS, 23 January 2008.

WaterML 2.0 Part 1 - Time Series Hydro Informatics Conference Workshop *Standardization of Water Data Exchange: WaterML2.0 and Beyond* Presented by : Open.

SOLAS and the British Atmospheric Data Centre Charles Kilburn Anne De Rudder.

1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)

1 Alison Pamment, 2 Calum Byrom, 1 Bryan Lawrence, 3 Roy Lowry 1 NCAS/BADC,Science and Technology Facilities Council, 2 Tessella plc, 3 British Oceanogrphic.

Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.

NOAA IOOS SOS Implementations in 2008 Jeff de La Beaujardière, PhD NOAA IOOS Program DIF Sr Systems Architect.

Using a Simple Knowledge Organization System to facilitate Catalogue and Search for the ESA CCI Open Data Portal EGU, 21 April 2016 Antony Wilson, Victoria.

Page 1Overview of the HMA Project, 8 March 2010 Overview of the HMA Project OGC TC Opening Plenary Pier Giorgio Marchetti European Space Agency.

ECMWF 24 th November 2008 Deploying secure OGC services in front of a heterogeneous data archive. Bryan Lawrence, Phil Kershaw, Dominic Lowe, and Stephen.

Reading e-Science Centre Technical Director Jon Blower ESSC Director Rachel Harrison CS Director Keith Haines ESSC Associated Personnel External Collaborations.

Beyond simple features: Do complex feature types need complex service descriptions?' B.N. Lawrence (1,2), D. Lowe (1,2), S. Pascoe (1,2) and A. Woolf (1).

DELIVERING ENVIRONMENTAL WEB SERVICES (DEWS)

NERC DataGrid: Googling for Secure Data

The NERC Metadata Gateway: a product of the NERC DataGrid

MERRA Data Access and Services

BADC, BODC, CCLRC, PML and SOC

Robert Dattore and Steven Worley

Presentation transcript:

NOCS, PML, STFC, BODC, BADC The NERC DataGrid = Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC etc) Representing the NDG and a dose of the “Portals Project”

Royal Society, April, British Atmospheric Data Centre British Oceanographic Data Centre Simulations Assimilation Complexity + Volume + Remote Access = Grid Challenge

Royal Society, April, 2008 Data Sets Ground based observation networks Met Office surface stations Model output NWP, ECMWF reanalyses & Climate models Satellite data TOMS, Envisat & MSG NERC programmes data UTLS, CWVC & URGENT “A collection of files with a common theme and administration” Datasets statistics 147 datasets ~130TB (unique) ~80 Million files

Royal Society, April, 2008 The NDG Use Cases 1.Find data 2.Find out about data 3.Subset and difference data 4.Visualise data 5.All using SECURE interdisciplinary technology. Sound easy doesn’t it? Want interdisciplinary semantic access to information, not abstract data –getData(potential temperature from ERA-40 dataset in North Atlantic from 1990 to 2000) –not: getData(“era40.nc”, ‘PTMP’, 20:50, 300:340, 190:200) –or even worse: for j=1990:2000 getData(“era40_”+j+“.nc”, ‘PTMP’, 20:50, 300:340)

Royal Society, April, 2008 The big: Climate in 2010 – A graphic Illustration Figures from Gary Strand, NCAR, ESG website

Royal Society, April, 2008 … and the little … land use!

Royal Society, April, 2008 ISO19109 in a nutshell

Royal Society, April, 2008 Dataset Metadata

Royal Society, April, 2008 MOVIE(s)

Royal Society, April, 2008 NDG Deployment Vocab Server: 120,000+ concepts 100 lists 75,000 Relationships (SKOS )

Royal Society, April, 2008 Dataset Metadata

Royal Society, April, 2008 MOLES: Metadata Objects for Linking Environmental Science A data production tool deployed at an observation station on behalf of an activity produces a dataset. Datasets have provenance

Royal Society, April, 2008 Getting down to the data itself A-Metdata We choose to deploy an “Application Schema” of the “Geographic Markup Language”. This allows us to define our data to support RE-USE by other communities – interdisciplinarity, and to be INSPIRE compliant!

Royal Society, April, 2008 CSML Dataset Dictionaries: coordinate reference systems, phenomena (parameters) units of measure Feature Collections CSML Features are SPECIALISATIONS of GML Features! Storage Descriptor Re-usable description of storage (NB: CSML Software Stack, WMS, WCS etc)

Royal Society, April, 2008 CSML Feature Types Abstract Specific: point data, time-series of point data, trajectory data, point collections, profiles, profile series, Ragged profile series, section, ragged section, scanning radar, grid, gridseries and swath

Royal Society, April, 2008 Beyond measuring things at points CF concepts: Cell-Methods (statistical e.g means etc) Cell-Bounds (intensive quantities, Extensive quantities, Relationship to coverage domain) New work: Exploiting Sensor Web Enablement

Royal Society, April, 2008 Storage Descriptor

Royal Society, April, 2008 Data Modelling Workshop April 2007, at The Cosenors House, Abingdon Brought NDG, Unidata, OGC, Italian NRC together to get a shared vision for scientific data types. Roadmap for bringing our activities together over next few years. Harmonising: –NDG CSML –OGC Observations and Measurements (with CSML and MOLES) –UNIDATA Community Data Model –INRC NCML-G

Royal Society, April, 2008 Data Model Relationships Route to harmonising GeoSciML and CSML

Royal Society, April, 2008 The Original Vision

Royal Society, April, 2008 What we’ve done (are doing) Schema: MOLES CSML DIF + Lots of Population Issues Protocols: OGC +OAI Pylons based portal Python WCS/WMS Clients CSML Toolbox Vocab Server Pylons Interface & portals

Royal Society, April, 2008 Now and the Future More protocol improvements (= wider community of providers and users) More deployment. More population. Evolution WITH the web.