The use of standard OGC web services in integrating distributed model, satellite and in-situ datasets Alastair Gemmell Jon Blower Keith Haines Environmental Systems Science Centre & Reading e-Science Centre, University of Reading, UK Martin Price UK Met Office Keiran Millard Quillon Harpham HR Wallingford, UK AGU Fall Meeting December 2008

Talk Outline Context to the work Potential problems Useful tools and technologies Our solution Summary

Context to the work Part of ECOOP project: European Coastal Operational Oceanography. 71 partners across most European countries. Task was to integrate model, in-situ, and satellite data feeds from various project partners into one portal. Users should be able to compare observed data with the model outputs. Initial focus was on Ecosystem-relevant data in the North Sea. This has been expanded to other data.

Potential problems Lack of interoperability Heterogeneity in data formats Data dispersed around Europe

Useful tools and technologies OGC standards ( ) Web Map Service (WMS) Web Feature Service (WFS) Climate Science Modelling Language (CSML) THREDDS / OPeNDAP

Useful tools and technologies – OGC standards WMS = Web Map Service. Serve geo-referenced images. Ideal for model output (also satellite data). WFS = Web Feature Service. Serve geo-referenced points, lines, polygons. Ideal for in-situ observations, trajectories etc. OGC has specifications for these services, allowing data to be served in a consistent manner. Applications know what data format to expect and how to ask for it. ncWMS: OGC-compliant WMS for NetCDF developed at Reading e- Science Centre (ncwms.sf.net ). Connects to Godiva2 web client which uses OpenLayers to display data.

Useful tools and technologies – CSML Climate Science Modelling Language (csml.badc.rl.ac.uk ) A standards-based way of representing data features pertinent to the Climate Sciences. 13 main feature types including profiles, trajectories, swaths, timeseries. Provides a common view onto datasets, independent of their storage format or physical location. Ideal for integrating diverse data products We are developing a set of reusable Java libraries that embody the CSML concepts - can then apply these techniques to a number of other projects.

Useful tools and technologies – THREDDS THematic Realtime Environmental Distributed Data Services ( Enables data providers to serve NetCDF and similar data easily online via OPeNDAP protocol. Subsetting of data can be built into request THREDDS now contains ncWMS bundled in ( Data can be served by THREDDS either via OPeNDAP (e.g. obs data), or WMS (e.g. model images). Diverse datasets held in different places can read in via THREDDS servers at each institute.

Our solution ESSC Web Portal ESSC ncWMS web app. CSML ESSC ECOOP Obs web app. ESSC POLCOMS MRCS model (Physical) CEFAS SmartBuoys SMHI SEPRISE In-situ data ESSC Ferrybox (NetCDF) HR Wallingford WFS ASCII LOCAL REMOTE PML THREDDS Server CF-NetCDF PML THREDDS Server PML POLCOMS- ERSEM MRCS model (Biological)

Comparing / co-plotting datasets can catch errors! This looks like a suspiciously large and constant difference between obs and model

Summary Historically data providers in different disciplines used different formats. This created barriers to multidisciplinary work. Increasingly there are common standards and conventions for formatting and serving data. There is gradual uptake of these by providers. We have found OGC WMS and WFS effective solutions for working with model and observed data respectively. CSML was useful as a means of representing the diverse data as common feature types. These tools and technologies allows diverse data to be brought together for the first time – this can reveal new trends in the data, as well as helping with quality control.

Summary Thanks for your attention