ODV for Data Products Reiner Schlitzer, Alfred Wegener Institute, Bremerhaven, Germany.

Slides:



Advertisements
Similar presentations
Reiner Schlitzer & Stephan Heckendorff Alfred Wegener Institute for Polar and Marine Research Multi-file Data Aggregation with Ocean Data View.
Advertisements

WP1 - Data collection and metadata compilation in sea regions Sissy Iona (HCMR/HNODC) EMODNET Chemistry th Steering Committee, 2-3 December 2014,
EDMODnet Chemistry - DIVA Products from the Baltic Region Örjan Bäck, Lotta Fyrberg, Martin Hansson, Magnus Wenzer Swedish Meteorological and Hydrological.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV Gridding Methods.
Status of advanced access and viewing services following SeaDataNet D5.6 and D8.7 By Dick M.A. Schaap – Technical Coordinator Barcelona – Spain, 19 – 20.
Quality Control Standards for SeaDataNet Review status at 1 st Annual Meeting (March 2007) Review developments over last year Current status Future work.
Synthesis of Incomplete and Qualified Data using the GCE Data Toolbox Wade Sheldon Georgia Coastal Ecosystems LTER University of Georgia.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View - Available Data Collections and Data Model.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View 4 - Data Quality Control.
WP2: Data products generation in Mediterranean Sea EMODNET Chemistry Data Products, Experts Workshop Split (Croatia), June 2014 Sissy Iona, HCMR-HNODC.
Marine Data Management within EMODNet Chemistry project: data aggregation, quality control and products preparation A.S. Iona 1, P. Karagevrekis 1, S.Balopoulou.
OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 rd SeaDataNet training course – Ostende – June 2008 NEMO reformatting tool v1 M. Fichaut.
1 st year meeting SPLIT, Croatia June 2014 REPORT ON ATLANTIC AREA With IFREMER, RIHMI-WDC, IHPT and IEO contributions.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View.
QC/QA and Diva Products in Mediterranean Sea EMODNET Chemistry 2 Meeting on ODV QC and Diva Products Ifremer Headquarter, Paris, 3 September 2014 Sissy.
GCE Data Toolbox -- metadata-based tools for automated data processing and analysis Wade Sheldon University of Georgia GCE-LTER.
EMODnet Chemistry 2 Service Contract MARE/2012/10 S Progress of the CDI service By Dick M.A. Schaap – Technical Coordinator Istanbul – Turkey,
Reiner Schlitzer & Stephan Heckendorff Alfred Wegener Institute for Polar and Marine Research Ocean Data View – New Developments.
OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet annual meeting, Madrid, March 2009 How to prepare data for integration in SeaDataNet.
Annual Meeting, June , Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet.
Chemical lot - HOW: Infrastructure set up based on SeaDataNet V1 efficient distributed Marine Data Management Infrastructure; Principle of “ADOPTED AND.
Annual Meeting, June , Istanbul, Turkey WP1: Data collection and metadata compilation in sea regions: current status EMODnet Chemistry Partner.
Annual Meeting, June , Istanbul, Turkey WP1: Data collection and metadata compilation in sea regions: current status EMODnet Chemistry Partner.
EMODnet Chemistry 2 Technical progress and new challenges Service Contract MARE/2012/10 S By Dick M.A. Schaap – Technical Coordinator Venice,
Hernan E. Garcia (U.S. NODC, IODE Group of Experts on Biological and Chemical Data Management and Exchange Practices) 2nd IQuOD Workshop 2014, Silver Spring,
Java OceanAtlas A Cross-Platform Application for Visualization and Selection of Oceanographic Profile Data John R. Osborne
Java OceanAtlas A Cross-Platform Application for Visualization and Selection of Oceanographic Profile Data John R. Osborne ftp://odf.ucsd.edu/pub/OceanAtlas/
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV – New Developments.
CTD Data Processing Current BIO Procedure. Current Processing Software Matlab Migrating to R & Python Code Version Control SVN Migrating to GitHub.
Ocean Data View Course Objective: Import, quality control and visualize own or external datasets. What you learn: Large publicly available datasets exist.
The SeaDataNet data products regional temperature and salinity historical data collections S. Simoncelli 1, C. Coatanoan 2, O. Bäck 3, H. Sagen 4, S.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Data Quality Control and Visualization with Ocean Data View 4.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View Achievements and Future Developments.
EDMODnet Chemistry, Baltic Sea - Oxygen, Chlorophyll and contaminants - General issues Magnus Wenzer, Lotta Fyrberg, Örjan Bäck Swedish Meteorological.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View and itsRole in SeaDatanet and its Role in SeaDatanet.
EMODnet Chemistry 2 Service Contract MARE/2012/10 S Progress of the CDI service By Dick M.A. Schaap – Technical Coordinator Helsinki – Finland,
DATA AGGREGATION AND PRODUCTS GENERATION
NEMO – Reformating tool
Fujimi SPC System using JSL June 22, 2016
Partner contribution:
Data aggregation and products generation in the Mediterranean Sea
Using Ocean Data View for EMODnet Chemistry Reiner Schlitzer
EMODnet Chemistry WP2: Data aggregation and products generation
OCTOPUS – SeaDataNet Format conversion tool
WP3: Data aggregation and products generation NE Atlantic Sea
Everything You Need in One Simple Overview…. July 27, 2016
Status of ODV developments
User working in web-browser
Meeting on ODV QC and DIVA Products
WP2 Products in Mediterranean Sea
ISRAMAR Work Flow for SeaDataNet
Dick M.A. Schaap – Technical Coordinator SeaDataNet Training Workshop
Tools for generation of SDN NetCDF (CFPOINT) files
Ocean Data View Reiner Schlitzer
Ocean Data View 4 - Data Quality Control Reiner Schlitzer
Ocean Data View 4 - Data Quality Control Reiner Schlitzer
EMODnet Chemistry WP1: Data collection and metadata compilation in sea regions: current status Partner 40: State Oceanographic Institute Alexander Korshenko,
Data aggregation and products generation
Data aggregation and products generation in the Mediterranean Sea
EMODnet Chemistry WP1: Data collection and metadata compilation in sea regions: current status Partner contribution: BODC Annual Meeting, June ,
ODV Gridding Methods Reiner Schlitzer
Ocean Data View Spreadsheet Format Reiner Schlitzer
EMODnet Chemistry WP1: Data collection and metadata compilation in sea regions: current status Partner contribution: Latvian Institute of Aquatic Ecology.
Ocean Data View Reiner Schlitzer
ODV beta Multiple views, new views bar
EUTROPHICATION – NE ATLANTIC
Generation of data products in the Norwegian and Barents Seas
SDN Data access search:
EMODnet Chemistry 3 Generation of data products in the Baltic Sea
Generation of data products in the Mediterranean Sea
Presentation transcript:

ODV for Data Products Reiner Schlitzer, Alfred Wegener Institute, Bremerhaven, Germany

1.Providing data QC as well as outlier flagging and filtering. 2.Providing data file aggregation and production of large integrated datasets. 3.Providing production of time-series plots, maps, sections and animated 3D plots.

Visual Quality Control easy spotting and identification of outliers or offsets easy spotting and identification of outliers or offsets painless editing of data value and quality flag painless editing of data value and quality flag logging of all value or flag modifications logging of all value or flag modifications

Automatic Range Checks full sample set full sample set sample sub-setsample sub-set

QF Mass Assignments all samples of many or all stations (initialization)all samples of many or all stations (initialization) all samples in a windowall samples in a window all or some selected samples of current stationall or some selected samples of current station current sample onlycurrent sample only

Using Quality Flag Filters Before filtering After filtering

File Aggregation Multi-file import (analysis and aggregation phases).

Use //SDN_parameter_mapping sectionUse //SDN_parameter_mapping section Determine type of data (profile, time-series, trajectory).Determine type of data (profile, time-series, trajectory). //SDN_parameter_mapping // SDN:LOCAL:PRESSURE SDN:P011::PRESPS01 SDN:P061::UPDB // SDN:LOCAL:T90 SDN:P011::TEMPS901 SDN:P061::UPAA // SDN:LOCAL:Salinity SDN:P011::PSALPR02 SDN:P061::UUUU // SDN:LOCAL:fluorescence SDN:P011::CPHLPM01 SDN:P061::UGPL // SDN:LOCAL:Trans_red_25cm SDN:P011::POPTDR01 SDN:P061::UPCT // SDN:LOCAL:POTM SDN:P011::POTMCV01 SDN:P061::UPAA // SDN:LOCAL:Density SDN:P011::SIGTPR01 SDN:P061::UKMC File Analysis

ODV collections are created for every data type found. ODV collections are created for every data type found. Data of a given file are imported into the corresponding ODV collection. Data of a given file are imported into the corresponding ODV collection. Process is fully automatic. Process is fully automatic. Results in one or more ODV collections containing all data of a given type. Results in one or more ODV collections containing all data of a given type. Aggregation

// //SDN_parameter_mapping // SDN:LOCAL:DEPTH SDN:P011::ADEPZZ01 SDN:P061::ULAA // SDN:LOCAL:Temperature SDN:P011::TEMPS901 SDN:P061::UPAA // SDN:LOCAL:Salinity SDN:P011::ODSDM021 SDN:P061::UUUU // SDN:LOCAL:Oxygen SDN:P011::DOXYCZ01 SDN:P061::KGUM // //SDN_parameter_mapping // SDN:LOCAL:DEPH SDN:P011::ADEPZZ01 SDN:P061::ULAA // SDN:LOCAL:PRES SDN:P011::PRESPR01 SDN:P061::UPDB // SDN:LOCAL:TEMP SDN:P011::TEMPPR01 SDN:P061::UPAA // SDN:LOCAL:PSAL SDN:P011::PSLTZZ01 SDN:P061::UUUU // SDN:LOCAL:DOX1 SDN:P011::DOXYZZXX SDN:P061::UMLL // SDN:LOCAL:DOXY SDN:P011::DOXYZZXX SDN:P061::UPOX // SDN:LOCAL:TUR4 SDN:P011::TURBXXXX SDN:P061::USTU // SDN:LOCAL:NTRA SDN:P011::NTRAZZXX SDN:P061::UPOX // SDN:LOCAL:SLCA SDN:P011::SLCAZZXX SDN:P061::UPOX // SDN:LOCAL:PHPH SDN:P011::PHXXZZXX SDN:P061::UUPH // SDN:LOCAL:CORG SDN:P011::CORGZZZX SDN:P061::UPOX // SDN:LOCAL:POCP SDN:P011::MDMAP010 SDN:P061::UMMC // SDN:LOCAL:TSMP SDN:P011::TSEDZZZZ SDN:P061::UMGL // Aggregated Variables

When aggregating SDN files different BODC parameter/unit codes are maintained as separate variables (e.g., 3 oxygen variables). User may combine separate variables by means of aggregated derived variables (e.g., 1 oxygen variable for scientific usage).

Aggregated derived variable Oxygen [ml/l] = 1 * (DOX1 DISSOLVED OXYGEN ml/l) * (DOX2 DISSOLVED OXYGEN micromole/kg) * (DOXY DISSOLVED OXYGEN millimole/m3) + 0 Merging Principles: Merging Type (exclusive use of first available value, average over all available values) Order of input variables matters (in case of eclusive merge) Standard or user-specified conversion formulas Quality flag inheritence

CDI catalogue as of July 2012

Time-series Plots

Distribution Maps Sections

3D Animations