CARBOOCEAN Data management and SOCAT Benjamin Pfeil, Are Olsen, Jeremy Malzcyk, Steve Hanhin, Alex Kozyr and many others Partner 16 WDC-MARE Partner 19.

Slides:



Advertisements
Similar presentations
The Live Access Server (Access to observational data) Jonathan Callahan (University of Washington) Steve Hankin (NOAA/PMEL – PI) Roland Schweitzer, Kevin.
Advertisements

Intro to Version Control Have you ever …? Had an application crash and lose ALL of your work Made changes to a file for the worse and wished you could.
DataTools Models Data, models and tools: Dealing with any complex hydraulic engineering problem invariable use is made of: data, models and tools.
Groom-gliders data-management workshop Brest, December 2012 Groom gliders data management n In 2012 : vertical profiles from 26 platforms.
Source Control in MATLAB A tool for tracking changes in software development projects. Stuart Nelis & Rachel Sheldon.
Using subversion COMP 2400 Prof. Chris GauthierDickey.
The MashMyData project Combining and comparing environmental science data on the web Alastair Gemmell 1, Jon Blower 1, Keith Haines 1, Stephen Pascoe 2,
Subversion Takes Back the Night How Version Control makes web development better.
CS 501 : An Introduction to SCM & GForge An Introduction to SCM & GForge Lin Guo
1 CMPT 275 Software Engineering Revision Control.
SubVersioN – the new Central Service at DESY by Marian Gawron.
Report on CARBOOCEAN data management B. Pfeil (UiB); A. Kozyr (CDIAC); R. Huber; N. Dittert; U. Schindler (all WDC-MARE); A. Brian; T. Carval (both IFREMER),
European Organization for Nuclear Research Source Control Management Service (Subversion) Brice Copy, Michel Bornand EN-ICE 13 May 2009.
Source Code Revision Control Software CVS and Subversion (svn)
Version Control with git. Version Control Version control is a system that records changes to a file or set of files over time so that you can recall.
Version Control with Subversion. What is Version Control Good For? Maintaining project/file history - so you don’t have to worry about it Managing collaboration.
CEOS/WGISS 20, Kyev, September 12-16, WTF-CEOP Implementation Plan #1 Status (WTF-CEOP first prototype, by JAXA) September 12, 2005 Osamu Ochiai.
Coordinated Energy and water-cycle Observations Peroject A Well Organized Data Archive System Data Integrating/Archiving Center at University of Tokyo.
AON Data Questionnaire Results 21 Respondents Last Updated 27 March 2007 First AON PI Meeting Scot Loehrer, Jim Moore.
SOCAT Surface Ocean CO 2 ATlas Are Olsen 1, Benjamin Pfeil 1, Dorothee Bakker 2, Maria Hood 3, Nicolas Metzl 4, Christopher Sabine 5, Alex Kozyr 6
Surface Ocean CO 2 Atlas -a showcase for transparent data management and international collaboration Benjamin Pfeil, Are Olsen, Dorothee Bakker, Steven.
Version Control. What is it? Software to help keep track of changes made to files Tracks the history of your work Helps you collaborate with others.
Design and Programming Chapter 7 Applied Software Project Management, Stellman & Greene See also:
…using Git/Tortoise Git
Information Systems and Network Engineering Laboratory II DR. KEN COSH WEEK 1.
Access to CARBOOCEAN and related data. Data is standardized and homogenised (parameters, metadata, etc.) quality checked well documented international.
Object-Oriented Analysis & Design Subversion. Contents  Configuration management  The repository  Versioning  Tags  Branches  Subversion 2.
CDIAC Global Ocean CO 2 Data Management Alex Kozyr Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory.
Integrated Model Data Management S.Hankin ESMF July ‘04 Integrated data management in the ESMF (ESME) Steve Hankin (NOAA/PMEL & IOOS/DMAC) ESMF Team meeting.
Version Control Systems with Subversion (SVN) and Tortoise.
Introduction to Version Control SE-2030 Dr. Rob Hasker 1 Based on material at and slides written.
What is Sure Stats? Sure Stats is an add-on for SAP that provides Organizations with detailed Statistical Information about how their SAP system is being.
WDC-MARE – World Data Center for Marine Environmental Sciences Data portal based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler,
WP 19:Data management Highlights B. Pfeil, A. Kozyr, M. Diepenbroek, N. Dittert, U. Schindler, R. Huber, A. Olsen & many more Partner 16 WDC-MARE Partner.
Observing System Monitoring Center (OSMC) Status Update April 2005 Steve Hankin – PMEL (co-PI) Kevin Kern – NDBC (co-PI)
Version Control with SVN Images from TortoiseSVN documentation
Uwe SchindlerGES 2007 – May 2-4, 2007 Data Information Service based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler 1, Benny Bräuer.
Dealing with Conflicting Updates in Git CS 5010 Program Design Paradigms “Bootcamp” Lesson 0.6 © Mitchell Wand, This work is licensed under a.
Version Control Systems. Version Control Manage changes to software code – Preserve history – Facilitate multiple users / versions.
Metadata Input Tool for CADIS Scientists and Data Managers by D. Stott August 8, 2007.
20081 Converting workspaces and using SALT & subversion to maintain them. V1.02.
Observing System Monitoring Center (OSMC) Work in progress in brief June 2005 Steve Hankin, Kevin O’Brien – PMEL.
12 CVS Mauro Jaskelioff (originally by Gail Hopkins)
An Introduction to the Argo Data Sytem South Pacific Workshop 11 – 14 October 2005 Mark Ignaszewski FNMOC.
WP 19 CARBOOCEAN data management Partner 16 WDC-MARE Partner 19 Ifremer SSC CDIAC Partner 1 UIB B. Pfeil (UiB), R. Huber; N. Dittert, U. Schindler (all.
Presentation OLOMOLA,Afolabi( ). Update Changes in CSV/SVN.
 A content management system ( CMS ) is a system providing a collection of procedures used to manage work flow in a collaborative environment. These.
Information Systems and Network Engineering Laboratory I DR. KEN COSH WEEK 1.
IGBP Carbon Data Syntheses and the problems they will (help us) solve Are Olsen Institute of Marine Research & Bjerknes Centre for Climate Research.
© CGI Group Inc. User Guide Subversion client TortoiseSVN.
1 Ivan Marsic Rutgers University LECTURE 2: Software Configuration Management.
A Practical Approach to Version Control for SQL Server Steve Jones SQLServerCentral Redgate Software.
Source Control Dr. Scott Schaefer. Version Control Systems Allow for maintenance and archiving of multiple versions of code / other files Designed for.
Fusion Tables.
Information Systems and Network Engineering Laboratory II
Statistical Information Systems Introducing SIS tool .Stat
LECTURE 2: Software Configuration Management
Source Control Dr. Scott Schaefer.
Version Control with Subversion (SVN)
User Guide PrimePortal – File Archive
Subversion.
Content Management Systems
LECTURE 3: Software Configuration Management
Getting Started with Git and Bitbucket
User Guide Subversion client TortoiseSVN
User Guide PrimePortal – File Archive
Title Month Year Chris Patel EMC Centera Strategic Alliance Manager
ORNL is Operated by UT-Battelle for DOE
VERSION CONTROL SVN (SubVersioN)
Palestinian Central Bureau of Statistics
Presentation transcript:

CARBOOCEAN Data management and SOCAT Benjamin Pfeil, Are Olsen, Jeremy Malzcyk, Steve Hanhin, Alex Kozyr and many others Partner 16 WDC-MARE Partner 19 Ifremer-Sismer SSC CDIAC Partner 1 UIB 4 th annual CARBOOCEAN meeting, Dourdon, France

What has been happening in 2008? Business as usual (new data from around 200 cruises were archived) SOCAT dataset has been growing and is ready for secondary QC CARINA is finished Collaboration with other projects EPOCA, SOPRAN, etc

SOCAT in numbers changes since last year December 2007December 2008 Number of cruises, time series station, buoys< 1300> 2100 Total amount of measurements (eg SST, SSS)5.5 million10 million Amount of CO 2 measurements< 4.5 million> 7 million Amount of cruises from CARBOOCEAN< 50 (4 %)> 300 (14.5 %) Amount of cruises from CARBOOCEAN PIs< 250 (20 %)> 500 (24 %) North America EuropeJapanothers Origin of data44 %36 %18 %< 2 %

SOCAT in numbers December 2007December 2008 Number of cruises, time series station, buoys< 1300> 2100 Total amount of measurements (eg SST, SSS)5.5 million10 million Amount of CO 2 measurements< 4.5 million> 7 million Amount of cruises from CARBOOCEAN< 50 (4 %)> 300 (14.5 %) Amount of cruises from CARBOOCEAN PIs< 250 (20 %)> 500 (24 %) Major increase of CARBOOCEAN data! Many CARBOOCEAN PIs agreed to include their data within SOCAT

How will data be made available for secondary QC and afterwards to the community?

Live Access Server and Subversion for SOCAT Jeremy Malczyk and Steve Hankin (NOAA/PMEL)

Live Access Server (LAS) A web server for visualizing gridded and in-situ data Multiple configurable data sources (netCDF, databases, etc) Highly configurable and extendable output Wide range of data products for interpolation, comparison, visualization, and analysis Updated version to the needs of SOCAT will soon be available

A centralized system for sharing information Commonly used to maintain a code base with a complete version history. Very powerful to manage conflicting changes from multiple users Will show differences between versions Will be able to retrieve older versions if mistakes are made What is Subversion (SVN)?

SOCAT Workflow Cruise files (text MatLab tables) and QC documents are committed to the Subversion Repository LAS provides visualization, analysis, and data retrieval tools to help scientists interact with the repository QC feedback is committed to the repository in QC documents Revised data files may be committed as well

Workflow within SOCAT SVN repository Cruise files QC Documents Scientists QC the data QC documents Revised QC documents (data quality flags and notes) Revised data files LAS

Why are we using Subversion? What are the benefits? Data can never be lost or overwritten Management responsibility is removed from a single entity (the repository can be mirrored or moved with its revision history intact) A complete revision history (of both the QC and data) is maintained for repeatability and documentation purposes Changes by individual users are tracked The data is accessible at any time

CARBOOCEAN SOCAT underway database But there is a large overlap Within SOCAT is just CARBOOCEAN data included if the PI agreed Small reminder from last year’s presentation ≠

Combined underway database SOCAT + Will be or is public data Non public data = CARBOOCEAN data

What happens in the future when CARBOOCEAN underway data will be published? SOCAT version x CARBOOCEAN Underway DB ∧

Already publically available CARBOOCEAN data > 50 cruises Available at the data portal and at CDIAC with detailed metadata If your data gets published please inform me

Thank you!

More Information SOCAT #SOCAT SVN LAS CARBOOCEAN