WHOI and SIO (II): Next Steps Towards Multi-Institution Archiving of Shipboard and Deep Submergence Vehicle Data (IN51A-0306) The Woods Hole Oceanographic.

Slides:

Advertisements

Similar presentations

National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.

Advertisements

Prototype Phase SIO Accomplishments

Rolling Deck to Repository: Transforming the United States Academic Fleet Into an Integrated Global Observing System Suzanne M. Carbotte, Robert Arko,

The Storage Resource Broker and.

ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.

Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.

NATIONAL PARTNERSHIP FOR ADVANCED COMPUTATIONAL INFRASTRUCTURE SAN DIEGO SUPERCOMPUTER CENTER Particle Physics Data Grid PPDG Data Handling System Reagan.

San Diego Supercomputer CenterNational Partnership for Advanced Computational Infrastructure1 Grid Based Solutions for Distributed Data Management Reagan.

Ocean Data Interoperability Platform EU-US-Australia collaborative project Grant Number: Call: FP7-INFRASTRUCTURES INFSO Activity: INFRA :

Background Chronopolis Goals Data Grid supporting a Long-term Preservation Service Data Migration Data Migration to next generation technologies Trust.

A Framework for Earth Science Search Interface Development Designing and Implementing S2S Eric Rozell, Tetherless World Constellation, RPI.

A Very Brief Introduction to iRODS

Chronopolis: Preserving Our Digital Heritage David Minor UC San Diego San Diego Supercomputer Center.

Chad Berkley National Center for Ecological Analysis and Synthesis (NCEAS), University of California, Santa Barbara February.

Abstract Complex real-time seismic networks often develop an extensive set of dataflow connections, especially when one includes the virtual networks established.

Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.

T.Sharon-A.Frank 1 Internet Resources Discovery (IRD) FDL Examples.

Dogan Seber, PhD San Diego Supercomputer Center University of California, San Diego I. DLESE Library II. DISCOVER OUR EARTH Earth Science Resources for.

An Oceanographic Event Logger James R. Wilkinson and Karen S. Baker Scripps Institution of Oceanography, University of California San Diego Field Practices.

Tools for e-Research Mat Wyatt. 2 e-Research Sensor nets data compute… Models/ software/ workflows colleagues instruments.

ABSTRACT Real-time systems applied to seismic data acquisition, asynchronous processing, and data archiving tasks have clearly demonstrated their utility.

January, 23, 2006 Ilkay Altintas

Case History: Library of Congress Audio-Visual Prototyping Project METS Opening Day October 27, 2003 Carl Fleischhauer Office of Strategic Initiatives.

Multi-Institution Testbed for Scalable Digital Archiving NSF CISE/Library of Congress DIGARCH Award Stephen Miller Scripps Institution of Oceanography.

Helen Glaves (NERC- BGS), Dick Schaap (MARIS), Robert Arko (LDEO) and Roger Proctor (IMOS)

San Diego Supercomputer CenterUniversity of California, San Diego Preservation Research Roadmap Reagan W. Moore San Diego Supercomputer Center

Project Planning Workshop Woods Hole July 11-13, 2005 Multi-Institution Testbed for Scalable Digital Archiving NSF CISE/Library of Congress DIGARCH Award.

Jan Storage Resource Broker Managing Distributed Data in a Grid A discussion of a paper published by a group of researchers at the San Diego Supercomputer.

Rule-Based Data Management Systems Reagan W. Moore Wayne Schroeder Mike Wan Arcot Rajasekar {moore, schroede, mwan, {moore, schroede, mwan,

IUScholarWorks is a set of services to make the work of IU scholars freely available. Allows IU departments, institutes, centers and research units to.

Z-Geoinfo Inc. Capability Briefing June 21, 2011.

San Diego Supercomputer Center SDSC Storage Resource Broker Data Grid Automation Arun Jagatheesan et al., San Diego Supercomputer Center University of.

Fedora Content Models for the National Science Digital Library Data Repository Fedora User’s Group Meeting Copenhagen, September 28, 2005 Carl Lagoze Cornell.

Production Data Grids SRB - iRODS Storage Resource Broker Reagan W. Moore

Mind the Gap: Finding Data Across Decades and Disciplines with the SSDB Stephen P. Miller 1, P. Dru Clark 1, Jacob M. Perez 1, Aaron D. Sweeney 1, John.

Mind the Gap: Finding Data Across Decades and Disciplines with the SSDB Stephen P. Miller 1, P. Dru Clark 1, Jacob M. Perez 1, Aaron D. Sweeney 1, John.

National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.

National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Archive for the NSDL Reagan W. Moore Charlie Cowart.

Policy Based Data Management Data-Intensive Computing Distributed Collections Grid-Enabled Storage iRODS Reagan W. Moore 1.

Uwe SchindlerGES 2007 – May 2-4, 2007 Data Information Service based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler 1, Benny Bräuer.

Rolling Deck to Repository II: Getting Control of Provenance and Quality AGU Poster IN43A-1169 AGU Fall Meeting December 17, Stephen.

This presentation describes the development and implementation of WSU Research Exchange, a permanent digital repository system that is being, adding WSU.

June 20, 2007ESRI Intl. User Conference Dawn Wright - Oregon State University Val Cummins - Coastal & Marine Resources Centre, IRELAND Liz O’Dea - Coastal.

The Global Land Cover Facility is sponsored by NASA and the University of Maryland.The GLCF is a founding member of the Federation of Earth Science Information.

Introduction to The Storage Resource.

Millman—Nov 04—1 An Update on Digital Libraries David Millman Director of Research & Development Academic Information Systems Columbia University

E-Curator: A Web-based Curatorial Tool Ian Brown, Mona Hess Sally MacDonald, Francesca Millar Yean-Hoon Ong, Stuart Robson Graeme Were UCL Museums & Collections.

System Development & Operations NSF DataNet site visit to MIT February 8, /8/20101NSF Site Visit to MIT DataSpace DataSpace.

Advanced Semantic Technologies Project S2S Framework Evaluation Eric Rozell, Tetherless World Constellation.

1 1 NOAA Office of Ocean Exploration End-to-End Data Management: A Success Story NOAA Tech Conference November 2005 Susan Gottfried National Coastal Data.

Partnerships in Innovation: Serving a Networked Nation Grid Technologies: Foundations for Preservation Environments Portals for managing user interactions.

Glossary WMS – OGC Web Mapping Services WFS – OGC Web Feature Services XML- Extensible Markup Language OGC – Open GIS Consortium ADN –

SIOExplorer Stephen Miller Scripps Institution of Oceanography USA International Data Exchange Workshop Building a Global Data Network for Studies of Earth.

The Storage Resource Broker and.

SAN DIEGO SUPERCOMPUTER CENTER Replication Policies for Federated Digital Repositories Robert H. McDonald Chronopolis Project Manager

SSDB Progress Report Site Survey Panel Meeting CIRE, Sapporo, Japan July 22, 2006 John Weatherford San Diego Supercomputer Center Subcontract to IODP-MI.

The launching of an expedition has its own brand of excitement, with the sound of the main engines firing up, and the lifting of the gangway in a foreign.

Physical Oceanography Distributed Active Archive Center THUANG June 9-13, 20089th GHRSST-PP Science Team Meeting GHRSST GDAC and EOSDIS PO.DAAC.

Working prototype Multi-Institution Testbed for Scalable Digital Archiving Three institutions are working together to rescue at-risk media, establish interoperability,

Rolling Deck to Repository (R2R): How to Systematically Document Quality for the New Era of Data Re-Usability? AGU Poster IN21B-1048 AGU Fall Meeting December.

Biological and Chemical Oceanography Data Management Office slide 1 of 10 U.S. GEOTRACES Data Management Cyndy Chandler BCO-DMO ~ WHOI 23 September 2008.

Collection-Based Persistent Archives Arcot Rajasekar, Richard Marciano, Reagan Moore San Diego Supercomputer Center Presented by: Preetham A Gowda.

SIOExplorer: Digital Library Projects R/V Alexander Agassiz November, 1907 UCSD Libraries Scripps Institution of Oceanography San Diego Supercomputer Center.

Developing a Dark Archive for OJS Journals Yu-Hung Lin, Metadata Librarian for Continuing Resources, Scholarship and Data Rutgers University 1 10/7/2015.

Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.

CONTENTdm A proven solution September A complete digital collection management software solution Stores, manages and provides access for all digital.

Data Grids, Digital Libraries and Persistent Archives: An Integrated Approach to Publishing, Sharing and Archiving Data. Written By: R. Moore, A. Rajasekar,

Sampling Method #3 Submersibles.

Technical Issues in Sustainability

Presentation transcript:

WHOI and SIO (II): Next Steps Towards Multi-Institution Archiving of Shipboard and Deep Submergence Vehicle Data (IN51A-0306) The Woods Hole Oceanographic Institution (WHOI), Scripps Institution of Oceanography (SIO), and the San Diego Supercomputer Center (SDSC) are working together on a digital data archiving and preservation project that will establish interoperability between existing data repositories and provide community access to shipboard and deep submergence vehicle data. The prototype system establishes a two-node, federated data network initially populated with WHOI and SIO data from the Galapagos Island area. SIO and SDSC are contributing software and experience developed for their SIOExplorer application including data and metadata harvesting, federated digital libraries, and related web-based and Java-based clients. WHOI is contributing software developed for a number of its GeoBrowser technologies and GIS Server based applications. 1.Source data files are copied from various sources to a local server either at WHOI or SDSC. 2.A single program, metadata_Creator.pl, harvests source- specific metadata from raw data files derived from sensors and samples. 3.These source-specific metadata are then operated on by another program ADOcreator.bash (&.pl) to generate the {ADO, mif} pairs that are then loaded into the digital library. 4.The data files are ingested as arbitrary data objects (ADOs) into the Storage Resource Broker (SRB) and the metadata (*.mif) into the metadata catalogue implemented using a Postgres relational database. 5.Web-browser and Java clients (shown in blue) are used to access the archive. Maffei, Helly, Clark, Detrick, Gaylord, Goldsmith, Lemmond, Lerner, Miller, Norton, Tivey, Walden Woods Hole Oceanographic Institution, Scripps Institution of Oceanography, San Diego Supercomputer Center We thank the DIGARCH Program of the National Science Foundation and the Library of Congress for their support (NSF IIS ). This figure shows how cruise, Alvin, Jason2 and data are fed into WHOI’s GeoBrowser and GIS Server applications. This figure shows how metadata is harvested from SIO cruise data files, ingested into the digital library and then made accessible via the SIOExplorer Graphical User Interfaces (GUIs). OverviewCurrent SIOExplorer Cruise Data Workflow Current WHOI GeoBrowser & GIS Data Workflow Acknowledgements Examples of Current Web Clients Combined SIOExplorer, WHOI GeoBrowser, and WHOI GIS Data Workflow 1.WHOI cruise/vehicle data files collected from shipboard dataloggers and other locations. 2.Geobrowser applications sample some of the cruise/vehicle data (video, shipboard data, etc.) in order to create metadata in form of Electronic Index Cards (EICs). 3.Historical data transcribed from archives. 4.GeoBrower application provides access to Electronic Index Card (EIC) snapshots and summaries of data collected. 5. WHOI Cruises application provides access to cruise/vehicle metadata via web-based GIS map interface. 1.WHOI and SIO Galapagos-related cruise/vehicle data is collected from various sources. 2.WHOI and SIO Galapagos cruise/vehicle data is re- cast into a Common Cruise Canonical Directory Structure (CCCDS) 3.Metadata plugin routines generate intermediate metadata files for collection, cruise, file, sensor (video, ctd, multi-beam, underway-data, etc.), and other data. We call these intermediate entities Oceanographic Metadata Files (OMFs). 4.WHOI GeoBrowser routines employ CCCDS and OMFs to create Electronic Index Card (EIC) collections. 5.WHOI GIS Server routines employ CCCDS and OMFs to create entries in the GIS server. Sample Search for Galapagos Data WHOI’s Jason2 Virtual Control Van WHOI’s Cruises GIS Server WHOI’s Alvin FrameGrabber WHOI’s Shipboard DataGrabber SIO’s Digital Library WebForm SIO’s Digital Library JAVA Front end 6.SIOExplorer routines use the CCCDS and OMFs to create ADOs and MIFs and ingest them into the federated digital archive. 7.During the prototype phase of the project, the web client front-ends will be updated so that they can access the three server types via their existing Application Programming Interfaces (APIs). During the next phase of this work, a common Web services API will be defined and implemented on each of the servers. Dive 4119 Step 1 (above) - use SIOExplorer to find all SIO/WHOI ship tracks in area; locate AT11-27 Step 4 (below) - use PetDB to find chemical composition of sample Step 2 (above) - use WHOICruise GUI to locate Alvin dive 4119 on AT11-27 Step 3 (left) - use Alvin FrameGrabber to locate detailed dive track and video of basalt sample being taken