WDC-MARE – World Data Center for Marine Environmental Sciences Data portal based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler,

Slides:

Advertisements

Similar presentations

Technical Highlights 25th August 2011 Sebastian Peters German National Library of Science and Technology.

Advertisements

IRRA DSpace April 2006 Claire Knowles University of Edinburgh.

IST Humboldt University Berlin, Germany – Computer and Media Service – Electronic Publishing Group Birgit Matthaei, 4th Sept. 2003, Bath,

IST Humboldt-University, Berlin, Germany - Electronic Publishing Group - Computing Centre / University Library Susanne Dobratz, 28. March.

Heinrich Stamerjohanns Institute for Science Networking Distributed Open Archives Dr. Heinrich Stamerjohanns Institute for Science Networking at the University.

Possibility in Digital Collection Management Introduction to CONTENTdm TM Hitoshi Kamada University of Arizona Presentation for OCLC-CJK Users Group Annual.

Open Scholarship 2006 Bielefeld Academic Search Engine a Scientific Search Service for Institutional Repositories Open Scholarship 2006 New Challenges.

OAI and Publishers metadata Using the static repositories approach to disclose small journals.

14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.

1 panFMP - Ein XML-basiertes Framework für Metadaten- Portale Vortrag und „hands-on“ Seminar am GFZ Potsdam Uwe Schindler MARUM – Universität Bremen PANGAEA.

DSpace Devika P. Madalli DRTC, ISI Bangalore.

ARCHIMÈDE Presented by Guy Teasdale Directeur, Services soutien et développement Bibliothèque de l’Université Laval CARL Workshop on Institutional Repositories.

The Open Archives Initiative and OAIster: Past, Present and Future Kat Hagedorn University of Michigan Libraries April 6, 2006.

NAL-Institutional Repository: A Case Study CSIR Metadata Harvester I.R.N. Goudar Head, ICAST, NAL National Symposium on Open Access and.

The Bremen core repositories and data curation with PANGAEA Hannes Grobe Alfred Wegener Institute for Polar and Marine Research.

Implementing search with free software An introduction to Solr By Mick England.

DEF System Architecture XML Web Services Fedora and the Zebra Search Engine in an OAI Eprints Application by Gert Schmeltz Pedersen, DTV

WP 9 (former Task 1b of WP 1): Data infrastructure Robert Huber UNI-HB Esonet 2nd all regions workshop, Paris

IUScholarWorks is a set of services to make the work of IU scholars freely available. Allows IU departments, institutes, centers and research units to.

E-Learning standards and meta-data: Case study ดร. น้ำทิพย์ วิภาวิน Sripatum University Library.

Indo-US Workshop, June23-25, 2003 Building Digital Libraries for Communities using Kepler Framework M. Zubair Old Dominion University.

University of North Texas Libraries Building Search Systems for Digital Library Collections Mark E. Phillips Texas Conference on Digital Libraries May.

World Data Center for Marine Environmental Sciences.

19/10/20151 Semantic WEB Scientific Data Integration Vladimir Serebryakov Computing Centre of the Russian Academy of Science Proposal: SkTech.RC/IT/Madnick.

Royal Netherlands Academy of Arts and Sciences NARCIS, Integrating CRIS, OAI and Web Crawling Elly Dijk, Arjan Hogenaar and Marga van Meel Department of.

Design of a Search Engine for Metadata Search Based on Metalogy Ing-Xiang Chen, Che-Min Chen,and Cheng-Zen Yang Dept. of Computer Engineering and Science.

INTEGRATED OCEAN DRILLING PROGRAM MANAGEMENT INTERNATIONAL International Data Exchange Workshop – Kiel, Germany – May 9-11, 2007 SEDIS Scientific Earth.

Unit no. 5 Digital Library Adolf Knoll National Library of the Czech Republic © Adolf Knoll, National Library of the Czech Republic.

Introduction to Web Services Eric Lease Morgan University Libraries of Notre Dame June 24, 2005.

International Data Exchange Workshop, Kiel, PANGAEA Publishing Network for Geoscientific & Environmental Data.

1 By: Suman Negi, Technical Officer ‘B’ DESIDOC, DRDO, Delhi Presentation at NACLIN 14 (During 9-11 December 2014, Pondicherry) Design and Development.

IUScholarWorks Technical Overview Randall Floyd Digital Library Program Programmer/Database Administrator.

Uwe SchindlerGES 2007 – May 2-4, 2007 Data Information Service based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler 1, Benny Bräuer.

Web Portal Design Workshop, Boulder (CO), Jan 2003 Luca Cinquini (NCAR, ESG) The ESG and NCAR Web Portals Luca Cinquini NCAR, ESG Outline: 1.ESG Data Services.

The multilingual catalogue of digital cultural heritage in Europe Pier Giacomo SOLA.

This presentation describes the development and implementation of WSU Research Exchange, a permanent digital repository system that is being, adding WSU.

Caltech CODA CODA: Collection of Digital Archives Caltech Scholarly Communication.

OAI Overview DLESE OAI Workshop April 29-30, 2002 John Weatherley

Core Integration Web Services Dean Krafft, Cornell University

Mercury – A Service Oriented Web-based system for finding and retrieving Biogeochemical, Ecological and other land- based data National Aeronautics and.

Registering Earth Science Data and Data Related Services Using NASA’s Global Change Master Directory (GCMD) Tyler Stevens (GIS/Services Coordinator) ESIP.

Hussein Suleman University of Cape Town Department of Computer Science Digital Libraries Laboratory February 2008 Data Curation Repositories:

GEOSS Global Earth Observation System of Systems.

The Open Archives Initiative Marshall Breeding Director for Innovative Technologies and Research Vanderbilt University

H. Widmann (M&D) Data Discovery and Processing within C3Grid GO-ESSP/LLNL / June, 19 th 2006 / 1 Data Discovery and Basic Processing within the German.

DSpace - Digital Library Software

 Allow access to observational, model and forecast data  Likely to be in the form of a portal with consistent meta data and pointer to other online location,

Serenate1 The librarian’s view Raf Dekeyser K.U.Leuven.

Data portal technology standards and protocols involved.

VuFind Digital Libraries à la Carte International Ticer School 2009 Tilburg University 31 July, 2009 Benoit PAUWELS Université Libre de Bruxelles (ULB)

Steven Perry Dave Vieglais. W a s a b i Web Applications for the Semantic Architecture of Biodiversity Informatics Overview WASABI is a framework for.

Metadata-based Discovery: Experience in Crystallography UKOLN is supported by: Monica Duke UKOLN, University of Bath, UK A centre of.

1 CS 430: Information Discovery Lecture 26 Architecture of Information Retrieval Systems 1.

CERN Document Server 19 tth January 2006 CERN Document Server Jean-Yves Le Meur 19 th January 2006.

Breeda Herlihy, IR Manager, UCC Library. UCC selected DSpace in 2008 Software selection group Staff from Library IT, Computer Centre, Special Collections,

1 ABCD as a digital library tool An introduction on the concept and implementation by Egbert de Smet Univ. of Antwerp.

Introduction: AstroGrid increases scientific research possibilities by enabling access to distributed astronomical data and information resources. AstroGrid.

High performance, full-featured text search engine written in Java. Technology suitable for nearly any application requiring full-text search, especially.

TRIG: Truckee River Info Gateway Dave Waetjen Graduate Student in Geography Information Center for the Environement (ICE) University of California, Davis.

Bielefeld Academic Search Engine

Flanders Marine Institute (VLIZ)

Repository Software - Standards

Introduction, Features & Technology

Building Search Systems for Digital Library Collections

VI-SEEM Data Repository

Data Discovery Boulder, CO May 15, 2006 Scott Ritz

Introduction to DSpace

OAI and Metadata Harvesting

The New Face of Information Retrieval: The Ankara University Open Access Platform Prof. Dr. Sekine Karakaş Prof. Dr. Doğan.

ORNL is Operated by UT-Battelle for DOE

Presentation transcript:

WDC-MARE – World Data Center for Marine Environmental Sciences Data portal based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler, Michael Diepenbroek, MARUM, University of Bremen, Germany EGU 2006, Vienna, WDC-MARE – World Data Center for Marine Environmental Sciences

Data Portals WDC-MARE with its information system PANGAEA provides data portals for several EU/international projects: CARBOOCEAN, EUR-OCEANS, IODP Problem: Not all data are stored centralized, so all datasets provided in portals must be consolidated from different sources!

WDC-MARE – World Data Center for Marine Environmental Sciences Example: CARBOOCEAN data portal Data stays at the data providers Metadata is harvested by the portal Search queries are handled by the centralized catalogue Scientist gets link to data at the provider

WDC-MARE – World Data Center for Marine Environmental Sciences Open Archives Protocol The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed by the Open Archives Initiative. uses it during web crawling ( Scholar) Almost all digital libraries support it (most famous ones: arXiv and the CERN Document Server) Very simple to implement (XML over HTTP based) Repository software for databases or file system metadata providers is widely available

WDC-MARE – World Data Center for Marine Environmental Sciences Current OAI-PMH software 1.Limited to Dublin Core metadata (libraries)! 2.Limited full text search functionality due to relational databases in the background! 3.No geographic retrievals (because of Dublin Core limitation)! 4.End user interface is part of the software, this limits usability in CMS systems

WDC-MARE – World Data Center for Marine Environmental Sciences Requirements for portal software 1.Open for any XML metadata format 2.Any mappings to document fields should be done by XPath 3.Possibility to map incompatible XML schemas during harvesting by XSL 4.No relational database, only a full text search engine, that contains everything needed for operation 5.Range queries for specific fields (date/time or numeric) 6.Web service interface for the end user software that is accessible from any language (Java/JSP, PHP, Perl,...)

WDC-MARE – World Data Center for Marine Environmental Sciences Lucene XML- Files OAI- PMH OAI- PMH OAI- Harvester OAI- Harvester Filesystem- Harvester OAI protocol in HTTP OAI protocol in HTTP (specific set) filesystem directory, FTP,… Mini PanHTTP Server Jetty HTTP Server Tomcat Apache Axis Virtual Index Virtual Index XSL Portal 1 (Webserver, PHP) Portal 2 (Webserver, JSP) Stored: xmldata (same format everywhere, XSL before indexing), identifier, lastModified, sets Searchable: field1: “/oai_dc:dc/dc:author” field2: “/oai_dc:dc/dc:title” field3: “java:org.test.LatLon.parse(/oai_dc:dc/dc:coverage)” * default: “.” *) xmlns:java=“ MetadataPortal Java Package

WDC-MARE – World Data Center for Marine Environmental Sciences Metadata standard harvested for search: DIF v9.4 Searchable fields: Bounding box, date/time, parameters, authors, investigators, title Data centers: World Data Center for Marine Environmental Sciences (WDC-MARE), University of Bremen and Alfred-Wegener- Institute in Bremerhaven, Germany Carbon Dioxide Information Analysis Center (CDIAC), Environmental Sciences Division at Oak Ridge National Laboratory, USA French National Oceanographic Data Centre, SISMER (Systèmes d'Informations Scientifiques pour la Mer) at the Ifremer in Brest, France CARBOOCEAN Data Portal

WDC-MARE – World Data Center for Marine Environmental Sciences Thank you!