Bill Roberts, PresDB 07 Database Preservation: A success story and an unsolved problem Bill Roberts 23 March 2007 PresDB, Edinburgh.

Slides:

Advertisements

Similar presentations

Data Publishing Service Indiana University Stacy Kowalczyk April 9, 2010.

Advertisements

DRIVER Long Term Preservation for Enhanced Publications in the DRIVER Infrastructure 1 WePreserve Workshop, October 2008 Dale Peters, Scientific Technical.

Introduction to Planets Hans Hofman Nationaal Archief Netherlands Prague, 17 October 2008.

Digital Preservation Tools for Repository Managers A practical course in five parts presented by the KeepIt project in association with Module 5, Trust.

Joint Information Systems Committee Digital Library Services BL/JISC Workshop Rachel Bruce JISC Programme Director The Digital Library and its Services,

UKOLN is supported by: JISC Information Environment update Repositories and Preservation Programme meeting, October 24-25, 2006 Rachel Heery UKOLN

T HE V ALUE OF E NTERPRISE S EARCH Robert Gill & Pieter-Jan De Boeck.

Organising and Documenting Data Stuart Macdonald EDINA & Data Library DIY Research Data Management Training Kit for Librarians.

APPLYING KNOWLEDGE Examples of practical results of using advanced (AI) solutions in law.

National Digital Repository ® Preserving the imperfect: reflections from NDAD and elsewhere Kevin Ashley Head of Digital Archives Group ULCC.

Transformations at GPO: An Update on the Government Printing Office's Future Digital System George Barnum Coalition for Networked Information December.

Preservation and Long-term access through Networked Services Adam Farquhar, The British Library iPres2006 Cornell University, October 2006.

Preservation Metadata Presentation to the CUL Metadata Working Group 15 February 2002 Nancy McGovern, Digital Preservation Officer, IRIS Bill Kehoe, Digital.

© HATII, University of Glasgow Uncertainty, Risk, Trust, and Digital Persistency Seamus Ross, Director HATII University of Glasgow 2006 NHPRC Electronic.

Active Data Curation in Libraries: Issues and Challenges ASEE ELD Presentation June 27, 2011 William H. Mischo & Mary C. Schlembach.

Co-funded by the European Union under FP7-ICT Alliance Permanent Access to the Records of Science in Europe Network Co-ordinated by aparsen.eu #APARSEN.

Libraries and Institutional Content Management Systems

Flow Cytometry Shared Resource Bioinformatics Improvements/Bluearc Storage.

Managing digital records Records Managers’ Forum 30 March 2009.

North Carolina Geospatial Data Archiving Project (NCGDAP) Project Overview Partnership –University library (NCSU) and state agency (NCCGIA) –$520,000 funding,

CONTI’2008, 5-6 June 2008, TIMISOARA 1 Towards a digital content management system Gheorghe Sebestyen-Pal, Tünde Bálint, Bogdan Moscaliuc, Agnes Sebestyen-Pal.

David Giaretta Associate Director (Development) Funders: DCC Development Digital Curation Centre a centre of expertise in data curation and preservation.

What is Workflow?  Workflow or Business Process Management (BPM) consists of Processes, States and Actions.  A Process (e.g. Customer Order fulfillment)

US GPO AIP Independence Test CS 496A – Senior Design Team members: Antonio Castillo, Johnny Ng, Aram Weintraub, Tin-Shuk Wong Faculty advisor: Dr. Russ.

Franklin Consulting Programme X The Innovation Base The e-Framework: What do they mean for programme management? Tom Franklin Franklin Consulting Richard.

San Diego Supercomputer CenterUniversity of California, San Diego Preservation Research Roadmap Reagan W. Moore San Diego Supercomputer Center

ResearchData.arts.ac.uk The Rococo Project – A case study.

Caring and Sharing Collaboration in Digital Curation outside North America Ross Harvey Simmons College, Boston Curation Matters: 17 June 2010.

Themes Architecture Content Metadata Interoperability Standards Knowledge Organisation Systems Use and Users Legal and Economic Issues The Future.

DDI-RDF Leveraging the DDI Model for the Linked Data Web.

IAEA International Atomic Energy Agency Open Data at NIS United Nations Library and Information Network for Knowledge Sharing (UN-LINKS) October.

Some comments on using research data in the social sciences Paul Lambert, School of Applied Social Science, University of Stirling, 25 March 2013.

Because good research needs good data Funded by: Digital Curation for Researchers, 28th February 2013 The Shifting Research Data Management Policy Landscape.

Long-Term Knowledge Retention Joshua Lubell Manufacturing Systems Integration Division, NIST FIRM’s Forum at FOSE March 20, 2007.

Robert Spindler University Archivist Arizona State University Libraries

Centre for eResearch The University of Auckland Research Data–Preserve, Share, Reuse, Publish, or Perish Mark Gahegan Director, Centre for eResearch 24.

1 Digital Preservation Testbed Database Preservation Issues Remco Verdegem Bern, 9 April 2003.

National Geospatial Digital Archive Greg Janée University of California at Santa Barbara.

UKOLN is supported by: Digital Preservation Benefits Tools Project Dissemination Workshop Dr Liz Lyon, Associate Director, UK Digital Curation Centre Director,

Data in the NEES Data Repository Conditions for Current and Future Use and Re-Use Quake Summit 2012, Boston, Massachusetts July 12, 2012 Stanislav Pejša.

Fachstelle ARELDA Schweizerisches Bundesarchiv 1 SIARD: Software Invariant Archiving of Relational Databases at the Swiss Federal Archives Contents: 

GPO’s Federal Digital System December 10, 2009 U.S. Government Printing Office.

Lucia Lötter NeDICC 26 February 2014 Lucia Lötter NeDICC 26 February 2014 Social science that makes a difference Research Methodology Centre Research Data.

Data Management & the Library. FACT #1 Research is increasingly digital and produces digital data.

Integrating Access to Digital Content Sarah Shreeves University of Illinois at Urbana-Champaign Visual Resources Association 23 rd Annual Conference Miami.

DSpace vs Fedora Ralph LeVan OCLC Research. What Do You Want From a Repository? How do you create your metadata? How do you assemble your objects? How.

Selene Dalecky March 20, 2007 FDsys: GPO’s Digital Content System.

How to Implement an Institutional Repository: Part II A NASIG 2006 Pre-Conference May 4, 2006 Technical Issues.

Funded by: © AHDS Preservation in Institutional Repositories Preliminary conclusions of the SHERPA DP project Gareth Knight Digital Preservation Officer.

Hussein Suleman University of Cape Town Department of Computer Science Digital Libraries Laboratory February 2008 Data Curation Repositories:

National Library of the Czech Republic Integration of digital materials into EDL Adolf Knoll National Library of the Czech Republic Helsinki CENL Workshop.

11 Researcher practice in data management Margaret Henty.

Lifecycle Metadata for Digital Objects November 15, 2004 Preservation Metadata.

Infrastructure Breakout What capacities should we build now to manage data and migrate it over the future generations of technologies, standards, formats,

Sharing Research Data with: OC Data Portal: ocdp.lib.uci.edu UC Irvine Dash: dash.lib.uci.edu Dan Tsang, Data Librarian Julia Gelfand, Applied Sciences.

Data Preservation at Rutherford Lab David Corney 9 th July 2010 KEK.

Donald G. Davis Collection 392K Amy Baker, Megan Peck, Zach Vowell.

Metrics for Repository Impact Mark MacGillivray Cottage Labs

Research Data Management 26 th April 2016 Federica Fina, Data Scientist, University of St Andrews Library.

Informatics for Scientific Data Bio-informatics and Medical Informatics Week 9 Lecture notes INF 380E: Perspectives on Information.

Session 2b, 25 November 2015 eChallenges e-2015 Copyright 2015 The National Archives of Estonia Current lack of interoperability among submission information.

Building A Repository for Digital Objects

An Introduction to Tessella and The Safety Deposit Box Platform

Research Data Context Preservation in SCAPE

Active Data Management in Space 20m DG

SowiDataNet - A User-Driven Repository for Data Sharing and Centralizing Research Data from the Social and Economic Sciences in Germany Monika Linne, 30.

Implementing an Institutional Repository: Part II

Automation and Scalability in Digital Preservation

Implementing an Institutional Repository: Part II

Big data and Analytics for non-financial information

Presentation transcript:

Bill Roberts, PresDB 07 Database Preservation: A success story and an unsolved problem Bill Roberts 23 March 2007 PresDB, Edinburgh

Bill Roberts, PresDB 07 Digital preservation: why is it hard? PEBKAC:

Bill Roberts, PresDB 07 MeThem

Bill Roberts, PresDB 07 Databases: what to preserve? Contents of tables: the data Structure Semantics Context Business/scientific process volatility-permanence-databases-en.pdf OAIS representation information

Bill Roberts, PresDB 07 JET data preservation Similar experimental processes repeated many times, 1983  Well defined format for processed data 2000: IBM mainframe  Unix (~8 TB) New NetCDF/XDR file format + relational metadata database Old API still supported All data still accessible Fusion Engineering and Design, Volume 60, Issue 3, June 2002, Richard Layne and Martin Wheatley

Bill Roberts, PresDB 07 Why a success? Single organisation Small number of formats Carefully designed from the start Continuously managed Still in active use Data curators part of user community MeThem

Bill Roberts, PresDB 07 Multinational company data Regulatory IP protection Litigation Knowledge Office documents Instrument data Records of experiments Analysed data Regulatory submissions Lab notebooks Mostly in relational databases

Bill Roberts, PresDB 07 “Easy vs Hard” Few activities Consistent approach Control of data formats Standardisation Record of data Many activities Rapid changes of science, technology, methods, formats, management Formats driven externally Freedom to innovate Trail of analysis and basis of decisions

Bill Roberts, PresDB 07 Solutions?

Bill Roberts, PresDB 07 Active Preservation Storage Archive Management Workflow Automation Characterisation tools Preservation action tools Planning tools Testbed

Bill Roberts, PresDB 07 Data silos Representation information! ‘Merge’ the silos: Interoperability now between groups Interoperability between now and future

Bill Roberts, PresDB 07 RECOMMENDATIONS: Design for data interoperability and re-use Consider whole life-cycle cost Automate metadata harvesting Make it easy for data creators to do the right thing

Bill Roberts, PresDB 07