a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Funded by: This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 2.5 UK: Scotland License. To view a copy of this license, visit sa/2.5/scotland/ ; or, (b) send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Digital Archiving or Digital Archaeology? Dealing with Digital Records Maureen Pennock Digital Curation Centre

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Todays talk The DCC Background & Context What We Do Digital Preservation & Archiving Preservation & Curation: Issues & challenges Specific challenges for repositories receiving digital deposits Examples Proactive solutions

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 UK Digital Curation Centre JISC Circular 6/03 called for bids in digital curation JISC and the e-Science Core Programme funding for development, services and outreach in digital curation for a research programme Impetus to action Growth in e-Science activity and data creation Recognition that continuing access to digital information is needed

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Partners University of Edinburgh (lead site) Chris Rusbridge, Prof Peter Buneman University of Glasgow - HATII Prof Seamus Ross, Director of HATII and Erpanet University of Bath - UKOLN Dr Liz Lyon, Director of UKOLN Councils for the Central Laboratory of the Research Councils (CCLRC) Dr David Giaretta

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Objectives Lead a vibrant international research programme to improve quality in data curation and digital preservation Deliver effective, efficient and high demand services undertake evaluation of tools, methods, standards and policies work with the community to establish registries of tools and technical information Create an active, innovative and collaborative Associates Network Connect communities Universities and Research institutions Scientific data and documents International & cross-sector

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Collaborative Associates Network of Data Organisations curation organisations eg DPC

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Research Annotation in Databases Data archiving Socio-economic and legal issues Metadata extraction and curation Provenance and databases Data transformation, integration and publishing Security Supporting technologies Organisational and cultural challenges to digital curation

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Development DCC Approach to Digital Curation (white paper) – sets out the path for development activities: Monitoring international standards Development of a Representation Information Registry/Repository (DCC RIR) Development of recommendations for tools and methods for generating Representation Information Creating testbeds for digital curation tools Creating auditing and certification processes for trusted repositories

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Services Information Services Community-developed Digital Curation Manual Briefing Papers & FAQs Technology Watch Case Studies Best Practice Checklists Advisory Services Events: information days, workshops, training, conferences Helpdesk Audit and Certification Services

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Out of scope We dont Accept digital archival materials for curation, storage, or preservation Maintain a computer hardware museum Offer file migration or conversion Repair damaged data resources Carry out digital archaeology

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Summary Support and promote continuing improvement in the quality of data curation and preservation activity Nurture strong community relationships between practitioners, researchers, and curators Address digital curation from all aspects of the records life-cycle Develop and promote curation knowledge, tools and techniques Identify and research new organisational, technical, and supporting curation challenges

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Digital Curation Digital curation is all about maintaining and adding value to a trusted body of digital information for current and future use; specifically, we mean the active management and appraisal of data over the life-cycle of scholarly and scientific materials. Digital Curation brings a whole host of challenges The range of stakeholders that affect the survival of digital material cuts across the whole life-cycle Everyone plays an important role

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Preserving Digital Archives I The main problems: Technology obsolescence Hardware and software dependencies Leading to authenticity and integrity issues Fragility of digital media Bit deterioration Media breakage Lack of good practice examples for Data creation Data documentation

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Preserving Digital Archives II Some other problems: Cultural Creators dont (want to) understand the technical issues Contextual Metadata identifying resources is often insufficient Financial Difficult to anticipate costs of different activities Organisational Need for collaboration between parties is often overlooked Infrastructure and existing procedures often inadequate Legal Largely an unknown quantity

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Receiving Digital Archives I Specific challenges for organisations receiving externally originating digital deposits Possible lack of control/influence over resource creation resource & collection documentation transfer metadata technical metadata file storage format media storage format

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Receiving Digital Archives II Further challenges: Lack of required hardware to read discs Lack of required software to read file formats Lack of in-house knowledge about digital deposits May be a hybrid collection – paper & digital This causes further challenges Final archive may only accept limited formats You *might* have to convert the material for archiving Solving all of these issues may cause you money

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Digital archaeology for digital archiving Digital archaeology A form of data recovery Exploring what is buried below the surface Rescuing neglected and damaged data resources May reveal a lot of unwanted material, but… … may also reveals some valuable treasures May be necessary if repositories receiving digital deposits are to ascertain What material has been deposited Whether the material is valuable How to access and store the material for the longer-term

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Digital Archaeology - Examples NCUACS Received Old BBC Discs – obsolete storage format National Archives/CAMiLEON project Domesday Project – rescuing data from obsolete media and data formats NARA Hurricane Marilyn resulted in weather damaged diskettes and 12 inch WORM discs German Unification West German Archivists recovering obsolete East German data

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 What can you do? Find a reliable way to explore deposits and ascertain the value of the records Some records may be damaged or fragmented Rescue records where possible Store those that are not You never know what we will be capable of in the future Most data can be rescued – given enough time and money

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 How can you do it? Learn from experience What you have done in the past What others have done in the past Collaborate and share knowledge Be prepared To change processes To embark on a learning curve To communicate issues to potential depositors Develop in-house knowledge Cuts costs But be prepared to combine it with specialised external knowledge when necessary

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 What can help you? Identify and use tools developed by others DCC tools such as the RIR Erpanet tools on Ingest, Costing, Selecting Technologies, Developing a Policy Tools from the NAA Tools from the NLA Tools from the Digital Preservation Testbed Identify and use valuable case studies Ross & Gow: Digital Archaeology: Rescuing Neglected and Damaged Data Resources Jeremy John, British Library: Digital Manuscripts Project Studies presented at this workshop

a centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Thank you. Questions? Maureen Pennock Join the DCC Associates Network at