1. Capturing preservation metadata from institutional repositories Preserv Project Presented by Steve Hitchcock Intelligence Agents Multimedia Group, School of Electronics and Computer Science (ECS), Southampton University DCC Workshop on the Long-term Curation within Digital Repositories Cambridge, 6 July 2005

2. Abstract Preservation scenarios are often based on hypothetical situations, generalised applications such as digital libraries or cultural heritage organisations, or on specific applications such as digitisation. This presentation will consider the emerging but real scenario of preservation in the context of institutional repositories (IRs), being investigated by the JISC Preserv project. While on the surface this particular scenario may not seem to differ significantly from others, we will build a picture of relationships between repositories and preservation service providers to reveal what differences there may be with other scenarios and to understand the implications. We will use this analysis to inform the capture of some preservation metadata from IRs through the user deposit interface, perhaps the most critical data capture point in the IR preservation chain. Some initial ideas on formalising these IR preservation elements will be proposed for consultation, with a view to learning from, and possibly contributing to, the standard reference in this area, currently the PREMIS Working Group Data Dictionary for Preservation Metadata.

3. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005

4. Preservation Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005

5. Preservation storage media, media refreshing, reformatting, backups and disaster recovery, environment, audit, security, preservation strategy, migration, technology preservation, emulation, records management, etc. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005

6. Preservation storage media, migration, etc. Preserv partner British Library

7. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library

8. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Institutional repository Eprints.org, DSpace, FAIR, JISC DRs

9. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc.

10. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess

11. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess

12. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI

13. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI Access is still not the primary purpose of a preservation system Cornell OAIS tutorial

14. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI

15. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess M I/F(OAI) M I/F

16. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI

17. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface

18. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface It is important to build the concept of preservation from the outset" (JISC Circular 4/04)

19. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI Eprints deposit interface

20. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI Eprints deposit interface Contents of IRs Many types of digital objects, formats Versioning issues, some duplication Different degrees of moderation: institutional membership is selection baseline

21. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom

22. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom Influence/feedback

23. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom

24. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom Users IR managers and admins Heads of institutions Research funders Course leaders Teachers Education funders Stakeholders

25. Connecting IRs (content providers) and preservation services How far can we apply the OAIS model across our IR-preservation model? How can we embrace preservation metadata in this model? To what extent do these apply just to the preservation component? It looks as if many of the ideas focus on the preservation archive rather than the content provider How can we connect content providers with preservation services?

26. Preservation metadata is seldom shared across organizations Clifford Lynch "there has been some useful work done on metadata standards for preservation, although that work is not highly advanced. Part of the problem is that a lot of the work on preservation metadata has given rise to organizational guidance, the kinds of things you should think about as you attach metadata to objects when you want to preserve them, rather than hard specifics that would be more typical in interchange format, because preservation metadata today is seldom shared across organizations. * Since this talk was given there has been a great deal of progress in relevant areas here. I would point the interested reader at the work on METS, PREMIS, and the NISO Still Image Technical Metadata draft standard Preserving Digital Documents: Choices, Approaches, and Standards, Law Library Journal, 96 (4), 2004 http://www.aallnet.org/products/2004-40.pdf

27. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 OAIS model This model is not very different from the schematic sketched for Preserv, especially in terms of the core components – Ingest, Data Management, Archival Store, Access – but how effectively can OAIS be applied across different organisations?

28. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom Users IR managers and admins Heads of institutions Research funders Course leaders Teachers Education funders Stakeholders

29. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Ingest relies upon rules established to determine the metadata that must be present, the formats that are acceptable, the means that may be used for transferring objects, and the quality checks that must be performed Archival Storage functions are like storage functions that are performed in all kinds of digital storage environments, whether long-term preservation is a goal or not. The difference lies in the added rigor in error checking, media replacement, and disaster recovery. Data management provides the glue for the system by capturing and managing all of the metadata that is needed to operate the system. As in Archival Storage, the functions of Data Management are familiar to anyone who has worked with production databases. Access in OAIS may provide objects to an intermediary system that then interacts directly with users, or it may deliver directly to users. Access is still not the primary purpose of a preservation system. From Digital Preservation Management, 4B. The OAIS Reference Model. A Cornell tutorial http://www.library.cornell.edu/iris/dpworkshop/working/index.html

30. OAIS information model: AIP Archival Information Package (AIP): –Content Information Original target of preservation Information Object (Data Object & Representation Information) –Preservation Description Information (PDI) Other information (metadata) "which will allow the understanding of the Content Information over an indefinite period of time From Michael Day, Categories, uses and challenges of metadata and process documentation http://www.ukoln.ac.uk/preservation/presentations/2005/delos-summerschool/slides.ppt

31. OAIS information model: PDI Preservation Description Information Reference Information Provenance Information Context Information Fixity Information PDI Preservation Description Information (Figure 4-16) From Michael Day, Categories, uses and challenges of metadata and process documentation http://www.ukoln.ac.uk/preservation/presentations/2005/delos-summerschool/slides.ppt

32. A suggestion PDI – or somewhere - ought also to indicate what you want to do with the object, and perhaps act as the basis of selection for preservation services. What isn't clear is how these features could be incorporated.

33. Preservation metadata PREMIS = Preservation Metadata: Implementation Strategies Preservation metadata = "the information a repository uses to support the digital preservation process" The PREMIS Data Dictionary for Preservation Metadata (May 2005) http://www.oclc.org/research/projects/pmwg/

34. PREMIS data model Intellectual entities Objects Events Rights Agents

35. PREMIS Data Dictionary, v 1.0 Defines semantic units for Objects, Events, Agents and Rights –Object: objectIdentifier, preservationLevel, objectCategory, objectCharacteristics (format, significant properties, etc.), creatingApplication, storageMedium, environment (dependencies, hardware and software details, etc), relationship, … –Event: eventIdentifier, eventType (from a controlled list, e.g. ingestion, migration, normalization), eventDateTime, eventDetail, eventOutcomeInformation, linkingAgentIdentifier, … –Agent: agentIdentifier, agentName, agentType, … –Rights: permissionStatement, …

36. Limits to scope of PREMIS dictionary –Does not focus on descriptive metadata Domain specific and dealt with by many other schemes –Does not deal with technical metadata for all different types of digital file (left to format experts) –Does not consider in detail the business rules of a repository, e.g. roles, policies, and strategies (but this could be added to data model)

37. Resonance Points raised by PREMIS that have resonance for Preserv are: "Questions about business plans, policies, preservation strategies, as well as metadata" "Recognition of the need for automatic capture of metadata

38. Some questions Is there a need or scope for Preserv to describe semantic units within the PREMIS data dictionary that may be relevant to preservation metadata for IRs? Is what we need already included? For example, might the result of interaction with Pronom produce data residing in the Object entity (but noting PREMIS "Does not deal with technical metadata)? Which other entities might we contribute to? Events is one possibility. It's possible that some information we'd like to capture, e.g. funder, might fall within the scope of the Intellectual entity, which is outside the scope of PREMIS.

39. Preserv, Capturing preservation metadata from institutional repositories, DCC, Cambridge, 6 July 2005 Preservation storage media, migration, etc. Preserv partner British Library Other preservation service providers Preserv partners eprints.soton Oxford Univ. IRs, Eprints.org, DSpace, etc. User/authorUser/reader DepositAccess Machine interface OAI IR author deposit interface FormatFormat Format ID TNA + Pronom Users IR managers and admins Heads of institutions Research funders Course leaders Teachers Education funders Stakeholders Q. Preservation metadata or Selection metadata?

40. Preservation metadata or selection metadata? Who are the key stakeholders in selecting materials to preserve? Institutions – selection by admission to IR, or other criteria? Research funders – preserve outputs of a funded programme Authors – objective or subjective? Preservation business models for IRs: perhaps the answer lies in who pays?

41. Credits Southampton University Les Carr, Tim Brody, Jessie Hey, Steve Hitchcock British Library Richard Boulderstone, Adam Farquhar, Richard Masters National Archives Adrian Brown Oxford University David Price, Frances Boyle, Neil Jefferies, Michael Popham