DRS 2 Metadata Migration June 25, 2013

Agenda Introduction Preliminary results - content analysis Metadata options Next steps Questions

INTRODUCTION

Reason for metadata migration Different data model – File -> Object (a coherent set of content that is considered a single intellectual unit for purposes of description, use and/or management: for example a particular book, web harvest, serial or photograph.) Different metadata schemas – Many locally-defined -> community-standard Different packaging of metadata – Use of METS in some cases -> consistent use of METS

Path to metadata migration Analysis Metadata Content Users Prototype Proof-of- concept Time estimates Migration plan Sequence Schedule Develop tools Dashboard Object builders Metadata migration We are here

Key feedback points Analysis Metadata Content Users Prototype Proof-of- concept Time estimates Migration plan Sequence Schedule Develop tools Dashboard Object builders Metadata migration Technical options Process options

Timing Analysis Metadata Content Users Prototype Proof-of- concept Time estimates Migration plan Sequence Schedule Develop tools Dashboard Object builders Metadata migration Next 3 months

What does it involve? Aggregate DRS1 files into objects – Different object types = content models Generate an object descriptor per object

Document example PDF file

Document example PDF file New object (content model = DOCUMENT)

Document example PDF file Descriptor file New object (content model = DOCUMENT)

Still image example Archival master image file

Still image example Archival master image file Production master image file

Still image example Archival master image file Deliverable image file Production master image file

Still image example Archival master image file New object (content model = STILL IMAGE) Deliverable image file Production master image file

Still image example Archival master image file Descriptor file Deliverable image file Production master image file New object (content model = STILL IMAGE)

Aggregate DRS1 files into objects One content file per object – Color profile – Document – Google document container 1 – Google document container 2 – Google document container 3 – Opaque container – Text

Aggregate DRS1 files into objects Multiple content files per object – Audio – Web harvest – Biomedical image – PDS document – Target image – MOA2 – Still image

Generate object descriptors METS format – Embedded schemas (PREMIS, MODS, MIX, etc.) Metadata sources – DRS1 database – DRS1 METS files where they exist – Examining the content files – Catalog records?

PRELIMINARY RESULTS: CONTENT ANALYSIS

Preliminary content analysis Conceptually “built” objects for 13/14 content models (~36 million / 44 million files) – All but still image – Order helps! Still Image MOA2 Biomedical Image PDS Document

Preliminary content analysis 1,091,670 objects from 36,190,120 files – ~33 files per object Relatively few surprises but content analysis is not complete

Content cleanup MOA2 files (8,024) Index maps (2,686) Entity files (1) Merged PDS descriptors (22,203)

Content cleanup Orphaned target image (5), target description files (4) Orphaned audio files (71)

METADATA OPTIONS

O DRS1DRS2 e.g., billingCode ownerCode accessFlag tech metadata owner-suppliedName role purpose quality usageClass e.g., accessFlag tech metadata owner-suppliedName role processing quality usageClass e.g., billingCode ownerCode owner-suppliedName FILE INFO OBJECT INFO DESCRIPTOR

O DRS1DRS2 e.g., billingCode ownerCode accessFlag tech metadata owner-suppliedName role purpose quality usageClass e.g., accessFlag tech metadata owner-suppliedName role processing quality usageClass e.g., billingCode ownerCode owner-suppliedName FILE INFO OBJECT INFO DESCRIPTOR

O DRS1DRS2 e.g., billingCode ownerCode accessFlag tech metadata owner-suppliedName role purpose quality usageClass accessFlag tech metadata owner-suppliedName role processing quality usageClass billingCode ownerCode owner-suppliedName caption unit name view text FILE INFO OBJECT INFO DESCRIPTOR METS Object Label MODS PDS info, etc. Object Label Object-level MODS

Objects Owner supplied name is required Need to generate during migration Four cases – A METS file exists – New object will be built from a single content file – New object will be built from multiple content files – No OSN (potential case) Proposal for most cases: – add prefix or suffix to METS or content file owner supplied name

Objects Other required object elements – insertionDate date of earliest file? – captionBehavior for existing objects, set based on billing code prospectively, set by depositor – viewText available for all objects, not just PDS default to off

Objects Descriptive metadata – Take MODS from existing METS as is or import new From Aleph From Finding Aid – If re-imported, update METS label or not? – Import from OLIVIA based on owner supplied name for the file?

Objects from existing METS Identifiers for Harvard metadata – Identify finding aid identifiers – Convert “Old HOLLIS” numbers – Aleph IDs: include check digit or not? – Convert to URIs or actionable URNs from plain IDs Could DRS format such URIs for new DRS2 input?

Objects from existing METS PDS elements – PDF owner text becomes caption unit name – viewOcr function becomes viewText – goto function will be automatically determined by presence of structMap/div attributes Caption behavior – for existing objects, set by billing code

Files Run automated processes to identify, validate and characterize file technical characteristics Extract technical metadata

Files isFirstGenerationinDrs – Values: yes, no, unspecified – Should we supply “yes” for archival masters and/or top of derivation chain?

Image Files Converting from local scheme to MIX Local field questions – Methodology – History – Source – Enhancements

Text files Converting from local scheme to textMD Descriptor_type will be absorbed into different places in DRS2 Extracted metadata can supply markup_basis markup_language for specific schemas possibly other elements

Audio files Moving from local schema to AES : Audio object structures for preservation and restoration

Versioned metadata History will be tracked for key administrative elements: – Access flag – Admin flag (new) – Billing code – Owner code What values to assign for required creation date and agent for migrated content?

NEXT STEPS

Next steps Continue analysis and development of technical requirements Build prototype September check-in on progress Create metadata migration plan Open meeting to review plan

OPEN FOR QUESTIONS