1. November 22, 2003DASER Conference. Copyright MIT, 20031 METS: Metadata Encoding & Transmission Standard

2. November 22, 2003DASER Conference. Copyright MIT, 20032 Part One: Problem definition

3. November 22, 2003DASER Conference. Copyright MIT, 20033 Digital (Library) Objects Reformatted to digital scanned photographs, books and journals digitized audio/video files Born digital TEI-encoded texts digital images, audio, video files GIS, statistical datasets interactive content

4. November 22, 2003DASER Conference. Copyright MIT, 20034 Digital (Library) Objects Simple Objects –single files, e.g. visual TIFF images MP3 files TEI-encoded text –objects stand alone no relationships to other objects

5. November 22, 2003DASER Conference. Copyright MIT, 20035 Digital (Library) Objects Complex Objects –multiple related files, e.g. –page images from books or articles –multiple channels in digital audio files –related sound and text files (multimedia) –statistical dataset and codebook –objects cannot stand alone multiple files required to interpret the object requires structural metadata to model

6. November 22, 2003DASER Conference. Copyright MIT, 20036 Structural metadata Maps physical files (digital assets) to logical items (complex digital objects) Examples –Scanned print material complex publication structures (e.g. journals runs) ordered relationship between digital page images –A/V material multiple resolutions of an image multiple channels of an audio file

7. November 22, 2003DASER Conference. Copyright MIT, 20037 Structural metadata Examples, continued –Multimedia presentations relationship between images, text, sound, video, etc. (time-based or other) –Web sites linkages between web pages sitemaps –Databases table models and ER diagrams

8. November 22, 2003DASER Conference. Copyright MIT, 20038 Digital (Library) Objects Also have other (non-structural) metadata –descriptive MARC, DC, FGDC, VRA core, other ontologies –administrative rights, provenance –technical format details, OAIS representation information Standards exist or emerging for these

9. November 22, 2003DASER Conference. Copyright MIT, 20039 Part Two: Introduction to METS

10. November 22, 2003DASER Conference. Copyright MIT, 200310 METS Scope Supports –Structural metadata complex reformatted or born digital objects –Metadata wrapper framework descriptive, administrative, structural, etc. structural required others use namespaces to reference extension schemas

11. November 22, 2003DASER Conference. Copyright MIT, 200311 Brief History 1997-2001 Making Of America II project –Funded by DLF and NEH –Included Berkeley, Cornell, NYPL, Penn State, Stanford, U of Michigan –Designed for scanned archival collections –SGML DTD included pre-defined descriptive, administrative, structural metadata February 2001 DLF workshop on structural metadata produced METS framework

12. November 22, 2003DASER Conference. Copyright MIT, 200312 METSHeader Administrative metadata File Inventory Structure map Descriptive metadata Behavioral metadata METS metadata buckets optional required optional

13. November 22, 2003DASER Conference. Copyright MIT, 200313 METS metadata XML extension schemas –descriptive metadata Dublin Core, MARC, FGDC, VRA, etc. Berkeleys GDM schema (from MOA2) –administrative/technical metadata NISO image technical metadata LC schemas for A/V technical metadata Rights metadata (e.g. PRISM, XrML, etc.) Provenance metadata

14. November 22, 2003DASER Conference. Copyright MIT, 200314 Metadata Reference (mdRef): A link to external descriptive metadata. The type of link (URN/Handle/etc.) is included as an attribute, as is the metadata type. Metadata Wrapper (mdWrap): Included descriptive metadata, as either binary data (Base64 encoded) or arbitrary XML using namespace mechanism. The metadata type is specified as an attribute. METS Descriptive Metadata Section

15. November 22, 2003DASER Conference. Copyright MIT, 200315 Technical Metadata (techMD): technical metadata regarding content files IP Rights Metadata (rightsMD): rights metadata regarding content files or primary source material Source Metadata (sourceMD): provenance information for content files. Preservation Metadata (preservationMD): metadata to assist in preservation of digital content All sections use generic metadata reference and wrapper subelements. METS Administrative Metadata Section

16. November 22, 2003DASER Conference. Copyright MIT, 200316 File Group (fileGrp): provides mechanism for hierarchically subdividing physical files, for example by type File (file): provides a pointer to an external file (Flocat) or includes file content internally (Fcontent) in Base64 encoding METS File Inventory

17. November 22, 2003DASER Conference. Copyright MIT, 200317 The Structural Map provides a tree structure describing the original document. Each division (div) element is a node in that tree, and can identify content files associated with that division by a METS Pointer (mptr) or a File Pointer (fptr) METS Structural Map

18. November 22, 2003DASER Conference. Copyright MIT, 200318 METS Pointer and File Pointer METS Pointer (mptr): xlink to another METS file containing the content for the associated div. Useful for breaking up large objects (e.g., a journal run) into a series of smaller METS documents. File Pointer (fptr): Identifies one or more entries in the File Inventory section containing the content for the associated div element. Can also limit the link from a div element to a portion of a content file (e.g., a segment of an audio or video file, a subarea of an image or video file, etc.).

19. November 22, 2003DASER Conference. Copyright MIT, 200319 File Pointer (fptr): Can identify a single file in File Inventory using ID/IDREF linking Parallel/Sequential(par/seq): Allows a div to be associated with several content files that should be played/displayed in parallel (video with separate audio track file) or sequentially. Area (area): identifiers a point, linear segment, or 2D area within content file that corresponds with associated div element. METS File Pointer Mechanisms

20. November 22, 2003DASER Conference. Copyright MIT, 200320 METS Area Element Attribtes FILE:ID for File element in File Inventory SHAPE:As in HTML Area element COORDS:As in HTML Area element BEGIN:A start point within a file for defining a segment END:An end point within a file for defining a segment BETYPE:Begin/End type: IDREF, Byte Offset, or SMPTE time code EXTENT:Length Duration of Segment EXTYPE:Extent Type: Bytes, or SMPTE

21. November 22, 2003DASER Conference. Copyright MIT, 200321 Structure Example urn:x-nyu:violet42 <area FILE=f1 BEGIN=00:23:17:00 END=00:23:38:00 BETYPE=SMPTE>

22. November 22, 2003DASER Conference. Copyright MIT, 200322 Created for multimedia structural encoding SMIL has time-based orientation –for playing multimedia presentations Very complex May eventually be incorporated Related standards: SMIL (W3C), MPEG-7 (ISO)

23. November 22, 2003DASER Conference. Copyright MIT, 200323 Related standards: RDF (W3C) Also metadata wrapper framework Structural metadata could be supported, but doesnt specify how… Opaque to use No element semantics provided element names deliberately meaningless Originally designed for descriptive metadata

24. November 22, 2003DASER Conference. Copyright MIT, 200324 Related standards: OAIS framework

25. November 22, 2003DASER Conference. Copyright MIT, 200325 METS and OAIS framework Submission Information Package (SIP) METS as transfer syntax Dissemination Information Package (DIP) METS as tranfer syntax METS as input to display applications Archival Information Package (AIP) METS stored internally in an archive

26. November 22, 2003DASER Conference. Copyright MIT, 200326 Library Applications Digital Object transfer syntax –between systems enables interoperability –between institutions enables collection sharing –implements OAIS SIP/DIP/AIP

27. November 22, 2003DASER Conference. Copyright MIT, 200327 Library Applications Input to Digital Object delivery systems (aka disseminators) –Simple bit-streaming –XSL stylesheet –Custom program for complex digital object display

28. November 22, 2003DASER Conference. Copyright MIT, 200328 Part Three: METS Summary

29. November 22, 2003DASER Conference. Copyright MIT, 200329 METS summary Descriptive/technical/administrative metadata –not defined internally –points to external standard schemas Dublin Core, MARC, MPEG-7, etc. AES audio metadata –set of best practice schemas being identified

30. November 22, 2003DASER Conference. Copyright MIT, 200330 METS summary Structural metadata –defined internally and required –SMIL-lite simple support for multimedia, audio/visual SMIL may replace eventually

31. November 22, 2003DASER Conference. Copyright MIT, 200331 METS summary Current users include UC Berkeley (archival collections) Harvard (scanneded print publications, e- journals) Library of Congress (audio/visual collections) British Library RLG and OCLC EU METAe project (historic newspapers) Michigan State (oral history collections) Univ of Virginia (FEDORA digital objects) more daily...

32. November 22, 2003DASER Conference. Copyright MIT, 200332 METS summary Tools under development for –metadata capture –transformation –transfer –dissemination/display Profiles necessary for interoperation –Which extension schemas used? –How structure maps are organized…

33. November 22, 2003DASER Conference. Copyright MIT, 200333 METS summary Current status –version 1.3 available from LC –editorial board in place –LC standards office for maintenance agency –DLF and RLG underwriting RLG will host editorial board, offer documentation and training, develop tools –Several extension schemas available –Opening Day in October 2004

34. November 22, 2003DASER Conference. Copyright MIT, 200334 METS summary METS is not all things to all people… –Designed for local institutional application support Solving an immediate local problem Common to many institutions Flexible framework supports many institutional situations –Profiling necessary to interoperate For OAIS packages For shared tools For other kinds of interoperation (e.g. cross repository search)