Presentation on theme: "1 Digital Library Content Model Dagobert Soergel College of Information Studies University of Maryland Department of Library and Information Studies University."— Presentation transcript:
1 Digital Library Content Model Dagobert Soergel College of Information Studies University of Maryland Department of Library and Information Studies University at Buffalo
2 The Problem Digital libraries must 1.Store a wide variety of often complex information objects and display these objects on different platforms. This requires modeling information objects, their internal structure, and relationships among them. 2.Provide data that support discovery, interpretation, use, and management of information objects. This requires a good metadata model 3.Support annotation of information objects. Annotations turn out to be surprisingly diverse. An annotation my refer to only a part of an information object. This requires an elegant model that can deal with many cases.
3 Purpose of the talk To reexamine a number of basic notions regarding the content of a digital library (or, more generally, any information system) to achieve sound definitions Developed in the framework of the DELOS Digital Library Reference Model a framework for describing digital libraries, their content, users, and functions and, for each, their qualities and associated policies
4 Premisses Modeling the content domain is complex and much thinking is muddled Need to be able to handle both data and documents Any reference model needs to be abstract and must not commit to any particular standard or design decision rather, it must provide a framework for specifying the commitments of any particular DL (or information system)
5 Issues 0Scope of this talk and modeling constructs 1aContent in the overall context of a DL reference model 1b Modeling information objects 1cLevels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
6 Scope of this talk A reference model for a broadly conceived digital library will be able to model most any information system, thus will be useful very broadly. The focus on digital libraries is in the application, especially the type of collection, to which the model is applied.
7 Scope: level of abstraction The reference model should stay on an abstract level. It should not require specific standards but rather allow for plugging in any standard, such as RDA or DC. A DL should indicate to the users what standard it uses for things like time, place, type of relationship, type of resource The reference model should not require design choices but rather provide a framework for specifying design choices, such as selectivity of the collection. A DL will then indicate whether its collection is selective or fully inclusive
8 Modeling constructs The reference model should be based on an entity-relationship model (E-R model). Second-order logic: relationship instances are resources that can in turn be related to anything. Apply pragmatically for useful navigation and common-sense inferences; stay away from types of reasoning that run into problems with second order logic. Must add mechanisms for indicating the degree of precision or the degree of certainty of statements.
9 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
10 Content in the overall context of a DL reference model Resources Structured data Unstructured data, text Uses of data
11 Everything is a resource W3C definition A resource is anything that can be identified or named. Any resource is represented by a resource identifiern Resource includes external (non-digital) objects or events and digital object or event, wherever that digital object or event may reside or occur. Same as topic in topic maps In an E-R model, entity types, entity instances (entity values), relationship types, and relationship instances are all resources In RDA: Resource restricted to information object. Advantages of broader definition will become clear.
12 Structured data = statements Resource 1 Resource 2 SoftwareModule LegalEntity SoftwareModule Information object Event (Date1, Date2) Multi-way relationships, frames Statements are information objects, that is, they are resources that can in turn be related to anything Statement also called proposition or assertions (or fact)
13 More on structured data Data consist of statements about resources. Such statements can be conceived as relationship instances in which the resource in focus occupies one argument slot. A simple statement using a binary relationship or a multi-way relationship (a frame instance with slots filled) (objects in an object-oriented database) Drug treatment frame instance DrugTaxoteer treatsDiseaseCancer, estrogen-negative inPopulationGroupElderly hasSuccessRate55%
14 More on structured data Slot fillers are also known as data values. A data value makes sense only when it is seen in relation to one or more resources, for example as a slot filler in a frame. Examples The value 55% makes sense only in the right context, such as in the success slot of a drug treatment frame The value 185 cm makes sense only if we know it is the height of a person or the length of a pair of skis.
15 There are two ways to communicate such statements. 1.Structured data: One learns what one wants to know about the resource in focus immediately from a relationship instance. Hamlet Shakespeare The drug treatment frame on Taxoteer The actual data of interest are represented in a database
16 There are two ways to communicate such statements. 2.Unstructured data: One needs to extract what one wants to know from a text or image that is related to the resource in focus. Shakespeare schrieb den Hamlet im Jahre 1625 Hamlet wurde von Shakespeare verfasst Taxoteer ist effektiv in der Behandlung von Krebsen die keine Rezeptoren fuer Estrogen haben. In aelteren Personen liegt die Erfolgsrate bei 50%. The data of interest are stored in what is commonly known as document.
17 Functions of data Data about a resource may serve any of the following functions: learn about the resource and its various characteristics learn about the history and context of the resource learn how to use the resource manage the resource preserve the resource The sections about metadata (roughly: data about an information object) will specialize this list
18 Relationship as the basic modeling construct Important principle: Many concepts in a DL reference model are best modeled based on relationships rather than based on entities For example, annotation-hood resides not in an information object but in the relationship InformationObjectA InformatioObjectB InformationObject B InformationObjectA
19 Resource type examples Information objects Incl. documents, data streams, databases, queries and their results (virtual information objects, such as database reports, virtual collections) Actors that can search for, create, and manage resources Functions and services Software modules Policies Languages Ideas, concepts
20 Inheritance Many reference model constructs are specified at the level of resource. They inherit down to the different resource types, especially information objects For example, the following statement types are valid for Resource Resource Identifier Resource QualityParameter Resource Policy Therefore, they are also valid for InformationObject or Actor or Policy
21 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
22 Information objects 1 1.A formal relationship instance (such a row in a table or a structured data record) 2.A document (written or spoken text, image, sound) from which a human reader can learn about the resource in focus or about the relationships among several resources. Information extraction: document formal relationship instances. A collection of information objects is in turn an information object a table in a relational database = a collection of rows, each representing a relationship instance or a collection of relationship instances a collection of documents
23 Information objects 2 An information object may be a close representation of an external object or event, for example An image (photograph or painting) of a building. There may be many such images taken from different angles etc. A video recording of a soccer game. There may be several such video recordings, each capturing different scenes, or capturing the same scene from different angles, or following different players, etc. These are different information objects representing the same external event.
24 Real world objects, concepts, ideas To provide full access to the information objects it contains, a digital library must manage data about any kind of object (real world objects, concepts, ideas) in its subject domain. Why? 1.The DL may represent data in the form of a database 2.Users look for information objects that deal with or are digital representations of any kind of object. This idea underlies Topic Maps which were originally designed to improve access to documents by relating the topics discussed in these documents.
25 Real world objects, concepts, ideas Examples (these are all resources) People (focus of biographical reference tools) Organizations (focus of organization directories) Events (focus of developing "event gazetteers") Places (focus of gazetteers) Dates Mathematical theorems (focus of mathematical encyclopedias) Concepts, ideas Problems and proposed solutions Computer programs (focus of software directories or libraries) The reference model should have a more complete list and indicate sources dealing with these
26 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1cLevels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
27 Levels, versions, and relationships Work, manifestation, item (individual copy) Linked through relationships
28 Work Intellectual or artistic entity, as the abstract essence or as a text, image, or piece of music. Range: A basic story or theme the story of Faust the myth of the Great Flood A text telling the story, such as Goethe's Faust the account of the Great Flood in the Bible (original Hebrew) the account of the same myth in another culture A specific version of the account in the Hebrew Bible a Latin translation of the account in the Hebrew Bible
29 Manifestation A specific rendering of a work by means of a graphical image or sound, taken in the abstract; the idea of such a rendering. Examples: The text of Goethe's Faust printed in a particular typeface and layout A performance at which the text is recited also renders the text but is more properly considered a separate, but related, work. A specific score of a given version of Schubert's Fifth. A performance of that version of Schuberts Fifth also renders the piece of music but is considered a separate, but related, work. Also the rendering of a work in the form of digital storage that can be transformed to a graphical image or sound, again taken as the abstract pattern of digital signals.
30 Item, individual copy The embodiment of a manifestation in a physical object We can perceive the content of an manifestation only through an individual copy of it (unless we have memorized the visual expression manifest in a manifestation and can conjure it up from memory). There are works that have only one manifestation of which there is only one copy.
31 Relationships among information objects The story of Faust Pact with the devil The story of Faust Marlows Faust The story of Faust Goethes Faust Goethes Faust Goethe, Johann Wolfgang von Goethes Faust R1231 R1231 Cotta R1231 1871 R1232 R1231 R1232 (HRieth, 1896, 1956) R1232 (DSoergel, 1956, *)
32 Hierarchical inheritance Data about a work inherit to all works below it along, etc. Therefore Goethe' Faust Pact with the devil Data about a work inherit to all its manifestations. Therefore R1231 Goethe, Johann Wolfgang von Data about a manifestation inherit to all its items Hierarchical inheritance increases efficiency More efficient catalog input More efficient catalog storage More efficient representation and reading of search results
More relationships R271 The man I killed, by Michael Halliday R519 The man I killed, play by Christopher Wern R519 R271 R315 Handbook of commercial geography, by Robert Chisholm R783 Chisholm's handbook of commercial geography, entirely rewritten by L. Dudley Stamp and S. Carter Gilmour. R783 R315 33
34 Relationship to FRBR Notes on Terminology The FRBR distinction between work and expression should be rethought. It is unclear and consequently poorly understood, and it may not be necessary. Just have work. The intuition FRBR tries to capture in this distinction is better handled through relationships among works as defined here. Following FRBR I use the term manifestation. Other term: edition (in the sense of German Ausgabe), but edition also means German Auflage, so use of the term edition can be confusing. It would be nice to be able to use graphic expression as a synonym for rendering, but to avoid any further confusion with FRBR it is best not to use the term expression at all.
35 Version control Important, but not elaborated here
36 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
37 Composite information objects / resources Examples Book divided into chapters, sections, paragraphs, words (XML Document Object Model, DOM or TEI) Each part can be seen as a separate information object Movie with images, soundtrack, close captions, script, all coordinated (MPEG-7) A medical record with patient data, test data, images, live monitoring data streams, diagnoses, drugs prescribed, etc.
38 Composite information objects / resources Abstractly: Each component is a separate information object, composition expressed through relationships In practice: Many document models for composite (or compound) documents supporting presentation DL needs to allow specification, for each document, of the particular document model used
39 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
40 Identifying information objects 1Initial definition upon entry into the digital library. 2Definition on the spot Examples Annotate a specific segment of a text document or a region of an image or sound document or Anchor an annotation to a specific location in a document. The segment or anchor is a new information object that is included in the original information object, and this new information object is linked with any of several annotation relationships to a new information object created by the user. Related to composite objects. More on this under annotation
41 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
42 Data about information objects Metadata = data about information objects if used for discovering, interpreting, and using information objects Relate information objects to other types of resources. Examples: InformationObject Actor InformationObject Text (or, more specifically Word) Relate a word in a text to the concept that is the meaning in which the word is used in this particular position. InformationObjectA InformationObjectB InformationObjectA InformationObjectC InformationObjectD InformationObjectC
43 More on defining metadata The metadata-hood of an information object does not reside in the information object, but in its relationship to another information object and, more specifically, in its use A piece of data is used as metadata if it is used for the purpose of discovering, interpreting, and using information objects, which then give the ultimate data wanted. The same piece of data may fill the ultimate need to of the user in one situation and be used as metadata in another situation.
44 Not metadata Data about resources that are not information objects are not metadata even if they are similar in form. Data about information objects are not always used as metadata. For example, using author data to count a faculty members publications or citation data to compute impact Extensive discussion of what exactly is the definition of metadata is not a good use of resources. A system should provide the data that are useful to a user for whatever purpose; what each piece of data is called is less important.
45 Metadata typologies Metadata (and data in general) can be divided into categories from several perspectives, and within each perspective there exist several approaches. Some examples of how to categorize metadata by purposes or use. Since the same unit of metadata can be used for several purposes, the resulting categories overlap. by source, for example, extracted, assigned by cataloger, assigned by user (social tagging), from usage tracking by intrinsic characteristics, for example data about provenance or about the format of the information object
46 Some metadata uses ALearn about information objects and interpret them; this includes A1Learn about the identity and characteristics of information objects (descriptive metadata) A2Learn about the history and other features of the context of the information object (contextual metadata) BLearn how to use an information object, including B1Learn how to gain legal access (access and rights metadata) B2Learn how to gain technical access to the information object (what machinery and software is needed to access the information object for a given purpose, such as assimilation by a person or processing by a computer program) CManage information objects (administrative metadata), in particular C1Manage the preservation of information objects (preservation metadata).
47 Usage data Data on usage of resources and on usage rights, usage history, future use / preservation important for discovering, interpreting, and using resources as well as managing resources Some of these data can be collected automatically If the resource in question is an information object, this kind of data is often used as metadata
48 Issues 1aContent in the overall context of a DL reference model 1bModeling information objects 1c Levels, versions, and relationships 1dComposite information objects / resources 1eResource identifiers 2Metadata, including provenance, context, usage 3Annotation
49 Annotation InformationObjectA InformationObjectB InformationObjectB may be created on the spot in order to annotate A (InformationObjectB and the annotation relationship have the same author) or B may preexist (the annotation relationship between A and B is introduced by a third party) Specific type of annotation expressed by specializing the annotatedBy relationship, for example InformationObjectA InformationObjectB InformationObjectA InformationObjectC InformationObjectD InformationObjectC InformationObjectE PartOfSpeech Annotation-hood is in the relationship, not in the information object
50 Annotation Annotation-hood is in the relationship, not in the information object There is a wide range of relationship types that are called annotations. Linguists think of annotations differently than scholars making comments on a text. Rather than trying to define exactly what annotation means, the reference model should include a comprehensive list of relationship types that might be considered annotation by somebody so that anybody can define their meaning of annotation by giving the appropriate subset of annotation relationship types. The same thought applies to metadata, discussed on a later slide.
51 Special resource types for annotations Some annotations require special types of resources. Examples Annotate a text with part-of-speech indications annotated resource :a one-word fragment of the text annotating resource:a value from a list of parts of speech Annotate a text with meaning for word sense disambiguation annotated resource :a word or phrase in the text annotating resource:a value from a list of meanings defined in some way Annotation through underlining or other marks annotated resource :a fragment of text or other information object annotating resource:a pair (sign, meaning), e.g. (underline, important) or (?, check this out) or (X, nonsense) The annotated resource and the annotating resource may be very short
52 Annotation and metadata Metadata and annotation data overlap, and different communities and individuals have different definitions of what is included in metadata and what is included in annotations. The precise nature of a unit of data about an information object is determined by the relationship type and the resource that is linked to. The interpretation of each type of data is in the eye of the beholder. Need an inventory of relationship types (a type of ontology) For example, the CIDOC Content Reference Model (CIDOC/CRM) is an inventory of broad relationship types. In such an inventory, one could indicate who considers a given relationship type as usable as metadata and/or as belonging to annotation.
53 Take-home message 1 The entity-relationship model (E-R model) provides the unifying principle for a digital library content model The E-R model allows representation of structured data of any complexity on a conceptual level. Defining relationships between information objects handles Modeling information objects Levels, versions, and relationships Composite information objects / resources Metadata Annotation Many notions are captured better through relationships than fine distinctions of entity types
54 Take-home message 2 Any reference model needs to be abstract and must not commit to any particular standard or design decision rather, it must provide a framework for specifying the commitments of any particular DL (or information system) A reference model provides a systematic framework for description and analysis, not a prescription
Dagobert Soergel dsoergel at umd.edu www.dsoergel.com 55
57 Construction process Need to be sure all applicable concepts from various sources such as the 5S model and FRBR/CRM are included, either in the skeleton model or in a list of values / choices, as appropriate There is still work to be done to pull reference model subject matter out of the reference architecture document, and vice versa.
58 Construction process We should have an online version of the reference model document with the following properties Links to discussion of issues and underlying rationale, capturing some of the discussion in the group. Links from the reference model to the appropriate section of the reference architecture The Wiki page may not quite do it.
59 There are two ways to communicate such statements. 1.One learns what one wants to know about the resource in focus immediately from a relationship instance. Hamlet Shakespeare The drug treatment frame on Taxoteer The actual data of interest are represented in a database that captures these statements (relationship instances), such as a collection of Prolog statements a relational database an object-oriented database 2.One needs to consult an information object that is related to the resource in focus. Shakespeare schrieb den Hamlet im Jahre 1625 Hamlet wurde von Shakespeare verfasst Taxoteer ist effektiv in der Behandlung von Krebsen die keine Rezeptoren fuer Estrogen haben. In aelteren Personen liegt die Erfolgsrate bei 50%
The DL designer must decide how to identify the new resource that is a part of an existing resource and the new text object created by the annotator and how to store the link between these two information objects 60
61 Identifying information objects Architecture issues Definition on the spot, options (1)use completely independent identifiers and store the relationship explicitly (2)use dependent identifiers The part of a document can be identified by document identifier followed by information that uniquely identifies the part. The part relation is implied by the structure of the identifier. The annotation information object could be identified by the identifier of the resource being annotated followed by a short string that identifies the nth annotation of this resource (like a footnote). The relationship between the resource and the resource annotating it would be implied by the identifier (however, the specific type of the annotation relationship would not be captured this way). The resource that annotates still can be referenced from any other context. Implicit representation Embedded annotations: The annotation is embedded in the document, linked to a point in a text that is identified only by the place of the annotation. This could be converted to an explicit representation.
62 Some metadata uses This is a specialization of the functions of data given above Alearn about other data, that is, information objects, and understand them; this includes A1learn about the identity and characteristics of information objects (descriptive metadata) A2learn about the history and other features of the context of the information object (contextual metadata) Blearn how to use an information object (source of data), including B1learn how to gain legal access to the information object (access and use rights metadata) B2learn how to gain technical access to the information object (what machinery and software is needed to access the information object for a given purpose, such as assimilation by a person or processing by a computer program) Cmanage information objects (administrative metadata), in particular C1manage the preservation of information objects (preservation metadata).
63 Metadata in the reference model When describing a DL using the reference model, need to be able to indicate any typology of metadata used in the DL