MPEG-7 Interoperability Use Case

Motivation MPEG-7: set of standardized tools for describing multimedia content at different abstraction levels Implemented in the form of XML Schemas Lack of explicit semantics: intended semantics in form of documentation Lack of precise semantics: ambiguities resulting from flexibility in structuring the descriptions

Motivation (cont’d) Attach formal semantics to mpeg7 by translating it to an ontology The lack of precise and explicit semantics leaves room for different interpretations  interoperability issues raised among the developed mpeg7 ontologies

Examples I Semantically identically metadata can be represented in multiple ways E.g. annotating an image depicting ‘Zidane scoring against England’ Possible ways: using free annotation, keywords, structured annotations etc.

Examples I (cont’d) Zinedine Zidane scoring against England Zinedine Zidan scoring England goal Zinedine Zidane Zinedine Zidane scoring against England.

Example II The image representation in itself as a multimedia entity can be performed in multiple ways as well Possible ways: using the Stillregion DS or even the VideoSegment DS: no formal semantics to prevent one from selecting to interpret an image as a still region or as a frame of a video sequence

Example III Semantically different metadata can be represented using the same description tool E.g., the Multimedia Description Schemes:VideoSegment DS can be used to represent a single frame in a video, an arbitrary sequence of frames, or the full video sequence Three different semantic concepts: frame, shot, video that can’t be discriminated based on the provided XML Schemas

Example IV Loose (intended) semantics of MPEG-7 e.g. with respect to the intended meaning of the structure decomposition schemes E.g., not part-whole semantics: having an image annotated as ‘Zidane scoring’ and a second image, having a segment annotated the same, a single query wouldn’t be adequate to retrieve both

Approaches to building an MPEG-7 ontology The Hunter approach Initially a DAML+OIL enriched RDFS ontology, subsequently translated into OWL Intended to be used under a core ontology (ABC) to enable harmonization with domain ontologies Decomposition captured using subclass constructs However no axioms included (e.g. an image consists of a set of spatial, and not temporal, segments)

Approaches to building an MPEG-7 ontology (cont’d) The SMART Web project approach The MultimediaContent and Segment classes along with corresponding properties implement the recursive nature of multimedia content decomposition Partial axioms

Approaches to building an MPEG-7 ontology (cont’d) Tsinaraki’s approach Translate the MPEG-7 MDS into OWL to serve as a core ontology for the integration of domain specific knowledge into the MPEG-7 semantic metadata Domain ontologies need to be ‘mapped’ to the MPEG-7 schemas

The Hunter’s and Tsinaraki’s approaches in practice Use Case: Annotate and image of Zidane scoring :image01 rdf:type mpeg7:Image :goal01 rdf:type s:Goal :scoring01 rdf:type s:Scoring :image01 mpeg7:depicts :goal01 :goal01 abc:hasAction :scoring01 :scoring01 abc:hasAgent s:_b1 s:_b1 :hasName :“Zinedine Zidane” mpeg7:Image rdfs:subclass mpeg7:MultimediaContent mpeg7:MultimediaContent rdfs:subclass abc:Manifestation s:Scoring rdfs:subclass abc:Action s:Goal rdfs:subclass abc:Event :region01 rdf:type mpeg7:StillRegion (Hunter’s approach)

(cont’d) :Zidane rdf:type mpeg7:PlayerObject mpeg7:PlayerObject owl:subclass mpeg7:AgentObjectType mpeg7: AgentObjectType rdfs:subclass mpeg7:SemanticType :Zgoal rdf:type s:Goal s:Goal rdf:type mpeg7:EventType mpeg7: EventType rdfs:subclass mpeg7:SemanticType :Zidane mpeg7:Relation :Zgoal (Tsinaraki’s approach)

Possible solutions Define semantic mappings between the various mpeg7 ontologies (exploit the potential of using rules on top of ontologies and reason on them) Built a core multimedia ontology (exploit perhaps a foundational ontology’s axiomatization to ensure not only explicit but precise semantics as well) Use of ‘semantically enabled’ profiles (again formal ways to specify the semantics constraints are required)

Thank you!