2. MPEG-7 Multimedia Content Description Interface Presented by: Moustafa A. Hammad

3. Introduction yMore and more digital audio - visual information exists and increasing. yIncreasing Internet popularity. yMore audio-visual information processing systems emerged. How fast and easy can desirable information be made available? MPEG-1 “Standard for storage and retrieval” MPEG-2 “The digital television standard” MPEG-4 “ Multimedia production, distribution and content access” - developed a Syntactic Description Language. “ Fine - where is the semantic?”

4. Introduction (Cont.) zMPEG-7 “ Multimedia content description interface”. zRepresents information about the contents, but not the content itself. zSatisfies both database and signal processing communities. zGoal: audio-visual material as searchable as text. zWhat is the standard? (To be finalized mid 2001)

5. Topics of Discussion zScope of the standard zTerminology zInteraction between MPEG-7 and Applications. zRequirements zApplications zCase study : A Proposal for MPEG-7 Description Definition.

6. Scope of the Standard zMPEG-7 processing chain: yfeature extraction (analysis) ythe description itself ythe search engines (application) zWhat is in the standard? Description production Description consumption Standard Description

7. MPEG-7 Terminology Data, Feature, Descriptor (D), Descriptor value, Description Scheme (DS), Description, Coded Description and Description Definition Language (DDL) defines Descriptor describes *..1 Description Scheme 1..* 0..* * Data AV Content Item 1..* Feature signifies 1..* To Human or System

8. Interaction between MPEG-7 and Applications Description Generation MM Content Decoder MPEG-7 Coded Description Encoder MPEG-7 Description Description Definition Language (DDL) Description Schemes (DS) Descriptor (D) Filter Agents Search/ Query Engine User or data processing system Human or System

9. MPEG-7 Requirements Descriptors:( Cross modularity, Direct data manipulation, Data adaptation, Language of text based descriptions, Linking, Prioritization of related information, Unique identification) Description Schemes:( Description Scheme relationships, Prioritization of descriptors, Hierarchy of descriptors, Scalability of descriptors, Description of temporal range, Data adaptation) DDL: ( Compositional capabilities, Unique identification, Primitive/composite data types, Multiple media types, Relationships between description and data, Grammar, Intellectual Property Management and Protection (IPMP), Real time support …..Etc.) Descriptors requirement: zGeneral: (Types of features [N-dimensional spatio-temporal structure, Objectives, subjective, Production, composition, Concepts], Referencing analogue data,…..Etc.)

10. MPEG-7 Requirements (Cont.) zFunctional: ( Retrieval effectiveness, similarity-base retrieval ….Etc.) zCoding: (Description efficient representation, Description extraction…Etc.) zVisual specific: (Types of features (color, texture, sketch…), Visual data formats…Etc.) zAudio specific: (Types of features (Frequency contour, Harmony…), Auditory data formats…Etc.) zText specific: (Text retrieval, consistency of text description tools) (Types of features (Frequency contour, Harmony…), Auditory data formats…Etc.) System requirement: (multiplexing, Temporal synchronization, File format, IPMP…Etc.) Ref: MPEG Requirement Group, “MPEG-7 Requirement”, Doc, ISO/MPEG N2859, MPEG Vancouver Meeting, July 1999.

11. MPEG-7 Applications zPull applications: yvideo retrieval: (storage and retrieval of video database. Sound effects library, historical speech database…Etc.) zPush applications: yvideo selection and filtering :( Personalized television services, information access facilities for people with special needs..Etc.) zSpecialized professional and control applications. (Remote sensing applications, Surveillance applications..Etc.)

12. A proposal for an MPEG-7 Description Definition language (DDL)  Reference: [ J. Hunter (DSTC)] zA schema is based on different schemas; Resource Description Framework (RDF) Schema, XML Document Type Descriptors (DTD), Document Content Description (DCD), A Schema for Object-Oriented XML (SOX). zSatisfies The DDL requirements. zConsists of classes, properties and relations between classes. zUses of Dublin Core (DC) attributes. (Name, Identifier, Version, Registration Authority, Language, Definition, Obligation, Datatype, Maximum Occurrence, Comment)

13. The Description Scheme MM Document AudioVideoSpeed Track Music Track SoundFX Track List of soundFX Scope MIDI tempo Phoneme List Object1 Object2Object3 Frame 1 Frame 120 Sequence1 Sequence2Sequence3 Scene1.1 Scene1.2Scene1.3 Shot 1.1.1 Shot 1.1.2Shot 1.1.3 DC.Title DC. Creator DC.Subject DC.Publisher DC.Description DC.Contributor DC.Date DC.Type DC.Format DC.Identifier DC.Source DC.Language DC.Relation.HasPart DC.Rights DC.Subject DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max DC.Description DC.Type DC.Format.Type DC.Identifier DC.Relation.HasPart DC.Description DC.Type DC.Identifier Text Script Transcript EditList KeyFrame Locale Cast Objects Text Script Transcript EditList KeyFrame Camera.Dist Camera.Angle Camera.Motion Lighting OpenTrans CloseTrans Text Image Timestamp colour Anno.Text Anno.Posn Text Position shape Trajectory Speed Colour Texture Volume Anno.Text Anno.Posn MPEG-7

14. Features of the proposed MPEG-7 DDL zNamespace Declarations <x xmlns:dc=“http://purl.org/metadata/dublin_core#”> CNN News zThe Class Type declarations and Class Hierarchies

15. Features ….. (Cont.) zProperty type declaration, zThe relationship type declaration =domain.start)&&range[n].end

16. Features ….. (Cont.) zOrder and Occurs ( Seq, Bag, Alt, Par) zData typing & user defined datatypes zAttribute Definitions: zSynchronization and temporal specification zSpatial specification yBoth rectangle and polygon representation, HTML syntax and semantics. Audio1 5sAudio2

17. Example: MPEG-7 description …………………. ……… …………………………………………………….. ……………..…………………………………….. ……….. ”A red car which has been severely damaged by the exposition.” ” ………...

18. Conclusion zThe proposed DDL provides most of the DDL requirement. zThere are some remarks: yLack of provision to push applications: (filtering and selection, real time support) yNo representation for subjective and concept features. ySimple representation and support for spatial features.