Multimedia Content Description Interface Kunal Kishore A Presentation By :- MPEG-7 Cochin University of Science And Technology

Contant Introduction Need of MPEG-7 History of MPEG Elements Of MPEG-7 Parts of MPEG-7 Application Conclusion MPEG-7 Cochin University of Science And Technology

Introduction In September 2001, Moving Picture Experts Group (MPEG) specified an international standard called Multimedia Content Description Interface, or MPEG-7. MPEG-7 is a standardized description, based on XML Schema, of various types of AV content. Material (e.g., still pictures, graphics, audio, video) with associated MPEG-7 data can be efficiently indexed and searched for. MPEG-7 Cochin University of Science And Technology

+ Why do we need MPEG-7 ? Support for Advanced Query Visual Audio Fast And accurate acess Personalized Content Production and Consumption Content Management’ Automation + Visual Audio Sketch Support for Advanced Query MPEG-7 Cochin University of Science And Technology

MPEG- History…1 MPEG: Moving Picture Experts Group ISO / IEC/JTC1/SC29/WG11 Established in 1988 A Working Group of ISO/IEC in charge of the Development of Standards for Coded epresentation of Digital Audio and Video MPEG-7 Cochin University of Science And Technology

MPEG- History…2 MPEG-1: Interactive CD and MP3 11 / 1992 MPEG-2: DTV, STB, DVD 11 / 1994 MPEG-4: Web and Mobility ver1: 09 /1998, ver2: 11 /1999 MPEG-7: ??? 08 / 2001 MPEG-7 Cochin University of Science And Technology

Scope of MPEG-7 The description generation (feature extraction, indexing) is not a normative part of the standard The description consumption (search engine, retrieval process) is also not a normative part of the standard. Just the description is normative Standardizes the minimum needed to ensure interoperability MPEG-7 Cochin University of Science And Technology

MPEG: Main Elements…1 MPEG-7 Cochin University of Science And Technology

MPEG: Main Elements…2 Descriptors (Ds): representations of features To describe various types of features of multimedia information To define the syntax/semantics of each feature representation Color example – specify histogram in RGB color space Description Schemes (DSs) Specifies the structure and semantics of the relationships between its components, which may be either Ds or other DSs. MPEG-7 Cochin University of Science And Technology

MPEG: Main Elements…3 Description Definition Language (DDL) The language to specify DSs and possibly Ds Will allow the creation of new DSs (and possibly Ds) and the extension modification of existing DSs . System tools To support multiplexing of description, synchronization issues, transmission mechanisms, file format, etc. MPEG-7 Cochin University of Science And Technology

Parts of MPEG-7 Standered Systems Description Definition Language Visual Audio Multimedia Description Schemes Reference software Conformance Testing Extraction and use of descriptions MPEG-7 Cochin University of Science And Technology

MPEG-7: Systems It defines tools to: Provide for efficient storage and transport Synchronize between content and description Manage and protect intellectual property MPEG-7 Cochin University of Science And Technology

Basic Structure of Visual Descriptors Grid Layout Time Series 2D-3D Multiple View Spatial 2D Coordinates Temporal Interpolation MPEG-7 Cochin University of Science And Technology

MPEG-7 Visual Descriptors Color Texture Shape Motion Localization Face Recognition MPEG-7 Cochin University of Science And Technology

MPEG-7 : Audio Descriptors Sound Effects Music Instrument Timbre Spoken Content Melody Contour MPEG-7 Cochin University of Science And Technology

Low level Audio Visual descriptors Video segments Still regions Color Camera motion Motion activity Mosaic Color Shape Position Texture Moving regions Audio segments Color Motion trajectory Parametric motion Spatio-temporal shape Spoken content Spectral characterization Music: timbre, melody MPEG-7 Cochin University of Science And Technology

Multimedia Description Schemes- MDS MPEG-7 Multimedia Description Schemes (DSs) are metadata structures for describing and annotating audio-visual (AV) content. The DSs provide a standardized way of describing in XML the important concepts related to AV content description and content management in order to facilitate searching, indexing, filtering, and access. The DSs are defined using the MPEG-7 Description Definition Language (DDL), which is based on the XML Schema Language, and are instantiated as documents or streams. MPEG-7 Cochin University of Science And Technology

Organization of MDS tools MPEG-7 Cochin University of Science And Technology

Example of Segment Tree SR1: · Creation, Usage meta information Media description Textual annotation Color histogram, Texture SR2: · Shape Color Histogram Textual annotation SR6: SR5: · Shape Textual annotation SR4: Color Histogram SR3: Background Foreground MPEG-7 Cochin University of Science And Technology

DDL: XML & format XML was created to structure, store, and transport information. • It is just plain text. Software that can handle plain text can also handle XML. • It is just pure information wrapped in tags. Someone must write a piece of software to send, receive or display it. • XML-aware applications can handle the XML tags specially. The functional meaning of the tags depends on the nature of the application. MPEG-7 Cochin University of Science And Technology

Example MPEG-7 Cochin University of Science And Technology

Binary Format- BiM BiM is designed for transmission and storage BiM encoders and decoders can deal with any XML language. BiM is a schema oriented encoding scheme The binary format is deduced from the schema definition There is no need to define coding tables or a specific encoding mechanism MPEG-7 Cochin University of Science And Technology

Application Scenario MPEG-7 Cochin University of Science And Technology

Additional Application Mobile phone Personal video recording (PVR) systems Content delivery Surveillance, remote sensing, visual control equipment Database, server Storage and retrieval of audiovisual databases (image, film, radio archives) MPEG-7 Cochin University of Science And Technology

Some Industrial Tools IBM MPEG-7 Annotation Tool Ricoh MovieTool Canon MPEG-7 Speech Recognition Engine MPEG-7 Cochin University of Science And Technology

MPEG-7 Timeline Work plan Call for proposals Oct 1996 2001 2000 1999 1998 1996 Working draft Mar 2000 Committee draft Oct 2000 Draft international Standard July 2001 Final committee Draft Jan 2001 International Oct 2001 Divergence Convergence MPEG-7 Cochin University of Science And Technology

Conclusion MPEG-7: AV content description for interoperable application Description Definition Language: XML Schema (flexibility) + Binary version (efficiency) Description Schemes: Library of description tools Covers a wide range of generic needs MPEG-7 Cochin University of Science And Technology

Refrences Multimedia System Standered & n/w :Puri MPEG-7 Awareness event(http://www.mpeg-industry.com) http://www.mpeg.chiariglion.org (A MPEG Home Page ) http://www.mpeg-alliance.com http://www.w3.org/1999/05/06-xmlschema-1 http://www.dlib.org/dlib/september99/hunter/09hunter.html MPEG-7 Cochin University of Science And Technology

Questions ? MPEG-7 Cochin University of Science And Technology

MPEG-7 Cochin University of Science And Technology