Spring 2008MMDB - Introduction1 Introduction: Multimedia Databases

Spring 2008MMDB - Introduction2 Today’s Media Types Discrete media: Time is not part of the semantics of the media  Classical Data  Text/Documemt  Image Continuous media: Time or more exactly time-dependency between information items, is part of the information itself.  Video  Audio

Spring 2008MMDB - Introduction3 What is an MMDBMS? A multimedia database management system (MMDBMS) is a framework that manages different types of data potentially represented in a wide diversity of formats on a wide array of media sources.

Spring 2008MMDB - Introduction4 Abilities of MMDBMS Query data represented in different formats. Retrieve media objects from a storage device in a smooth jitter free manner. Take the answer of a query and develop a presentation of that answer in terms of audiovisual media. Deliver this presentation in a way that satisfies various quality of service requirements.

Spring 2008MMDB - Introduction5 Sample Multimedia Scenario Consider a police investigation of a large- scale drug operation. This investigation may generate the following types of data. Video data captured by surveillance cameras that record the activities taking place at various locations. Audio data captured by legally authorized telephone wiretaps. Image data consisting of still photographs taken by investigators.

Spring 2008MMDB - Introduction6 Document data seized by the police when raiding one or more places. Structure relational data containing background information, bank records, etc., of the suspects involved. Geographic information systems data containing geographic data relevant to the drug investigation being conducted. Sample Multimedia Scenario

Spring 2008MMDB - Introduction7 Data Sources Used Police application Surveillance video Audio data Document data Relational data Still image data Geographic information

Spring 2008MMDB - Introduction8 Characteristic of Multimedia Queries We normally retrieve a few records from a traditional DBMS through the specification of exact queries based on the notions of “equality”. The types of queries expected in a multimedia DBMS are relatively vague or fuzzy, and are based on the notion of “similarity”. The indexing structure should be able to satisfy similarity-based queries for a wide range of similarity measures.

Spring 2008MMDB - Introduction9 Image-based queries:  Police officer has a photograph in front of him.  He want to find the identity of the person in the picture.  Retrieve all images from the image library in which the person appearing in the photograph appears. To support this, we need to know what “similarity” means. We need to be able to efficiently support these operations. Example: Image Queries

Spring 2008MMDB - Introduction10 Example: Image Queries Keyword-based queries:  Police officer wants to examine picture of Bill Clinton.  Retrieve all images from the image library in which Bill Clinton appears. To support this, we need to know how to associate different attributes with images (or parts of images). We need to know how to effectively index and retrieve images based on such attributed.

Spring 2008MMDB - Introduction11 Query 1:  Police officer is listening to an audio surveillance tape.  Search through a voice log database to see if the identity of the person, whose voice is on the tape, can be ascertain. Query 2:  Police officer wants to review all audio logs that Bill Clinton participated in the conversations.  Find all the audio tapes in which Bill Clinton was a participant. Example: Audio Queries

Spring 2008MMDB - Introduction12 Text query:  Find all the news that deal with president scandals. Video query:  Find all video segments in which Bill Clinton appeared with Monica Lewinsky. Example: Text/Video Queries

Spring 2008MMDB - Introduction13 Need a single language within which multimedia data of different types can be accessed. The language must be able to merge, manipulate, and join together, results from different media sources. Once such language are devised, we need techniques to:  Optimize a single query  Develop servers that can optimize processing of a set of queries. MMDB Research Issues: Queries

Spring 2008MMDB - Introduction14 How should we extract the content of  an image  a video-clip  a audio-clip  a free/structured text document How should we index the results of this extracted content? What algorithms can be used to efficiently retrieve media data on the basis of similarity? MMDB Research Issues: Contents

Spring 2008MMDB - Introduction15 How is media data laid out on storage devices? How do we design disk/CD-ROM/DVD servers so as to optimally satisfy different clients concurrently, when these clients execute the following operations  Playback  Rewind  Fast forward  Pause MMDB Research Issues: Storage

Spring 2008MMDB - Introduction16 How can we deliver a multimedia presentation to users when there is  a need to interact with other remote servers to assemble the presentation?  a bound on the buffer, bandwidth, load, and other resources available on the system? How can such presentations optimize Quality of Service (QoS)? MMDB Research Issues: Presentations and Delivery