T.Sharon 1 Internet Resources Discovery (IRD) Video IR

2 T.Sharon Querying Video Movie - structure and characteristics Example methods Example query answer Problems, and Possible Queries

3 T.Sharon Movie - Characteristics Built from Frames - Each frame is an Image. Refresh Rate in Frames per Second (FPS). Needs around 10-15FPS for smooth motion. Clip Scene Shot Frame

4 T.Sharon Example Methods Finding and using transitions between shots: Cut, Fade, etc. Extracting few “key frames” from each shot Processing each key frame as image Much more complicated! Clip Scene Shot Frame

5 T.Sharon Example Query Answer  Thumbnails: Built from selected DC images (key frames).

6 T.Sharon Problems and Possible Queries Problems: –losing temporal and motion information: objects motion within a scene camera motion post processing effects (warping) Enables queries: –Find a video with a key-frame like a given image. –Rank the video clips in the video collection in order to find a similarity to a given video clip.

7 T.Sharon Advanced Topics Needed tools for VIR systems Classifying VIR systems

8 T.Sharon Needed Tools for Visual Query Formulation? Image processing Features manipulation Object specification Measuring Categorization spatial organization temporal organization Annotation Data definition

9 T.Sharon Highly automated systems use low level abstraction. Automation (human involvement). Multimedia characteristics: –several modalities –indexes and data structures for different modalities Adaptability: –static or dynamic characteristics Abstraction: Systems Classification Criteria (1) Characteristic (e.g., color, texture) objectsyntaxsemantics Low High

10 T.Sharon Generality: –restricted to one domain (e.g., medical imaging, remote sensing) or general. Content Collection: –information gathering way. Categorization: –to semantic anthologies. Compressed domain processing Systems Classification Criteria (2)