1. Automated Shot Boundary Detection in VIRS DJ Park Computer Science Department The University of Iowa

2. Introduction VIRS? VIRS? Visual Information Retrieval System Visual Information Retrieval System Works on multimedia information domain such as video, sound, etc Works on multimedia information domain such as video, sound, etc Aims for content-based analysis, retrieval and presentation Aims for content-based analysis, retrieval and presentation Find all videos that Iowa football team plays Find all videos that Iowa football team plays

3. Building VIRS Shot Boundary Detection Shot Boundary Detection Scene Boundary Detection Scene Boundary Detection Key Frame Generation Key Frame Generation Pattern Recognition Pattern Recognition Indexing with Meta Data Indexing with Meta Data Interface and Querying Method Interface and Querying Method

4. Shot Boundary Detection A sequence of frames captured by a single camera in a single continuous action A sequence of frames captured by a single camera in a single continuous action Most basic unit of video data Most basic unit of video data A scene is a logical group of shots into a semantic unit A scene is a logical group of shots into a semantic unit Many different styles and techniques ( cut, fade, wipe, dissolve ) Many different styles and techniques ( cut, fade, wipe, dissolve ) How to detect among these styles? How to detect among these styles? What about camera/object movements? What about camera/object movements?

5. General Approach Frame to Frame Comparison Frame to Frame Comparison Take 2 subsequent frames and measure their Take 2 subsequent frames and measure their differences How to define these differences discriminates each method How to define these differences discriminates each method Pixel Difference Pixel Difference Color Histogram Difference Color Histogram Difference Edge Difference Edge Difference

6. Pixel Difference One of the earliest methods One of the earliest methods How many pixels are different between subsequent frames? How many pixels are different between subsequent frames? Problems? Problems? Improvement – using blocks instead of pixels Improvement – using blocks instead of pixels (X,Y) Frame t Frame t+1

7. Color Histogram Difference Much more accurate than simple pixel-wise approach Much more accurate than simple pixel-wise approach Compare the number of occurrences for each color between subsequent frames Compare the number of occurrences for each color between subsequent frames Problems? Problems? Frame tHistogram t

8. Edge Difference No color information is needed No color information is needed Compare entering edge pixels and exiting edge pixels Compare entering edge pixels and exiting edge pixels Better than other methods in detecting fade, wipe, dissolve Better than other methods in detecting fade, wipe, dissolve Problems? Problems?

9. Future Works Evaluation and Combination of existing methods Evaluation and Combination of existing methods Relating shot boundary information with scene boundary – how to group shots into semantic unit? Relating shot boundary information with scene boundary – how to group shots into semantic unit? Using sound information for shot ( or scene ) boundary detection? Using sound information for shot ( or scene ) boundary detection? Using object-oriented approach or 3-D image modeling technique? Using object-oriented approach or 3-D image modeling technique?

10. Questions?