1. Video/Image Fingerprinting & Search Naren Chittar CS 223-B project, Winter 2008

2. Problem Definition Given an image, find copies on the web. Given a video clip, find movie that it belongs to. Two problems: –Feature Extraction –Efficient Search

3. Feature Extraction 0 0 0 0 0 0 35 12 21 14 0 0 0 0 0 13 26 20 20 25 0 0 0 0 0 21 19 20 34 25 0 0 0 0 0 31 16 29 51 21 0 0 0 0 1 36 15 29 18 22 0 0 0 0 7 33 19 39 24 22 Samples Gradient magnitude Partition into 10x10 blocks Normalized sum of gradients

4. Search: Inverted Index 0 0 0 0 0 0 35 12 21 14 0 0 0 0 0 13 26 20 20 25 0 0 0 0 0 21 19 20 34 25 0 0 0 0 0 31 16 29 51 21 0 0 0 0 1 36 15 29 18 22 0 0 0 0 7 33 19 39 24 22 Sliding and overlapping 3x3 window [0 0 0 0 0 0 0 0 0] … [0 0 35 0 13 26 0 21 19] … [29 51 21 29 18 22 39 24 22] 9 dim feature vectors DB (spatial data structure k-d tree)‏ [0 0...] -->doc1.jpg, doc3.jpg, doc10.jpg [0 2...] -->doc2.jpg doc10.jpg

5. Search: (continued)‏ query [0 0 0 0 0 0 0 0 0] … [0 0 35 0 13 26 0 21 19] … [29 51 21 29 18 22 39 24 22] Feature extraction Features hits Hits 70 – doc1234.jpg 20 – doc2345.jpg 5 – doc2222.jpg 1 – doc2356.jpg.....

6. Results Database : 10,000 images. From Flickr Test set : 30 images.

7. Conclusions and Future Work Technique robust to scaling, resolution loss. Works only for small amount of cropping. Implement kd-tree for faster search. Enlarge database to 1 million images and include videos. Augment feature vector with color attributes and other information. Compute relevance score based on position of feature vector in image.