1. Robust video fingerprinting system Daniel Pereirad.pereira@skillupjapan.co.jp Luis Loyolaloyola@skillupjapan.co.jp

2. Confidential 22 Robust video fingerprinting system Summary Purpose of the system What is video fingerprinting Practical problems to solve Proposed solution Results analysis

3. Confidential 33 Robust video fingerprinting system Purpose of the system SkillUpJapan distributes digital contents FujiTV, TV Tokyo, SkyPerfecTV, Warner Brothers Japan, … Our platform, Uliza, is an extensible digital content management system Piracy and DRM are of importance to digital contents rights holders

4. Confidential 44 Robust video fingerprinting system Video fingerprinting A way to effectively tie a video, or a segment of it, to a unique hash value Information needs to be stored and searched efficiently Avoid to store original contents provided by clients Contents should not be recreated from said fingerprint

5. Confidential 55 Robust video fingerprinting system Key technical aspects about video Measured characteristics Luma and chroma (brightness and color components) Edge detection, gradient orientation Time variance A movie is, after all, a sequence of images that change over time at a defined rate Amount of data per frame

6. Confidential 66 Robust video fingerprinting system Efficiency metrics Uniqueness Accurately find videos we search; not return videos that are not what we search Database Efficiently index the results in a database Solution must be fast Find the clip among many other videos in fastest time

7. Confidential 77 Robust video fingerprinting system Some practical problems to solve Current solutions have relatively accurate algorithms, however the process is computationally expensive Partitioning of frames, complex algorithm Database storage is not taken seriously It is unaffordable to store information for every frame, or large arrays of information for each fingerprint Slow search times when database grows

8. Confidential 88 Robust video fingerprinting system Proposed solution Lowers the needed resources and process time, while improving upon results (Luma and time based indexes) Addresses algorithmic complexity by using simple methods (Euclidean distance of vectors and Tanimoto correlation) Stores information in an efficient way, allowing for quick retrievals, with use of Look-up Tables

9. Confidential 99 Robust video fingerprinting system Proposed solution (video properties): Average the Luma value of each frame Luma values show prolonged, relatively constant, values that can be indexed to an interval of time Luma Time Threshold Luma Time

10. Confidential 10 Robust video fingerprinting system Proposed solution (video properties): Average Luma calculated according to Luma Duration [start-time, end- time[ (secs) Average Luma calculated in T seconds [2, 8[102 [8,13[107 [13,14[198 [14,19[85 [590,600]48 … Time

11. Confidential 11 Robust video fingerprinting system Proposed solution (database): Using those indexes to store only segments we can save lots of space Each segment of several seconds has a value of 2 bytes Luma values range from 0 to 255 Look-up table for segments 2.0 Luma 120 10.3140 16.0 19.4 85 110 21.4120 Time Luma time 2.010.316.019.4 120 85 140 21.4

12. Confidential 12 Robust video fingerprinting system Proposed solution (database): 4 Luma 10 15160 19 32 125 13 …… Time 2 Luma 120 5140 9 12 85 110 …… Time 2 Luma 120 5140 9 12 85 110 …… Time Comparisons Fingerprints on databaseFingerprint A … Fingerprint 1 Fingerprint 3 Time

13. Confidential 13 Robust video fingerprinting system Proposed solution (algorithm): Tanimoto Tanimoto makes a correlation between C and the remaining elements outside C Euclidean vector distance A B C

14. Confidential 14 Robust video fingerprinting system Proposed solution (algorithm): Hierarchical approach 1.Look-up Table of segments 2.Compares the time indexes 3.& 4. Tanimoto Correlation and Vector Distance of Luma Look-up Tables discard perceptually different movies efficiently Comparison of time indexes also behaves efficiently The number of movies that are ultimately analyzed with Tanimoto Correlation and Euclidean Vector Distance is very low

15. Confidential 15 Robust video fingerprinting system Evaluation of algorithm: 220 movies were analyzed with each other Quality varies from FullHD to SD Duration ranges from 15 second commercials to full length movies Frame-rate of movies varies from 15fps to 30fps Comparison against C.G.O. (Centroids of Gradient Orientation) [1] Tests were conducted by searching scenes of 10 seconds Evaluation compares algorithm, database size and robustness of solutions [1] Sunil Lee and Chang D. Yoo, “Robust Video Fingerprinting for Content-Based Video Identification”, IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 7, pp983-988, July, 2008

16. Confidential 16 Robust video fingerprinting system Obtained results (database size): SUJCGO DB ThresholdSize (Kbytes) 1 4,944 107,900 2 3,390 3 2,611 5 1,783 10 1,020 15 678 20 512 25 412 30 348

17. Confidential 17 Robust video fingerprinting system Obtained results (run-time): SUJCGO DB ThresholdTime (Seconds) 1 74.16 92.41 2 31.56 3 21.13 5 10.77 10 8.84 15 8.66 20 10.1 25 12.42 30 15.11

18. Confidential 18 Robust video fingerprinting system Obtained results (robustness): SUJCGO DB threshold False Positives False Negatives False Positives False Negatives 180 2111 2120 390 5110 10113 1588 2096 251114 30918

19. Confidential 19 Robust video fingerprinting system Obtained results (distortion robustness) CIF = 352x288 pixels QCIF = 176x144 pixels SUJ P FN CGO P FN Resize to CIF 0.000413 0.001963 Resize to QCIF 0.000331 0.001632 Lossy compression 0.000124 0.00188 Frame-rate change to 15 fps 0.000579 0.001983 Frame-rate change to 5 fps 0.001074 0.001612 Rotation of 1 degree 0.000331 0.001715 Rotation of 2 degrees 0.000413 0.001736 Rotation of 3 degrees 0.000455 0.001653

20. Confidential 20 Robust video fingerprinting system Summary State of the art solutions need to better address practical issues The proposed algorithm can improve upon state of the art algorithms on storage and speed of analysis Evaluation shows that the proposed solution also provides higher robustness

21. Confidential 21 Robust video fingerprinting system Questions? Daniel Pereirad.pereira@skillupjapan.co.jp Luis Loyolaloyola@skillupjapan.co.jp

22. Confidential 22 Robust video fingerprinting system Daniel Pereirad.pereira@skillupjapan.co.jp Luis Loyolaloyola@skillupjapan.co.jp