1. DIEGO AGUIRRE COMPUTER VISION INTRODUCTION 1

2. QUESTION What is Computer Vision? 2

3. COMPUTER VISION Computer Vision is the process of extracting knowledge about the world from one or more digital images. 3

4. DIGITAL IMAGES How is an image represented in a computer? 4

5. DIGITAL IMAGES are 2D arrays (matrices) of numbers:

6. DIGITAL IMAGES Color Images are formed with three 2-D arrays, representing the Red, Green and Blue components of the image.

7. DIGITAL IMAGES More components: Depth

8. APPLICATIONS Why do we need Computer Vision? Mention a few applications/products that make use of Computer Vision 8

9. SELF-DRIVING CARS 9

10. MICROSOFT KINECT 10

11. DETECTION 11

12. RECOGNITION 12

13. SURVEILLANCE 13 In London there are 1,000,000+ cameras

14. BIOMETRICS 14

15. TRACKING 15 Continuous detection of objects of interest in video streams

16. EMOTION DETECTION 16

17. RECONSTRUCTION 17

18. RECONSTRUCTION 18

19. DETECTING PULSE FROM HEAD MOTION 19

20. COMPUTER VISION IS HARD Believe me! It is! It is often said that 2/3 (60%+) of the brain is "involved" in vision. 20

21. ORGANIZATION OF A CV SYSTEM The organization of a computer vision system varies a lot from one to another. However, these are some of the typical tasks found in these systems: 21

22. ORGANIZATION OF A CV SYSTEM 1. Image Acquisition 22

23. ORGANIZATION OF A CV SYSTEM 2. Pre-processing 23

24. ORGANIZATION OF A CV SYSTEM 3. Feature Extraction Lines, edges, interest points 24

25. ORGANIZATION OF A CV SYSTEM 4. High-level Processing 25

26. ORGANIZATION OF A CV SYSTEM 5. Decision Making 26

27. FACE DETECTION 27

28. QUESTION How do you think this is done? (There are multiple solutions to this problem) 28

29. “TRADITIONAL” APPROACH 29 xy f

30. MACHINE LEARNING APPROACH 30 x y Training set Learning algorithm f

31. TRAINING SET 31 Face (1) !Face (0)

32. LEARNING ALGORITHM Learning algorithms work with vectors of feature values We need to go from matrices to vectors 32 Extract Features Classifier Face / !Face

33. LEARNING ALGORITHM How do we extract features from images? What classifier should we use? 33

34. FACE DETECTION First real-time face detector proposed by Viola & Jones in 2005 “Robust real-time face detection” 34

35. FACE DETECTION - VIOLA & JONES, 2005 Robust (very true positive rate, very low false positive rate) Real Time (very fast) 35

36. FACE DETECTION - VIOLA & JONES, 2005 Type of features they use: Haar-like features 36 Extract Features Classifier Face / !Face

37. FACE DETECTION - VIOLA & JONES, 2005 37

38. FACE DETECTION - VIOLA & JONES, 2005 38

39. FACE DETECTION - VIOLA & JONES, 2005 39 V A = 64V A ≈ 0 V A = 16V A = -127

40. FACE DETECTION - VIOLA & JONES, 2005 How long does it take to extract these features? 40

41. For example: ImageIntegral Image 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 Integral Image: A table that holds the sum of all pixel values to the left and top of a given pixel, inclusive. FACE DETECTION - VIOLA & JONES, 2005

42. For example: ImageIntegral Image 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623 988 134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 FACE DETECTION - VIOLA & JONES, 2005

43. For example: ImageIntegral Image 98110121 125122129 99110120 116 129 97109124 111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623 988 134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 FACE DETECTION - VIOLA & JONES, 2005

44. For example: ImageIntegral Image 98110121125 122129 99110120116 129 97109124111 123134 98112132108 123133 97113147108 125142 95111168122 130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061 2751 34904294 68014492433325341185052 FACE DETECTION - VIOLA & JONES, 2005

45. 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 ImageIntegral Image (II) FACE DETECTION - VIOLA & JONES, 2005 Fast summations of arbitrary rectangles using integral images.

46. 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 ImageIntegral Image (II) Sum = II P +… = 3490 + … P FACE DETECTION - VIOLA & JONES, 2005

47. 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 ImageIntegral Image (II) Sum = II P – II Q + … = 3490 – 1137 + … Q P FACE DETECTION - VIOLA & JONES, 2005

48. 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 ImageIntegral Image (II) Sum = II P – II Q – II S + … = 3490 – 1137 – 1249 + … Q PS FACE DETECTION - VIOLA & JONES, 2005

49. 98110121125122129 99110120116 129 97109124111123134 98112132108123133 97113147108125142 95111168122130137 96104172130126130 98208329454576705 19741765889911371395 294623988134017012093 3928331330179022742799 48910431687225528643531 58412492061275134904294 68014492433325341185052 ImageIntegral Image (II) Sum = II P – II Q – II S + II R = 3490 – 1137 – 1249 + 417 = 1521 QR PS Can be computed in constant time with only 4 references FACE DETECTION - VIOLA & JONES, 2005

50. Feature extraction – DONE! Classifier? 50 Extract Features Classifier Face / !Face

51. FACE DETECTION - VIOLA & JONES, 2005 They use a variation of AdaBoost to build a cascade of weak classifiers. 51 Where stage i is simpler (and faster) than stage i+1

52. FACE DETECTION - VIOLA & JONES, 2005 We have a classifier that tells us if a given image is a face or not. What if we want to detect multiple faces in an image? Sliding window! 52

53. Idea: Slide windows of different sizes across image. At each location match the window to a face model. I.1 FACE DETECTION

54. Dealing with multiple scales Obvious solution: Build a detector for each possible scale Better idea: Build a detector for a single scale During detection, scale the image FACE DETECTION - VIOLA & JONES, 2005

55. MATLAB CODE 55

56. THANK YOU Questions? 56