DIEGO AGUIRRE COMPUTER VISION INTRODUCTION 1

QUESTION What is Computer Vision? 2

COMPUTER VISION Computer Vision is the process of extracting knowledge about the world from one or more digital images. 3

DIGITAL IMAGES How is an image represented in a computer? 4

DIGITAL IMAGES are 2D arrays (matrices) of numbers:

DIGITAL IMAGES Color Images are formed with three 2-D arrays, representing the Red, Green and Blue components of the image.

DIGITAL IMAGES More components: Depth

APPLICATIONS Why do we need Computer Vision? Mention a few applications/products that make use of Computer Vision 8

SELF-DRIVING CARS 9

MICROSOFT KINECT 10

DETECTION 11

RECOGNITION 12

SURVEILLANCE 13 In London there are 1,000,000+ cameras

BIOMETRICS 14

TRACKING 15 Continuous detection of objects of interest in video streams

EMOTION DETECTION 16

RECONSTRUCTION 17

RECONSTRUCTION 18

DETECTING PULSE FROM HEAD MOTION 19

COMPUTER VISION IS HARD Believe me! It is! It is often said that 2/3 (60%+) of the brain is "involved" in vision. 20

ORGANIZATION OF A CV SYSTEM The organization of a computer vision system varies a lot from one to another. However, these are some of the typical tasks found in these systems: 21

ORGANIZATION OF A CV SYSTEM 1. Image Acquisition 22

ORGANIZATION OF A CV SYSTEM 2. Pre-processing 23

ORGANIZATION OF A CV SYSTEM 3. Feature Extraction Lines, edges, interest points 24

ORGANIZATION OF A CV SYSTEM 4. High-level Processing 25

ORGANIZATION OF A CV SYSTEM 5. Decision Making 26

FACE DETECTION 27

QUESTION How do you think this is done? (There are multiple solutions to this problem) 28

“TRADITIONAL” APPROACH 29 xy f

MACHINE LEARNING APPROACH 30 x y Training set Learning algorithm f

TRAINING SET 31 Face (1) !Face (0)

LEARNING ALGORITHM Learning algorithms work with vectors of feature values We need to go from matrices to vectors 32 Extract Features Classifier Face / !Face

LEARNING ALGORITHM How do we extract features from images? What classifier should we use? 33

FACE DETECTION First real-time face detector proposed by Viola & Jones in 2005 “Robust real-time face detection” 34

FACE DETECTION - VIOLA & JONES, 2005 Robust (very true positive rate, very low false positive rate) Real Time (very fast) 35

FACE DETECTION - VIOLA & JONES, 2005 Type of features they use: Haar-like features 36 Extract Features Classifier Face / !Face

FACE DETECTION - VIOLA & JONES,

FACE DETECTION - VIOLA & JONES,

FACE DETECTION - VIOLA & JONES, V A = 64V A ≈ 0 V A = 16V A = -127

FACE DETECTION - VIOLA & JONES, 2005 How long does it take to extract these features? 40

For example: ImageIntegral Image Integral Image: A table that holds the sum of all pixel values to the left and top of a given pixel, inclusive. FACE DETECTION - VIOLA & JONES, 2005

For example: ImageIntegral Image FACE DETECTION - VIOLA & JONES, 2005

For example: ImageIntegral Image FACE DETECTION - VIOLA & JONES, 2005

For example: ImageIntegral Image FACE DETECTION - VIOLA & JONES, 2005

ImageIntegral Image (II) FACE DETECTION - VIOLA & JONES, 2005 Fast summations of arbitrary rectangles using integral images.

ImageIntegral Image (II) Sum = II P +… = … P FACE DETECTION - VIOLA & JONES, 2005

ImageIntegral Image (II) Sum = II P – II Q + … = 3490 – … Q P FACE DETECTION - VIOLA & JONES, 2005

ImageIntegral Image (II) Sum = II P – II Q – II S + … = 3490 – 1137 – … Q PS FACE DETECTION - VIOLA & JONES, 2005

ImageIntegral Image (II) Sum = II P – II Q – II S + II R = 3490 – 1137 – = 1521 QR PS Can be computed in constant time with only 4 references FACE DETECTION - VIOLA & JONES, 2005

Feature extraction – DONE! Classifier? 50 Extract Features Classifier Face / !Face

FACE DETECTION - VIOLA & JONES, 2005 They use a variation of AdaBoost to build a cascade of weak classifiers. 51 Where stage i is simpler (and faster) than stage i+1

FACE DETECTION - VIOLA & JONES, 2005 We have a classifier that tells us if a given image is a face or not. What if we want to detect multiple faces in an image? Sliding window! 52

Idea: Slide windows of different sizes across image. At each location match the window to a face model. I.1 FACE DETECTION

Dealing with multiple scales Obvious solution: Build a detector for each possible scale Better idea: Build a detector for a single scale During detection, scale the image FACE DETECTION - VIOLA & JONES, 2005

MATLAB CODE 55

THANK YOU Questions? 56