Download presentation

Presentation is loading. Please wait.

Published byBaylee Seavy Modified over 2 years ago

1
Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction

2
Preprocessing The goal of pre-processing is – to try to reduce unwanted variation in image due to lighting, scale, deformation etc. – to reduce data to a manageable size Give the subsequent model a chance Preprocessing definition: deterministic transformation of pixels p to create data vector x Usually heuristics based on experience 2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

3
Structure 33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

4
Normalization Fix first and second moments to standard values Remove contrast and constant additive luminance variations Before After 4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

5
Histogram Equalization Make all of the moments the same by forcing the histogram of intensities to be the same Before/ normalized/ Histogram Equalized 5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

6
Histogram Equalization 6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

7
Convolution Takes pixel image P and applies a filter F Computes weighted sum of pixel values, where weights given by filter. Easiest to see with a concrete example 7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

8
Blurring (convolve with Gaussian) 8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

9
Gradient Filters Rule of thumb: big response when image matches filter 9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

10
Gabor Filters 10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

11
Haar Filters 11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

12
Local binary patterns 12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

13
Textons An attempt to characterize texture Replace each pixel with integer representing the texture ‘type’ 13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

14
Computing Textons Take a bank of filters and apply to lots of images Cluster in filter space For new pixel, filter surrounding region with same bank, and assign to nearest cluster 14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

15
Structure 15 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

16
Edges (from Elder and Goldberg 2000) 16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

17
Canny Edge Detector 17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

18
Canny Edge Detector Compute horizontal and vertical gradient images h and v 18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

19
Canny Edge Detector Quantize to 4 directions 19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

20
Canny Edge Detector Non-maximal suppression 20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

21
Canny Edge Detector Hysteresis Thresholding 21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

22
Harris Corner Detector Make decision based on image structure tensor 22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

23
SIFT Detector Filter with difference of Gaussian filters at increasing scales Build image stack (scale space) Find extrema in this 3D volume 23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

24
SIFT Detector Identified Corners Remove those on edges Remove those where contrast is low 24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

25
Assign Orientation Orientation assigned by looking at intensity gradients in region around point Form a histogram of these gradients by binning. Set orientation to peak of histogram. 25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

26
Structure 26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

27
Sift Descriptor Goal: produce a vector that describes the region around the interest point. All calculations are relative to the orientation and scale of the keypoint Makes descriptor invariant to rotation and scale 27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

28
Sift Descriptor 1. Compute image gradients 2. Pool into local histograms 3. Concatenate histograms 4. Normalize histograms 28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

29
HoG Descriptor 29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

30
Bag of words descriptor Compute visual features in image Compute descriptor around each Find closest match in library and assign index Compute histogram of these indices over the region Dictionary computed using K-means 30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

31
Shape context descriptor 31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

32
Structure 32 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

33
Dimensionality Reduction where is a function that takes the hidden variable and a set of parameters . Dimensionality reduction attempt to find a low dimensional (or hidden) representation h which can approximately explain the data x so that Typically, we choose the function family and then learn h and from training data 33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

34
Least Squares Criterion Choose the parameters and the hidden variables h so that they minimize the least squares approximation error (a measure of how well they can reconstruct the data x ). 34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

35
Simple Example Approximate each data example x with a scalar value h. Data is reconstructed by multiplying h by a parameter and adding the mean vector .... or even better, lets subtract from each data example to get mean-zero data 35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

36
Simple Example Approximate each data example x with a scalar value h. Data is reconstructed by multiplying h by a factor . Criterion: 36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

37
Criterion Problem: the problem is non-unique. If we multiply f by any constant and divide each of the hidden variables h 1...I by the same constant we get the same cost. (i.e. (f ) (h i / ) = fh i ) Solution: We make the solution unique by constraining the length of f to be 1 using a Lagrange multiplier. 37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

38
Criterion Now we have the new cost function: To optimize this we take derivatives with respect to and h i, equate the resulting expressions to zero and re-arrange. 38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

39
Solution To compute the hidden value, take dot product with the vector 39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

40
Solution To compute the hidden value, take dot product with the vector or where To compute the vector , compute the first eigenvector of the scatter matrix. 40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

41
Computing h To compute the hidden value, take dot product with the vector 41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

42
Reconstruction To reconstruct, multiply the hidden variable h by vector . 42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

43
Principal Components Analysis Same idea, but not the hidden variable h is multi-dimensional. Each components weights one column of matrix F so that data is approximated as This leads to cost function: This has a non-unique optimum so we enforce the constraint that F should be a (truncated) rotation matrix and F T F=I 43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

44
PCA Solution To compute the hidden vector, take dot product with each column of . To compute the matrix , compute the first eigenvectors of the scatter matrix. The basis functions in the columns of are called principal components and the entries of h are called loadings 44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

45
Dual PCA Problem: PCA as described has a major drawback. We need to compute the eigenvectors of the scatter matrix But this has size D x x D x. Visual data tends to be very high dimensional, so this may be extremely large. Solution: Reparameterize the principal components as weighted sums of the data...and solve for the new variables . 45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

46
Geometric Interpretation Each column of can be described as a weighted sum of the original datapoints. Weights given in the corresponding columns of the new variable . 46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

47
Motivation Solution: Reparameterize the principal components as weighted sums of the data...and solve for the new variables . Why? If the number of datapoints I is less than the number of observed dimension D x then the will be smaller than and the resulting optimization becomes easier. Intuition: we are not interested in principal components that are not in the subspace spanned by the data anyway. 47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

48
Cost functions...subject to T =I or T X T X =I....subject to T =I. Principal components analysis Dual principal components analysis 48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

49
Solution To compute the hidden vector, take dot product with each column of = X. 49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

50
Solution To compute the matrix , compute the first eigenvectors of the inner product matrix X T X. The inner product matrix has size I x I. If the number of examples I is less than the dimensionality of the data D x then this is a smaller eigenproblem. 50Computer vision: models, learning and inference. ©2011 Simon J.D. Prince To compute the hidden vector, take dot product with each column of = X.

51
K-Means algorithm Approximate data with a set of means 51Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Least squares criterion Alternate minimization

52
52Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

Similar presentations

OK

Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.

Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google