Download presentation

Presentation is loading. Please wait.

Published byCory Bricker Modified over 2 years ago

1
CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai

2
Today’s class Template matching Gaussian Pyramids Laplacian Pyramids Image denoising

3
Template matching Goal: find in image

4
Template matching Goal: find in image Main challenge: What is a good similarity or distance measure between two patches?

5
Template matching Goal: find in image Main challenge: What is a good similarity or distance measure between two patches? – Correlation – Zero-mean correlation – Sum Square Difference – Normalized Cross Correlation Slide: Hoiem

6
Matching with filters Goal: find in image Method 0: filter the image with eye patch Input Filtered Image What went wrong? f = image g = filter Slide: Hoiem

7
Problem with Correlation

8

9
Solution

10
Slide: Hoiem Matching with filters Goal: find in image Method 1: filter the image with zero-mean eye Input Filtered Image (scaled) Thresholded Image True detections False detections mean of f

11
Slide: Hoiem Matching with filters Goal: find in image Method 2: SSD Input 1- sqrt(SSD) Thresholded Image True detections

12
Relationship SSD and Correlation

13
Matching with filters Goal: find in image Method 2: SSD Input 1- sqrt(SSD) What’s the potential downside of SSD? Slide: Hoiem

14
Matching with filters Goal: find in image Method 3: Normalized cross-correlation - subtracting the mean - dividing by the standard deviationstandard deviation

15
Matching with filters Goal: find in image Method 3: Normalized cross-correlation Matlab: normxcorr2(template, im) mean image patch mean template

16
Slide: Hoiem Matching with filters Goal: find in image Method 3: Normalized cross-correlation Input Normalized X-Correlation Thresholded Image True detections

17
Matching with filters Goal: find in image Method 3: Normalized cross-correlation Input Normalized X-Correlation Thresholded Image True detections Slide: Hoiem

18
Q: What is the best method to use? A: Depends SSD: faster, sensitive to overall intensity Normalized cross-correlation: slower, invariant to local average intensity and contrast

19
Q: What if we want to find larger or smaller eyes? A: Image Pyramid

20
Gaussian Pyramid Low-Pass Filtered Image Image Gaussian Filter Sample Low-Res Image

21
Gaussian Pyramid filter mask Repeat –Filter –Subsample Until minimum resolution reached –can specify desired number of levels (e.g., 3-level pyramid )

22
Gaussian pyramid

23
Template Matching with Image Pyramids Input: Image, Template 1.Match template at current scale 2.Downsample image 3.Repeat 1-2 until image is very small 4.Take responses above some threshold, perhaps with non-maxima suppression

24
Laplacian pyramid

25
Laplacian Filter The filtering kernel for Laplacian is Finite difference allows us to approximate the Laplacian using the following filtering kernel.

26
Laplacian Pyramid Laplacian Pyramid (subband images) - Created from Gaussian pyramid by subtraction

27
The Laplacian of Gaussian Kernel is the Laplacian operator:

28
The Laplacian of Gaussian Filter Kernel

29
Laplacian Pyramid What happens in frequency domain?

30
Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid?

31
Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid? - Yes, laplacian pyramid is a complete image representation which encodes fine to course structure in a different level

32
Image representation Pixels: great for spatial resolution, poor access to frequency Fourier transform: great for frequency, not for spatial info such as locations. Pyramids/filter banks: balance between spatial and frequency information

33
Major uses of image pyramids Compression Object detection – Scale search – Features Detecting stable interest points Image blending/mosaicing Registration – Course-to-fine

34
Multi-resolution SIFT Feature Detection - Object recognition from local scale-invariant features [pdf link], ICCV 09pdf link - David G. Lowe, "Distinctive image features from scale- invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp

35
Pyramid Blending

36
laplacian level 4 laplacian level 2 laplacian level 0 left pyramidright pyramidblended pyramid

37
Hierarchical Image Registration 1.Compute Gaussian pyramid 2.Align with coarse pyramid 3.Successively align with finer pyramids – Search smaller range Why is this faster? - Hierarchical Model-Based Motion Estimation- Hierarchical Model-Based Motion Estimation, ECCV 92

38
Denoising Additive Gaussian Noise Gaussian Filter

39
Gaussian inputGaussian filter

40
Median Filter For each neighbor in image, sliding the window Sort pixel values Set the center pixel to the median

41
Median Filter input Gaussian filterMedian filter

42
Median Filter Examples inputMedian 7X7

43
Median Filter Examples Median 11X11 Median 3X3

44
Median Filter Examples Median 11X11 Median 3X3 Straight edges kept Sharp features lost

45
Median Filter Properties Can remove outliers (peppers and salts) Window size controls size of structure Preserve some details but sharp corners and edges might get lost

46
Comparison of Mean, Gaussian, and Median originalMean with 6 pixels

47
Comparison of Mean, Gaussian, and Median originalGaussian with 6 pixels

48
Comparison of Mean, Gaussian, and Median originalMedian with 6 pixels

49
Common Problems Mean: blurs image, removes simple noise, no details are preserved

50
Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ.

51
Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise

52
Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves sharp features such as edges?

53
Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves sharp features such as edges? -yes, Tomasi's Bilateral Filter!Tomasi's Bilateral Filter

54
Bilateral Filter Example Gaussian filter Bilateral filter

55
Bilateral Filter Example Gaussian filter Bilateral filter

56
1D Graphical Example Center Sample u It is clear that in weighting this neighborhood, we would like to preserve the step Neighborhood I(p) p

57
The Weights p I(p)

58
Filtered Values p I(p) Filtered value

59
Edges Are Smoothed p I(p) Filtered value

60
What Causes the Problem? p I(p) Filtered value

61
What Causes the Problem? p I(p) Filtered value Same weights for these two pixels!!

62
The Weights p I(p)

63
Bilateral filter Denoise Feature preserving Normalization Bilateral Filtering

64
Filter Parameters As proposed by Tomasi and Manduchi, the filter is controlled by 3 parameters: The filter can be applied for several iterations in order to further strengthen its edge-preserving smoothing N(u) – The neighbor size of the filter support, c – The variance of the spatial distances, s – The variance of the value distances,

65
Bilateral Filter Results Original

66
Bilateral Filter Results σ c = 3, σ s = 3

67
Bilateral Filter Results σ c = 6, σ s = 3

68
Bilateral Filter Results σ c = 12, σ s = 3

69
Bilateral Filter Results σ c = 12, σ s = 6

70
Bilateral Filter Results σ c = 15, σ s = 8

71
Summary – Template matching (SSD or Normxcorr2) SSD can be done with linear filters, is sensitive to overall intensity – Gaussian pyramid Coarse-to-fine search, multi-scale detection – Laplacian pyramid More compact image representation Can be used for compositing in graphics – Image denoising

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google