# CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai.

## Presentation on theme: "CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai."— Presentation transcript:

CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai

Today’s class Template matching Gaussian Pyramids Laplacian Pyramids Image denoising

Template matching Goal: find in image

Template matching Goal: find in image Main challenge: What is a good similarity or distance measure between two patches?

Template matching Goal: find in image Main challenge: What is a good similarity or distance measure between two patches? – Correlation – Zero-mean correlation – Sum Square Difference – Normalized Cross Correlation Slide: Hoiem

Matching with filters Goal: find in image Method 0: filter the image with eye patch Input Filtered Image What went wrong? f = image g = filter Slide: Hoiem

Problem with Correlation

Solution

Slide: Hoiem Matching with filters Goal: find in image Method 1: filter the image with zero-mean eye Input Filtered Image (scaled) Thresholded Image True detections False detections mean of f

Slide: Hoiem Matching with filters Goal: find in image Method 2: SSD Input 1- sqrt(SSD) Thresholded Image True detections

Relationship SSD and Correlation

Matching with filters Goal: find in image Method 2: SSD Input 1- sqrt(SSD) What’s the potential downside of SSD? Slide: Hoiem

Matching with filters Goal: find in image Method 3: Normalized cross-correlation - subtracting the mean - dividing by the standard deviationstandard deviation

Matching with filters Goal: find in image Method 3: Normalized cross-correlation Matlab: normxcorr2(template, im) mean image patch mean template

Slide: Hoiem Matching with filters Goal: find in image Method 3: Normalized cross-correlation Input Normalized X-Correlation Thresholded Image True detections

Matching with filters Goal: find in image Method 3: Normalized cross-correlation Input Normalized X-Correlation Thresholded Image True detections Slide: Hoiem

Q: What is the best method to use? A: Depends SSD: faster, sensitive to overall intensity Normalized cross-correlation: slower, invariant to local average intensity and contrast

Q: What if we want to find larger or smaller eyes? A: Image Pyramid

Gaussian Pyramid Low-Pass Filtered Image Image Gaussian Filter Sample Low-Res Image

Gaussian Pyramid filter mask Repeat –Filter –Subsample Until minimum resolution reached –can specify desired number of levels (e.g., 3-level pyramid )

Gaussian pyramid

Template Matching with Image Pyramids Input: Image, Template 1.Match template at current scale 2.Downsample image 3.Repeat 1-2 until image is very small 4.Take responses above some threshold, perhaps with non-maxima suppression

Laplacian pyramid

Laplacian Filter The filtering kernel for Laplacian is Finite difference allows us to approximate the Laplacian using the following filtering kernel.

Laplacian Pyramid Laplacian Pyramid (subband images) - Created from Gaussian pyramid by subtraction

The Laplacian of Gaussian Kernel is the Laplacian operator:

The Laplacian of Gaussian Filter Kernel

Laplacian Pyramid What happens in frequency domain?

Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid?

Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid? - Yes, laplacian pyramid is a complete image representation which encodes fine to course structure in a different level

Image representation Pixels: great for spatial resolution, poor access to frequency Fourier transform: great for frequency, not for spatial info such as locations. Pyramids/filter banks: balance between spatial and frequency information

Major uses of image pyramids Compression Object detection – Scale search – Features Detecting stable interest points Image blending/mosaicing Registration – Course-to-fine

Multi-resolution SIFT Feature Detection - Object recognition from local scale-invariant features [pdf link], ICCV 09pdf link - David G. Lowe, "Distinctive image features from scale- invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110

Pyramid Blending

laplacian level 4 laplacian level 2 laplacian level 0 left pyramidright pyramidblended pyramid

Hierarchical Image Registration 1.Compute Gaussian pyramid 2.Align with coarse pyramid 3.Successively align with finer pyramids – Search smaller range Why is this faster? - Hierarchical Model-Based Motion Estimation- Hierarchical Model-Based Motion Estimation, ECCV 92

Denoising Additive Gaussian Noise Gaussian Filter

Gaussian inputGaussian filter

Median Filter For each neighbor in image, sliding the window Sort pixel values Set the center pixel to the median

Median Filter input Gaussian filterMedian filter

Median Filter Examples inputMedian 7X7

Median Filter Examples Median 11X11 Median 3X3

Median Filter Examples Median 11X11 Median 3X3 Straight edges kept Sharp features lost

Median Filter Properties Can remove outliers (peppers and salts) Window size controls size of structure Preserve some details but sharp corners and edges might get lost

Comparison of Mean, Gaussian, and Median originalMean with 6 pixels

Comparison of Mean, Gaussian, and Median originalGaussian with 6 pixels

Comparison of Mean, Gaussian, and Median originalMedian with 6 pixels

Common Problems Mean: blurs image, removes simple noise, no details are preserved

Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ.

Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise

Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves sharp features such as edges?

Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves sharp features such as edges? -yes, Tomasi's Bilateral Filter!Tomasi's Bilateral Filter

Bilateral Filter Example Gaussian filter Bilateral filter

Bilateral Filter Example Gaussian filter Bilateral filter

1D Graphical Example Center Sample u It is clear that in weighting this neighborhood, we would like to preserve the step Neighborhood I(p) p

The Weights p I(p)

Filtered Values p I(p) Filtered value

Edges Are Smoothed p I(p) Filtered value

What Causes the Problem? p I(p) Filtered value

What Causes the Problem? p I(p) Filtered value Same weights for these two pixels!!

The Weights p I(p)

Bilateral filter Denoise Feature preserving Normalization Bilateral Filtering

Filter Parameters As proposed by Tomasi and Manduchi, the filter is controlled by 3 parameters: The filter can be applied for several iterations in order to further strengthen its edge-preserving smoothing N(u) – The neighbor size of the filter support,  c – The variance of the spatial distances,  s – The variance of the value distances,

Bilateral Filter Results Original

Bilateral Filter Results σ c = 3, σ s = 3

Bilateral Filter Results σ c = 6, σ s = 3

Bilateral Filter Results σ c = 12, σ s = 3

Bilateral Filter Results σ c = 12, σ s = 6

Bilateral Filter Results σ c = 15, σ s = 8

Summary – Template matching (SSD or Normxcorr2) SSD can be done with linear filters, is sensitive to overall intensity – Gaussian pyramid Coarse-to-fine search, multi-scale detection – Laplacian pyramid More compact image representation Can be used for compositing in graphics – Image denoising

Download ppt "CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai."

Similar presentations