Keypoint extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.

Slides:



Advertisements
Similar presentations
Feature extraction: Corners
Advertisements

Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.
CSE 473/573 Computer Vision and Image Processing (CVIP)
Interest Point Detectors (see CS485/685 notes for more details)
Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
Matching with Invariant Features
Computational Photography
Invariant Features.
Algorithms and Applications in Computer Vision
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Harris corner detector
Lecture 4: Feature matching
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Feature extraction: Corners and blobs
Interest Points and Corners Computer Vision CS 143, Brown James Hays Slides from Rick Szeliski, Svetlana Lazebnik, Derek Hoiem and Grauman&Leibe 2008 AAAI.
Detecting Patterns So far Specific patterns (eyes) Generally useful patterns (edges) Also (new) “Interesting” distinctive patterns ( No specific pattern:
Blob detection.
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
1 Image Features Hao Jiang Sept Image Matching 2.
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detection.
CS4670: Computer Vision Kavita Bala Lecture 8: Scale invariance.
1 Interest Operators Find “interesting” pieces of the image Multiple possible uses –image matching stereo pairs tracking in videos creating panoramas –object.
Summary of Previous Lecture A homography transforms one 3d plane to another 3d plane, under perspective projections. Those planes can be camera imaging.
CS 558 C OMPUTER V ISION Lecture VII: Corner and Blob Detection Slides adapted from S. Lazebnik.
Lecture 06 06/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Computer Vision Lecture 4 FEATURE DETECTION. Feature Detection Feature Description Feature Matching Today’s Topics 2.
Computational Photography Tamara Berg Features. Representing Images Keep all the pixels! Pros? Cons?
CSE 185 Introduction to Computer Vision Local Invariant Features.
CS 1699: Intro to Computer Vision Local Image Features: Extraction and Description Prof. Adriana Kovashka University of Pittsburgh September 17, 2015.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
Local invariant features 1 Thursday October 3 rd 2013 Neelima Chavali Virginia Tech.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Notes on the Harris Detector
Harris Corner Detector & Scale Invariant Feature Transform (SIFT)
Features Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/15 with slides by Trevor Darrell Cordelia Schmid, David Lowe, Darya Frolova, Denis Simakov,
Feature extraction: Corners and blobs. Why extract features? Motivation: panorama stitching We have two images – how do we combine them?
Local features and image matching October 1 st 2015 Devi Parikh Virginia Tech Disclaimer: Many slides have been borrowed from Kristen Grauman, who may.
Features Jan-Michael Frahm.
CS654: Digital Image Analysis
Instructor: Mircea Nicolescu Lecture 10 CS 485 / 685 Computer Vision.
CSE 185 Introduction to Computer Vision Local Invariant Features.
Lecture 10: Harris Corner Detector CS4670/5670: Computer Vision Kavita Bala.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
Lecture 5: Feature detection and matching CS4670 / 5670: Computer Vision Noah Snavely.
MAN-522: Computer Vision Edge detection Feature and blob detection
Interest Points EE/CSE 576 Linda Shapiro.
Distinctive Image Features from Scale-Invariant Keypoints
Local features: main components
3D Vision Interest Points.
Corners and interest points
TP12 - Local features: detection and description
Source: D. Lowe, L. Fei-Fei Recap: edge detection Source: D. Lowe, L. Fei-Fei.
Last lecture: Edges primer
Lecture 4: Harris corner detection
Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/22
Interest Points and Harris Corner Detector
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Lecture 5: Feature detection and matching
Local features and image matching
Lecture VI: Corner and Blob Detection
Lecture 5: Feature invariance
Presented by Xu Miao April 20, 2005
Lecture 5: Feature invariance
Corner Detection COMP 4900C Winter 2008.
Presentation transcript:

Keypoint extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC

Why extract keypoints? Motivation: panorama stitching We have two images – how do we combine them?

Why extract keypoints? Motivation: panorama stitching We have two images – how do we combine them? Step 1: extract keypoints Step 2: match keypoint features

Why extract keypoints? Motivation: panorama stitching We have two images – how do we combine them? Step 1: extract keypoints Step 2: match keypoint features Step 3: align images

Characteristics of good keypoints Repeatability The same keypoint can be found in several images despite geometric and photometric transformations Saliency Each keypoint is distinctive Compactness and efficiency Many fewer keypoints than image pixels Locality A keypoint occupies a relatively small area of the image; robust to clutter and occlusion

Applications Keypoints are used for: Image alignment 3D reconstruction Motion tracking Robot navigation Indexing and database retrieval Object recognition

Corner Detection: Basic Idea We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity “edge”: no change along the edge direction “corner”: significant change in all directions “flat” region: no change in all directions

Corner Detection: Mathematics Change in appearance of window W for the shift [u,v]: I(x, y) E(u, v) E(3,2)

Corner Detection: Mathematics I(x, y) E(u, v) E(0,0) Change in appearance of window W for the shift [u,v]:

Corner Detection: Mathematics We want to find out how this function behaves for small shifts E(u, v) Change in appearance of window W for the shift [u,v]:

Corner Detection: Mathematics First-order Taylor approximation for small motions [u, v]: Let’s plug this into E(u,v):

Corner Detection: Mathematics The quadratic approximation can be written as where M is a second moment matrix computed from image derivatives: (the sums are over all the pixels in the window W)

The surface E(u,v) is locally approximated by a quadratic form. Let’s try to understand its shape. Specifically, in which directions does it have the smallest/greatest change? Interpreting the second moment matrix E(u, v)

Consider a horizontal “slice” of E(u, v): Interpreting the second moment matrix This is the equation of an ellipse.

Consider a horizontal “slice” of E(u, v): Interpreting the second moment matrix This is the equation of an ellipse. The axis lengths of the ellipse are determined by the eigenvalues and the orientation is determined by R direction of the slowest change direction of the fastest change ( max ) -1/2 ( min ) -1/2 Diagonalization of M:

Consider the axis-aligned case (gradients are either horizontal or vertical) If either a or b is close to 0, then this is not a corner, so look for locations where both are large. Interpreting the second moment matrix

Visualization of second moment matrices

Interpreting the eigenvalues 1 2 “Corner” 1 and 2 are large, 1 ~ 2 ; E increases in all directions 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Edge” 2 >> 1 “Flat” region Classification of image points using eigenvalues of M:

Corner response function “Corner” R > 0 “Edge” R < 0 “Flat” region |R| small α : constant (0.04 to 0.06)

The Harris corner detector 1.Compute partial derivatives at each pixel 2.Compute second moment matrix M in a Gaussian window around each pixel: C.Harris and M.Stephens. “A Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference: pages 147—151, “A Combined Corner and Edge Detector.”

The Harris corner detector 1.Compute partial derivatives at each pixel 2.Compute second moment matrix M in a Gaussian window around each pixel 3.Compute corner response function R C.Harris and M.Stephens. “A Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference: pages 147—151, “A Combined Corner and Edge Detector.”

Harris Detector: Steps

Compute corner response R

The Harris corner detector 1.Compute partial derivatives at each pixel 2.Compute second moment matrix M in a Gaussian window around each pixel 3.Compute corner response function R 4.Threshold R 5.Find local maxima of response function (nonmaximum suppression) C.Harris and M.Stephens. “A Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference: pages 147—151, “A Combined Corner and Edge Detector.”

Harris Detector: Steps Find points with large corner response: R > threshold

Harris Detector: Steps Take only the points of local maxima of R

Harris Detector: Steps

Robustness of corner features What happens to corner features when the image undergoes geometric or photometric transformations?

Affine intensity change Only derivatives are used => invariance to intensity shift I  I + b Intensity scaling: I  a I R x (image coordinate) threshold R x (image coordinate) Partially invariant to affine intensity change I  a I + b

Image translation Derivatives and window function are shift-invariant Corner location is covariant w.r.t. translation

Image rotation Second moment ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner location is covariant w.r.t. rotation

Scaling All points will be classified as edges Corner Corner location is not covariant to scaling!