Download presentation

1
**Computational Photography**

Prof. Feng Liu Spring 2015 04/22/2015 Computer Graphics

2
Last Time Re-lighting HDR

3
**Today Panorama Mid-term project presentation Overview**

Feature detection Mid-term project presentation Not real mid-term 4 minutes presentation Schedule May 6 With slides by Prof. C. Dyer and K. Grauman

4
**Panorama Building: History**

Along the River During Ching Ming Festival by Z.D Zhang ( ) San Francisco from Rincon Hill, 1851, by Martin Behrmanx

5
**Panorama Building: A Concise History**

The state of the art and practice is good at assembling images into panoramas Mid 90s –Commercial Players (e.g. QuicktimeVR) Late 90s –Robust stitchers(in research) Early 00s –Consumer stitching common Mid 00s –Automation

6
**Stitching Recipe Align pairs of images Align all to a common frame**

Adjust (Global) & Blend

7
**Stitching Images Together**

8
**When do two images “stitch”?**

9
**Images can be transformed to match**

10
**Images are related by Homographies**

11
Compute Homographies

12
**Automatic Feature Points Matching**

Match local neighborhoods around points Use descriptors to efficiently compare SIFT [Lowe 04] most common choice

13
**Stitching Recipe Align pairs of images Align all to a common frame**

Adjust (Global) & Blend

14
**Wide Baseline Matching**

Images taken by cameras that are far apart make the correspondence problem very difficult Feature-based approach: Detect and match feature points in pairs of images Credit: C. Dyer Credit: C. Dyer

15
**Matching with Features**

Detect feature points Find corresponding pairs Slide credit: C. Dyer Credit: C. Dyer

16
**Matching with Features**

Problem 1: Detect the same point independently in both images no chance to match! We need a repeatable detector Credit: C. Dyer

17
**Matching with Features**

Problem 2: For each point correctly recognize the corresponding point ? We need a reliable and distinctive descriptor Credit: C. Dyer

18
**Properties of an Ideal Feature**

Local: features are local, so robust to occlusion and clutter (no prior segmentation) Invariant (or covariant) to many kinds of geometric and photometric transformations Robust: noise, blur, discretization, compression, etc. do not have a big impact on the feature Distinctive: individual features can be matched to a large database of objects Quantity: many features can be generated for even small objects Accurate: precise localization Efficient: close to real-time performance Credit: C. Dyer

19
**Problem 1: Detecting Good Feature Points**

[Image from T. Tuytelaars ECCV 2006 tutorial] Credit: C. Dyer

20
**Feature Detectors Hessian Harris Lowe: SIFT (DoG)**

Mikolajczyk & Schmid: Hessian/Harris-Laplacian/Affine Tuytelaars & Van Gool: EBR and IBR Matas: MSER Kadir & Brady: Salient Regions Others Credit: C. Dyer

21
**Harris Corner Point Detector**

C. Harris, M. Stephens, “A Combined Corner and Edge Detector,” 1988 Credit: C. Dyer

22
**Harris Detector: Basic Idea**

We should recognize the point by looking through a small window Shifting a window in any direction should give a large change in response Credit: C. Dyer

23
**Harris Detector: Basic Idea**

“flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions Credit: C. Dyer

24
**Harris Detector: Derivation**

Change of intensity for a (small) shift by [u,v] in image I: Intensity Weighting function Shifted intensity or Weighting function w(x,y) = Gaussian 1 in window, 0 outside Credit: R. Szeliski

25
**Harris Detector Apply 2nd order Taylor series expansion:**

Credit: R. Szeliski

26
**Harris Corner Detector**

Expanding E(u,v) in a 2nd order Taylor series, we have, for small shifts, [u,v], a bilinear approximation: where M is a 2 2 matrix computed from image derivatives: Note: Sum computed over small neighborhood around given pixel Credit: R. Szeliski

27
**Harris Corner Detector**

Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change Ellipse E(u,v) = const direction of the slowest change (max)-1/2 (min)-1/2 Credit: R. Szeliski

28
**Selecting Good Features**

Image patch SSD surface l1 and l2 both large Credit: C. Dyer

29
**Selecting Good Features**

SSD surface large l1, small l2 Credit: C. Dyer

30
**Selecting Good Features**

SSD surface small l1, small l2 Credit: C. Dyer

31
**Harris Corner Detector**

2 “Edge” 2 >> 1 “Corner” 1 and 2 both large, 1 ~ 2; E increases in all directions Classification of image points using eigenvalues of M: 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1 Credit: C. Dyer

32
**Harris Corner Detector**

Measure of corner response: k is an empirically-determined constant; e.g., k = 0.05 Credit: C. Dyer

33
**Harris Corner Detector**

2 “Edge” “Corner” R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge |R| is small for a flat region R < 0 R > 0 “Flat” “Edge” |R| small R < 0 1 Credit: C. Dyer

34
**Harris Corner Detector: Algorithm**

Find points with large corner response function R (i.e., R > threshold) Take the points of local maxima of R (for localization) by non-maximum suppression Credit: C. Dyer

35
**Harris Detector: Example**

Credit: C. Dyer

36
**Harris Detector: Example**

Compute corner response R = 12 – k(1 + 2)2 Credit: C. Dyer

37
**Harris Detector: Example**

Find points with large corner response: R > threshold Credit: C. Dyer

38
**Harris Detector: Example**

Take only the points of local maxima of R Credit: C. Dyer

39
**Harris Detector: Example**

Credit: C. Dyer

40
**Harris Detector: Example**

Interest points extracted with Harris (~ 500 points) Credit: C. Dyer

41
**Harris Detector: Example**

Credit: C. Dyer

42
**Harris Detector: Summary**

Average intensity change in direction [u,v] can be expressed in bilinear form: Describe a point in terms of eigenvalues of M: measure of corner response: A good (corner) point should have a large intensity change in all directions, i.e., R should be a large positive value Credit: C. Dyer

43
**Harris Detector Properties**

Rotation invariance Ellipse rotates but its shape (i.e., eigenvalues) remains the same Corner response R is invariant to image rotation Credit: C. Dyer

44
**Harris Detector Properties**

But not invariant to image scale Fine scale: All points will be classified as edges Coarse scale: Corner Credit: C. Dyer

45
**Harris Detector Properties**

Quality of Harris detector for different scale changes Repeatability rate: # correct correspondences # possible correspondences C. Schmid et al., “Evaluation of Interest Point Detectors,” IJCV 2000 Credit: C. Dyer

46
**Invariant Local Features**

Goal: Detect the same interest points regardless of image changes due to translation, rotation, scale, viewpoint

47
**Models of Image Change Geometry Photometry Rotation**

Similarity (rotation + uniform scale) Affine (scale dependent on direction) valid for: orthographic camera, locally planar object Photometry Affine intensity change (I a I + b) Credit: C. Dyer

48
SIFT Detector [Lowe ’04] Difference-of-Gaussian (DoG) is an approximation of the Laplacian-of-Gaussian (LoG) = Lowe, D. G., “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, 60, 2, pp , 2004 Credit: C. Dyer

49
SIFT Detector Credit: C. Dyer

50
SIFT Detector Credit: C. Dyer

51
**SIFT Detector Algorithm Summary**

Detect local maxima in position and scale of squared values of difference-of-Gaussian Fit a quadratic to surrounding values for sub-pixel and sub-scale interpolation Output = list of (x, y, ) points Credit: C. Dyer

52
**References on Feature Descriptors**

A performance evaluation of local descriptors, K. Mikolajczyk and C. Schmid, IEEE Trans. PAMI 27(10), 2005 Evaluation of features detectors and descriptors based on 3D objects, P. Moreels and P. Perona, Int. J. Computer Vision 73(3), 2007 Credit: C. Dyer

53
Next Time Panorama Feature matching

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google