Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computational Photography

Similar presentations

Presentation on theme: "Computational Photography"— Presentation transcript:

1 Computational Photography
Prof. Feng Liu Spring 2015 04/22/2015 Computer Graphics

2 Last Time Re-lighting HDR

3 Today Panorama Mid-term project presentation Overview
Feature detection Mid-term project presentation Not real mid-term 4 minutes presentation Schedule May 6 With slides by Prof. C. Dyer and K. Grauman

4 Panorama Building: History
Along the River During Ching Ming Festival by Z.D Zhang ( ) San Francisco from Rincon Hill, 1851, by Martin Behrmanx

5 Panorama Building: A Concise History
The state of the art and practice is good at assembling images into panoramas Mid 90s –Commercial Players (e.g. QuicktimeVR) Late 90s –Robust stitchers(in research) Early 00s –Consumer stitching common Mid 00s –Automation

6 Stitching Recipe Align pairs of images Align all to a common frame
Adjust (Global) & Blend

7 Stitching Images Together

8 When do two images “stitch”?

9 Images can be transformed to match

10 Images are related by Homographies

11 Compute Homographies

12 Automatic Feature Points Matching
Match local neighborhoods around points Use descriptors to efficiently compare SIFT [Lowe 04] most common choice

13 Stitching Recipe Align pairs of images Align all to a common frame
Adjust (Global) & Blend

14 Wide Baseline Matching
Images taken by cameras that are far apart make the correspondence problem very difficult Feature-based approach: Detect and match feature points in pairs of images Credit: C. Dyer Credit: C. Dyer

15 Matching with Features
Detect feature points Find corresponding pairs Slide credit: C. Dyer Credit: C. Dyer

16 Matching with Features
Problem 1: Detect the same point independently in both images no chance to match! We need a repeatable detector Credit: C. Dyer

17 Matching with Features
Problem 2: For each point correctly recognize the corresponding point ? We need a reliable and distinctive descriptor Credit: C. Dyer

18 Properties of an Ideal Feature
Local: features are local, so robust to occlusion and clutter (no prior segmentation) Invariant (or covariant) to many kinds of geometric and photometric transformations Robust: noise, blur, discretization, compression, etc. do not have a big impact on the feature Distinctive: individual features can be matched to a large database of objects Quantity: many features can be generated for even small objects Accurate: precise localization Efficient: close to real-time performance Credit: C. Dyer

19 Problem 1: Detecting Good Feature Points
[Image from T. Tuytelaars ECCV 2006 tutorial] Credit: C. Dyer

20 Feature Detectors Hessian Harris Lowe: SIFT (DoG)
Mikolajczyk & Schmid: Hessian/Harris-Laplacian/Affine Tuytelaars & Van Gool: EBR and IBR Matas: MSER Kadir & Brady: Salient Regions Others Credit: C. Dyer

21 Harris Corner Point Detector
C. Harris, M. Stephens, “A Combined Corner and Edge Detector,” 1988 Credit: C. Dyer

22 Harris Detector: Basic Idea
We should recognize the point by looking through a small window Shifting a window in any direction should give a large change in response Credit: C. Dyer

23 Harris Detector: Basic Idea
“flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions Credit: C. Dyer

24 Harris Detector: Derivation
Change of intensity for a (small) shift by [u,v] in image I: Intensity Weighting function Shifted intensity or Weighting function w(x,y) = Gaussian 1 in window, 0 outside Credit: R. Szeliski

25 Harris Detector Apply 2nd order Taylor series expansion:
Credit: R. Szeliski

26 Harris Corner Detector
Expanding E(u,v) in a 2nd order Taylor series, we have, for small shifts, [u,v], a bilinear approximation: where M is a 2  2 matrix computed from image derivatives: Note: Sum computed over small neighborhood around given pixel Credit: R. Szeliski

27 Harris Corner Detector
Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change Ellipse E(u,v) = const direction of the slowest change (max)-1/2 (min)-1/2 Credit: R. Szeliski

28 Selecting Good Features
Image patch SSD surface l1 and l2 both large Credit: C. Dyer

29 Selecting Good Features
SSD surface large l1, small l2 Credit: C. Dyer

30 Selecting Good Features
SSD surface small l1, small l2 Credit: C. Dyer

31 Harris Corner Detector
2 “Edge” 2 >> 1 “Corner” 1 and 2 both large, 1 ~ 2; E increases in all directions Classification of image points using eigenvalues of M: 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1 Credit: C. Dyer

32 Harris Corner Detector
Measure of corner response: k is an empirically-determined constant; e.g., k = 0.05 Credit: C. Dyer

33 Harris Corner Detector
2 “Edge” “Corner” R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge |R| is small for a flat region R < 0 R > 0 “Flat” “Edge” |R| small R < 0 1 Credit: C. Dyer

34 Harris Corner Detector: Algorithm
Find points with large corner response function R (i.e., R > threshold) Take the points of local maxima of R (for localization) by non-maximum suppression Credit: C. Dyer

35 Harris Detector: Example
Credit: C. Dyer

36 Harris Detector: Example
Compute corner response R = 12 – k(1 + 2)2 Credit: C. Dyer

37 Harris Detector: Example
Find points with large corner response: R > threshold Credit: C. Dyer

38 Harris Detector: Example
Take only the points of local maxima of R Credit: C. Dyer

39 Harris Detector: Example
Credit: C. Dyer

40 Harris Detector: Example
Interest points extracted with Harris (~ 500 points) Credit: C. Dyer

41 Harris Detector: Example
Credit: C. Dyer

42 Harris Detector: Summary
Average intensity change in direction [u,v] can be expressed in bilinear form: Describe a point in terms of eigenvalues of M: measure of corner response: A good (corner) point should have a large intensity change in all directions, i.e., R should be a large positive value Credit: C. Dyer

43 Harris Detector Properties
Rotation invariance Ellipse rotates but its shape (i.e., eigenvalues) remains the same Corner response R is invariant to image rotation Credit: C. Dyer

44 Harris Detector Properties
But not invariant to image scale Fine scale: All points will be classified as edges Coarse scale: Corner Credit: C. Dyer

45 Harris Detector Properties
Quality of Harris detector for different scale changes Repeatability rate: # correct correspondences # possible correspondences C. Schmid et al., “Evaluation of Interest Point Detectors,” IJCV 2000 Credit: C. Dyer

46 Invariant Local Features
Goal: Detect the same interest points regardless of image changes due to translation, rotation, scale, viewpoint

47 Models of Image Change Geometry Photometry Rotation
Similarity (rotation + uniform scale) Affine (scale dependent on direction) valid for: orthographic camera, locally planar object Photometry Affine intensity change (I  a I + b) Credit: C. Dyer

48 SIFT Detector [Lowe ’04] Difference-of-Gaussian (DoG) is an approximation of the Laplacian-of-Gaussian (LoG) = Lowe, D. G., “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, 60, 2, pp , 2004 Credit: C. Dyer

49 SIFT Detector Credit: C. Dyer

50 SIFT Detector Credit: C. Dyer

51 SIFT Detector Algorithm Summary
Detect local maxima in position and scale of squared values of difference-of-Gaussian Fit a quadratic to surrounding values for sub-pixel and sub-scale interpolation Output = list of (x, y, ) points Credit: C. Dyer

52 References on Feature Descriptors
A performance evaluation of local descriptors, K. Mikolajczyk and C. Schmid, IEEE Trans. PAMI 27(10), 2005 Evaluation of features detectors and descriptors based on 3D objects, P. Moreels and P. Perona, Int. J. Computer Vision 73(3), 2007 Credit: C. Dyer

53 Next Time Panorama Feature matching

Download ppt "Computational Photography"

Similar presentations

Ads by Google