Download presentation

Presentation is loading. Please wait.

1
**Multiple View Geometry**

Marc Pollefeys COMP 256

2
**Last class Gaussian pyramid Laplacian pyramid Gabor Fourier filters**

transform

3
**Histograms : co-occurrence matrix**

Not last class … Histograms : co-occurrence matrix

4
**Texture synthesis [Zalesny & Van Gool 2000]**

2 analysis iterations 6 analysis iterations 9 analysis iterations

5
**View-dependent texture synthesis [Zalesny & Van Gool 2000]**

6
**Efros & Leung ’99 p Assuming Markov property, compute P(p|N(p))**

non-parametric sampling Synthesizing a pixel Input image Assuming Markov property, compute P(p|N(p)) Building explicit probability tables infeasible Instead, let’s search the input image for all similar neighborhoods — that’s our histogram for p To synthesize p, just pick one match at random

7
**Efros & Leung ’99 extended**

p B Idea: unit of synthesis = block Exactly the same but now we want P(B|N(B)) Much faster: synthesize all pixels in a block at once Not the same as multi-scale! Synthesizing a block non-parametric sampling Input image Observation: neighbor pixels are highly correlated

8
**constrained by overlap**

block Input texture B1 B2 Neighboring blocks constrained by overlap B1 B2 Minimal error boundary cut B1 B2 Random placement of blocks

9
**Minimal error boundary**

overlapping blocks vertical boundary _ = 2 overlap error min. error boundary

13
**Why do we see more flowers in the distance?**

[Leung & Malik CVPR97] Perpendicular textures

14
Shape-from-texture

15
**Tentative class schedule**

Jan 16/18 - Introduction Jan 23/25 Cameras Radiometry Jan 30/Feb1 Sources & Shadows Color Feb 6/8 Linear filters & edges Texture Feb 13/15 Multi-View Geometry Stereo Feb 20/22 Optical flow Project proposals Feb27/Mar1 Affine SfM Projective SfM Mar 6/8 Camera Calibration Silhouettes and Photoconsistency Mar 13/15 Springbreak Mar 20/22 Segmentation Fitting Mar 27/29 Prob. Segmentation Project Update Apr 3/5 Tracking Apr 10/12 Object Recognition Apr 17/19 Range data Apr 24/26 Final project

16
**THE GEOMETRY OF MULTIPLE VIEWS**

Epipolar Geometry The Essential Matrix The Fundamental Matrix The Trifocal Tensor The Quadrifocal Tensor Reading: Chapter 10.

17
Epipolar Geometry Epipolar Plane Baseline Epipoles Epipolar Lines

18
**Potential matches for p have to lie on the corresponding**

Epipolar Constraint Potential matches for p have to lie on the corresponding epipolar line l’. Potential matches for p’ have to lie on the corresponding epipolar line l.

19
**Epipolar Constraint: Calibrated Case**

Essential Matrix (Longuet-Higgins, 1981)

20
**E p’ is the epipolar line associated with p’.**

Properties of the Essential Matrix T E p’ is the epipolar line associated with p’. ETp is the epipolar line associated with p. E e’=0 and ETe=0. E is singular. E has two equal non-zero singular values (Huang and Faugeras, 1989). T

21
**Epipolar Constraint: Small Motions**

To First-Order: Pure translation: Focus of Expansion

22
**Epipolar Constraint: Uncalibrated Case**

Fundamental Matrix (Faugeras and Luong, 1992)

23
**Properties of the Fundamental Matrix**

F p’ is the epipolar line associated with p’. FT p is the epipolar line associated with p. F e’=0 and FT e=0. F is singular. T T

24
**The Eight-Point Algorithm (Longuet-Higgins, 1981)**

|F| =1. Minimize: under the constraint 2

25
**Non-Linear Least-Squares Approach**

(Luong et al., 1993) Minimize with respect to the coefficients of F , using an appropriate rank-2 parameterization.

26
**Problem with eight-point algorithm**

linear least-squares: unit norm vector F yielding smallest residual What happens when there is noise?

27
**The Normalized Eight-Point Algorithm (Hartley, 1995)**

Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’. Use the eight-point algorithm to compute F from the points q and q’ . Enforce the rank-2 constraint. Output T F T’. i i i i i i T

28
**Epipolar geometry example**

29
**courtesy of Andrew Zisserman**

Example: converging cameras courtesy of Andrew Zisserman

30
**Example: motion parallel with image plane**

(simple for stereo rectification) courtesy of Andrew Zisserman

31
**courtesy of Andrew Zisserman**

Example: forward motion e’ e courtesy of Andrew Zisserman

32
**courtesy of Andrew Zisserman**

Fundamental matrix for pure translation auto-epipolar courtesy of Andrew Zisserman

33
**courtesy of Andrew Zisserman**

Fundamental matrix for pure translation courtesy of Andrew Zisserman

34
**Trinocular Epipolar Constraints**

These constraints are not independent!

35
**Trinocular Epipolar Constraints: Transfer**

Given p and p , p can be computed as the solution of linear equations. 1 2 3

36
**Trinocular Epipolar Constraints: Transfer**

problem for epipolar transfer in trifocal plane! There must be more to trifocal geometry… image from Hartley and Zisserman

37
Trifocal Constraints

38
**Trifocal Constraints Calibrated Case All 3x3 minors must be zero!**

Trifocal Tensor

39
Trifocal Constraints Uncalibrated Case Trifocal Tensor

40
**Trifocal Constraints: 3 Points**

Pick any two lines l and l through p and p . 2 3 2 3 Do it again. T( p , p , p )=0 1 2 3

41
**For any matching epipolar lines, l G l = 0.**

Properties of the Trifocal Tensor For any matching epipolar lines, l G l = 0. The matrices G are singular. They satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995). T i 2 1 3 i 1 Estimating the Trifocal Tensor Ignore the non-linear constraints and use linear least-squares Impose the constraints a posteriori.

42
**For any matching epipolar lines, l G l = 0.**

2 1 3 The backprojections of the two lines do not define a line!

43
**courtesy of Andrew Zisserman**

Trifocal Tensor Example 108 putative matches 18 outliers (26 samples) 88 inliers 95 final inliers (0.43) (0.23) (0.19) courtesy of Andrew Zisserman

44
**Trifocal Tensor Example**

additional line matches images courtesy of Andrew Zisserman

45
**Transfer: trifocal transfer**

(using tensor notation) doesn’t work if l’=epipolar line image courtesy of Hartley and Zisserman

46
**Image warping using T(1,2,N)**

(Avidan and Shashua `97)

47
**Multiple Views (Faugeras and Mourrain, 1995)**

48
Two Views Epipolar Constraint

49
Three Views Trifocal Constraint

50
Four Views Quadrifocal Constraint (Triggs, 1995)

51
**Geometrically, the four rays must intersect in P..**

52
Quadrifocal Tensor and Lines

53
**Quadrifocal tensor determinant is multilinear**

thus linear in coefficients of lines ! There must exist a tensor with 81 coefficients containing all possible combination of x,y,w coefficients for all 4 images: the quadrifocal tensor

54
**Scale-Restraint Condition from Photogrammetry**

55
**from perspective to omnidirectional cameras**

3 constraints allow to reconstruct 3D point perspective camera (2 constraints / feature) more constraints also tell something about cameras l=(y,-x) (x,y) (0,0) multilinear constraints known as epipolar, trifocal and quadrifocal constraints radial camera (uncalibrated) (1 constraints / feature)

56
**Radial quadrifocal tensor**

(x,y) Radial quadrifocal tensor Linearly compute radial quadrifocal tensor Qijkl from 15 pts in 4 views Reconstruct 3D scene and use it for calibration (2x2x2x2 tensor) Not easy for real data, hard to avoid degenerate cases (e.g. 3 optical axes intersect in single point). However, degenerate case leads to simpler 3 view algorithm for pure rotation Radial trifocal tensor Tijk from 7 points in 3 views Reconstruct 2D panorama and use it for calibration (2x2x2 tensor)

57
**Non-parametric distortion calibration**

(Thirthala and Pollefeys, ICCV’05) Models fish-eye lenses, cata-dioptric systems, etc. angle normalized radius

58
**Non-parametric distortion calibration**

(Thirthala and Pollefeys, ICCV’05) Models fish-eye lenses, cata-dioptric systems, etc. 90o angle normalized radius

59
**Next class: Stereo (x´,y´)=(x+D(x,y),y) F&P Chapter 11 image I´(x´,y´)**

Disparity map D(x,y) image I´(x´,y´) (x´,y´)=(x+D(x,y),y) F&P Chapter 11

Similar presentations

OK

Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)

Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on mammals and egg laying animals pictures Ppt on animal food habits Ppt on self discipline for students Ppt on resources and development Ppt on earth and space Ppt on bullet train Ppt on tourism in delhi and its promotion Ppt on company law in india Ppt on first conditional questions Free ppt on swine flu