Computer Vision 6 Efros & Leung 99 Assuming Markov property, compute P(p|N(p)) –Building explicit probability tables infeasible –Instead, lets search the input image for all similar neighborhoods thats our histogram for p To synthesize p, just pick one match at random p non-parametric sampling Input image Synthesizing a pixel
Computer Vision 7 p Efros & Leung 99 extended Observation: neighbor pixels are highly correlated Input image non-parametric sampling B Idea: unit of synthesis = block Exactly the same but now we want P(B|N(B)) Much faster: synthesize all pixels in a block at once Not the same as multi-scale! Synthesizing a block
Computer Vision 8 Input texture B1B2 Random placement of blocks block B1 B2 Neighboring blocks constrained by overlap B1B2 Minimal error boundary cut
Computer Vision 15 Jan 16/18-Introduction Jan 23/25CamerasRadiometry Jan 30/Feb1Sources & ShadowsColor Feb 6/8Linear filters & edgesTexture Feb 13/15Multi-View GeometryStereo Feb 20/22Optical flowProject proposals Feb27/Mar1Affine SfMProjective SfM Mar 6/8Camera CalibrationSilhouettes and Photoconsistency Mar 13/15Springbreak Mar 20/22SegmentationFitting Mar 27/29Prob. SegmentationProject Update Apr 3/5Tracking Apr 10/12Object Recognition Apr 17/19Range data Apr 24/26Final project Tentative class schedule
Computer Vision 16 THE GEOMETRY OF MULTIPLE VIEWS Reading: Chapter 10. Epipolar Geometry The Essential Matrix The Fundamental Matrix The Trifocal Tensor The Quadrifocal Tensor
Computer Vision 20 Properties of the Essential Matrix E p is the epipolar line associated with p. E T p is the epipolar line associated with p. E e=0 and E T e=0. E is singular. E has two equal non-zero singular values (Huang and Faugeras, 1989). T T
Computer Vision 21 Epipolar Constraint: Small Motions To First-Order: Pure translation: Focus of Expansion
Computer Vision 22 Epipolar Constraint: Uncalibrated Case Fundamental Matrix (Faugeras and Luong, 1992)
Computer Vision 23 Properties of the Fundamental Matrix F p is the epipolar line associated with p. F T p is the epipolar line associated with p. F e=0 and F T e=0. F is singular. T T
Computer Vision 24 The Eight-Point Algorithm (Longuet-Higgins, 1981) | F | =1. Minimize: under the constraint 2
Computer Vision 25 Non-Linear Least-Squares Approach (Luong et al., 1993) Minimize with respect to the coefficients of F, using an appropriate rank-2 parameterization.
Computer Vision 26 Problem with eight-point algorithm linear least-squares: unit norm vector F yielding smallest residual What happens when there is noise?
Computer Vision 27 The Normalized Eight-Point Algorithm (Hartley, 1995) Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q = T p, q = T p. Use the eight-point algorithm to compute F from the points q and q. Enforce the rank-2 constraint. Output T F T. T iiii ii
Computer Vision 38 Trifocal Constraints All 3x3 minors must be zero! Calibrated Case Trifocal Tensor
Computer Vision 39 Trifocal Constraints Uncalibrated Case Trifocal Tensor
Computer Vision 40 Trifocal Constraints: 3 Points Pick any two lines l and l through p and p. Do it again. 2323 T( p, p, p )=0 1 2 3
Computer Vision 41 Properties of the Trifocal Tensor Estimating the Trifocal Tensor Ignore the non-linear constraints and use linear least-squares Impose the constraints a posteriori. For any matching epipolar lines, l G l = 0. The matrices G are singular. They satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995). 213 Ti 1 i
Computer Vision 42 For any matching epipolar lines, l G l = 0. 213 Ti The backprojections of the two lines do not define a line!
Computer Vision 43 108 putative matches 18 outliers 88 inliers 95 final inliers (26 samples) (0.43) (0.23) (0.19) Trifocal Tensor Example courtesy of Andrew Zisserman
Computer Vision 44 additional line matches Trifocal Tensor Example images courtesy of Andrew Zisserman
Computer Vision 45 Transfer: trifocal transfer doesnt work if l=epipolar line image courtesy of Hartley and Zisserman (using tensor notation)
Computer Vision 46 Image warping using T(1,2,N) (Avidan and Shashua `97)
Computer Vision 47 Multiple Views (Faugeras and Mourrain, 1995)
Computer Vision 48 Two Views Epipolar Constraint
Computer Vision 49 Three Views Trifocal Constraint
Computer Vision 50 Four Views Quadrifocal Constraint (Triggs, 1995)
Computer Vision 51 Geometrically, the four rays must intersect in P..
Computer Vision 52 Quadrifocal Tensor and Lines
Computer Vision 53 Quadrifocal tensor determinant is multilinear thus linear in coefficients of lines ! There must exist a tensor with 81 coefficients containing all possible combination of x,y,w coefficients for all 4 images: the quadrifocal tensor
Computer Vision 54 Scale-Restraint Condition from Photogrammetry
Computer Vision 55 from perspective to omnidirectional cameras perspective camera (2 constraints / feature) radial camera (uncalibrated) (1 constraints / feature) 3 constraints allow to reconstruct 3D point more constraints also tell something about cameras multilinear constraints known as epipolar, trifocal and quadrifocal constraints (0,0) l =(y,-x) (x,y)
Computer Vision 56 Radial quadrifocal tensor Linearly compute radial quadrifocal tensor Q ijkl from 15 pts in 4 views Reconstruct 3D scene and use it for calibration (2x2x2x2 tensor) (2x2x2 tensor) Not easy for real data, hard to avoid degenerate cases (e.g. 3 optical axes intersect in single point). However, degenerate case leads to simpler 3 view algorithm for pure rotation Radial trifocal tensor T ijk from 7 points in 3 views Reconstruct 2D panorama and use it for calibration (x,y)
Computer Vision 57 Non-parametric distortion calibration (Thirthala and Pollefeys, ICCV05) normalized radius angle Models fish-eye lenses, cata-dioptric systems, etc.
Computer Vision 58 Non-parametric distortion calibration (Thirthala and Pollefeys, ICCV05) normalized radius angle 90 o Models fish-eye lenses, cata-dioptric systems, etc.