5View-dependent texture synthesis [Zalesny & Van Gool 2000]
6Efros & Leung ’99 p Assuming Markov property, compute P(p|N(p)) non-parametricsamplingSynthesizing a pixelInput imageAssuming Markov property, compute P(p|N(p))Building explicit probability tables infeasibleInstead, let’s search the input image for all similar neighborhoods — that’s our histogram for pTo synthesize p, just pick one match at random
7Efros & Leung ’99 extended pBIdea: unit of synthesis = blockExactly the same but now we want P(B|N(B))Much faster: synthesize all pixels in a block at onceNot the same as multi-scale!Synthesizing a blocknon-parametricsamplingInput imageObservation: neighbor pixels are highly correlated
8constrained by overlap blockInput textureB1B2Neighboring blocksconstrained by overlapB1B2Minimal errorboundary cutB1B2Random placementof blocks
18Potential matches for p have to lie on the corresponding Epipolar ConstraintPotential matches for p have to lie on the correspondingepipolar line l’.Potential matches for p’ have to lie on the correspondingepipolar line l.
19Epipolar Constraint: Calibrated Case Essential Matrix(Longuet-Higgins, 1981)
20E p’ is the epipolar line associated with p’. Properties of the Essential MatrixTE p’ is the epipolar line associated with p’.ETp is the epipolar line associated with p.E e’=0 and ETe=0.E is singular.E has two equal non-zero singular values(Huang and Faugeras, 1989).T
21Epipolar Constraint: Small Motions To First-Order:Pure translation:Focus of Expansion
22Epipolar Constraint: Uncalibrated Case Fundamental Matrix(Faugeras and Luong, 1992)
23Properties of the Fundamental Matrix F p’ is the epipolar line associated with p’.FT p is the epipolar line associated with p.F e’=0 and FT e=0.F is singular.TT
24The Eight-Point Algorithm (Longuet-Higgins, 1981) |F| =1.Minimize:under the constraint2
25Non-Linear Least-Squares Approach (Luong et al., 1993)Minimizewith respect to the coefficients of F , using anappropriate rank-2 parameterization.
26Problem with eight-point algorithm linear least-squares:unit norm vector F yielding smallest residualWhat happens when there is noise?
27The Normalized Eight-Point Algorithm (Hartley, 1995) Center the image data at the origin, and scale it so themean squared distance between the origin and the datapoints is 2 pixels: q = T p , q’ = T’ p’.Use the eight-point algorithm to compute F from thepoints q and q’ .Enforce the rank-2 constraint.Output T F T’.iiiiiiT
40Trifocal Constraints: 3 Points Pick any two lines l and l through p and p .2323Do it again.T( p , p , p )=0123
41For any matching epipolar lines, l G l = 0. Properties of the Trifocal TensorFor any matching epipolar lines, l G l = 0.The matrices G are singular.They satisfy 8 independent constraints in theuncalibrated case (Faugeras and Mourrain, 1995).Ti213i1Estimating the Trifocal TensorIgnore the non-linear constraints and use linear least-squaresImpose the constraints a posteriori.
42For any matching epipolar lines, l G l = 0. 213The backprojections of the two lines do not define a line!
43courtesy of Andrew Zisserman TrifocalTensorExample108 putative matches18 outliers(26 samples)88 inliers95 final inliers(0.43)(0.23)(0.19)courtesy of Andrew Zisserman
44Trifocal Tensor Example additional line matchesimages courtesy of Andrew Zisserman
45Transfer: trifocal transfer (using tensor notation)doesn’t work if l’=epipolar lineimage courtesy of Hartley and Zisserman
46Image warping using T(1,2,N) (Avidan and Shashua `97)
53Quadrifocal tensor determinant is multilinear thus linear in coefficients of lines !There must exist a tensor with 81 coefficients containing all possible combination of x,y,w coefficients for all 4 images: the quadrifocal tensor
55from perspective to omnidirectional cameras 3 constraints allow to reconstruct 3D pointperspective camera(2 constraints / feature)more constraints also tell something about camerasl=(y,-x)(x,y)(0,0)multilinear constraints known as epipolar, trifocal and quadrifocal constraintsradial camera (uncalibrated)(1 constraints / feature)
56Radial quadrifocal tensor (x,y)Radial quadrifocal tensorLinearly compute radial quadrifocal tensor Qijkl from 15 pts in 4 viewsReconstruct 3D scene and use it for calibration(2x2x2x2 tensor)Not easy for real data, hard to avoid degenerate cases (e.g. 3 optical axes intersect in single point). However, degenerate case leads to simpler 3 view algorithm for pure rotationRadial trifocal tensor Tijk from 7 points in 3 viewsReconstruct 2D panorama and use it for calibration(2x2x2 tensor)