Advanced Computer Vision Structure from Motion

Structure from Motion Geometric structure-from-motion problem: using image matches to estimate: The 3D positions of the corresponding scene points in some fixed coordinate (the scene structure) The projection matrices associated with the camera observing them (the motion of the points relative to the cameras)

Two Types Affine structure from motion Projective structure from motion

Basic Assumption The cameras’ positions and possibly their intrinsic parameters are a priori unknown and may change over time. Different from, say, stereopsis, where the camera parameters are known so we can focus on the correspondence problem. Ignore the correspondence problem, assuming that the projections of n points have been matched across m pictures.

Applications Image-based rendering: a video clip recorded by a hand-held camcorder, possibly zooming during the shoot, is used to capture the shape of an object and render it under new viewing conditions. Active vision system: robot probes.

Affine Structure from Motion Problem: estimate the m 2x4 matrices Mi and the n positions Pj from the mxn correspondences pij . i j ij 2mn equations in 8m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares!

Ambiguity of Solution If M and P are solutions, So are M’ and P’ where j So are M’ and P’ where i j and Q is an affine transformation.

The Projective Structure-from-Motion Problem Given m perspective images of n fixed points P we can write j Problem: estimate the m 3x4 matrices M and the n positions P from the mn correspondences p . i j ij 2mn equations in 11m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares!