Computer Vision : CISC4/689 Computer Graphics Output Image Model Synthetic Camera (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

Computer Vision : CISC4/689 Output Model Real Scene Real Cameras (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

Computer Vision : CISC4/689 Combined Output Image Model Real Scene Synthetic Camera Real Cameras (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

Computer Vision : CISC4/689 Pinhole cameras Abstract camera model - box with a small hole in it Pinhole cameras work in practice The point to make here is that each point on the image plane sees light from only one direction, the one that passes through the pinhole. Computer Vision : CISC4/689

Distant objects are smaller Computer Vision : CISC4/689

Consequences: Parallel lines meet There exist vanishing points Marc Pollefeys Computer Vision : CISC4/689

The Effect of Perspective Computer Vision : CISC4/689

Computer Vision : CISC4/689 Vanishing points H VPL VPR VP1 VP2 Different directions correspond to different vanishing points VP3 Marc Pollefeys Computer Vision : CISC4/689

Computer Vision : CISC4/689 Vanishing points each set of parallel lines (=direction) meets at a different point The vanishing point for this direction Sets of parallel lines on the same plane lead to collinear vanishing points. The line is called the horizon for that plane Good ways to spot faked images scale and perspective don’t work vanishing points behave badly supermarket tabloids are a great source. Computer Vision : CISC4/689

Computer Vision : CISC4/689 Slide credit: David Jacobs

Properties of Projection Points project to points Lines project to lines Vanishing points for parallel lines Parallel lines parallel to image plane donot converge Closer objects appear bigger Angles are not preserved Degenerate cases Line through focal point projects to a point. Plane through focal point projects to line Computer Vision : CISC4/689

Pinhole Camera Terminology Image plane Optical axis Principal point/ image center Focal length Camera center/ pinhole Camera point Image point Computer Vision : CISC4/689

The equation of projection Computer Vision : CISC4/689

The equation of projection Cartesian coordinates: We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f) Ignore the third coordinate, and get Computer Vision : CISC4/689

Computer Vision : CISC4/689 The camera matrix Turn previous expression into HC’s HC’s for 3D point are (X,Y,Z,T) HC’s for point in image are (U,V,W) Computer Vision : CISC4/689

Computer Vision : CISC4/689 Weak perspective Issue perspective effects, but not over the scale of individual objects collect points into a group at about the same depth, then divide each point by the depth of its group Adv: easy Disadv: wrong Computer Vision : CISC4/689

Weak Perspective Projection Z O -x Z Z f Computer Vision : CISC4/689

The Equation of Weak Perspective (scaled Orthographic) s is constant for all points. Parallel lines no longer converge, they remain parallel. Computer Vision : CISC4/689 Slide credit: David Jacobs

Computer Vision : CISC4/689 Generalization of Orthographic Projection When the camera is at a (roughly constant) distance from the scene, take m=1. Computer Vision : CISC4/689 Marc Pollefeys

Computer Vision : CISC4/689 Pictorial Comparison Weak perspective Perspective Computer Vision : CISC4/689  Marc Pollefeys

Summary: Perspective Laws Weak perspective Orthographic Computer Vision : CISC4/689

Pros and Cons of These Models Weak perspective has simpler math. Accurate when object is small and distant. Most useful for recognition. Pinhole perspective much more accurate for scenes. Used in structure from motion. When accuracy really matters, we must model the real camera Use perspective projection with other calibration parameters (e.g., radial lens distortion) Computer Vision : CISC4/689 Slide credit: David Jacobs

The projection matrix for orthographic projection Computer Vision : CISC4/689

Computer Vision : CISC4/689 Affine cameras Computer Vision : CISC4/689

The Image Formation Pipeline Computer Vision : CISC4/689

Computer Vision : CISC4/689 Camera parameters Issue camera may not be at the origin, looking down the z-axis extrinsic parameters one unit in camera coordinates may not be the same as one unit in world coordinates intrinsic parameters - focal length, principal point, aspect ratio, angle between axes, etc. Note the matrix dimensions Computer Vision : CISC4/689

Computer Vision : CISC4/689 Camera calibration Issues: what are intrinsic parameters of the camera? what is the camera matrix? (intrinsic+extrinsic) General strategy: view calibration object identify image points obtain camera matrix by minimizing error obtain intrinsic parameters from camera matrix Error minimization: Linear least squares easy problem numerically solution can be rather bad Minimize image distance more difficult numerical problem solution usually rather good, start with linear least squares Numerical scaling is an issue Computer Vision : CISC4/689

Computer Vision : CISC4/689 Outline Vector, matrix basics 2-D point transformations Translation, scaling, rotation, shear Homogeneous coordinates and transformations Homography, affine transformation Computer Vision : CISC4/689

Computer Vision : CISC4/689 Notes on Notation Vectors, points: x, v (assume column vectors) Matrices: R, T Scalars: x, a Axes, objects: X, Y, O Coordinate systems: W, C Number systems: R, Z Specials Transpose operator: xT (as opposed to x0) Identity matrix: Id Matrices/vectors of zeroes, ones: 0, 1 Computer Vision : CISC4/689

Block Notation for Matrices Often convenient to write matrices in terms of parts Smaller matrices for blocks Row, column vectors for ranges of entries on rows, columns, respectively E.g.: If A is 3 x 3 and : Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Transformations Types Scaling Rotation Shear Translation Mathematical representation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling sx 1 Horizontal shift proportional to horizontal position Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling sy 1 Vertical shift proportional to vertical position Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling Computer Vision : CISC4/689

Matrix form of 2-D Scaling Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Scaling Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Rotation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Rotation µ Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Rotation µ Computer Vision : CISC4/689

Matrix form of 2-D Rotation µ (this is a counterclockwise rotation; reverse signs of sines to get a clockwise one) Computer Vision : CISC4/689

Matrix form of 2-D Rotation µ Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Shear (Horizontal) Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Shear (Horizontal) Horizontal displacement proportional to vertical position Computer Vision : CISC4/689

2-D Shear (Horizontal) (Shear factor h is positive for the figure above) Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Shear (Horizontal) Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Shear (Vertical) (v is negative for the figure above) Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Translation Computer Vision : CISC4/689

Representing Transformations Note that we’ve defined translation as a vector addition but rotation, scaling, etc. as matrix multiplications It’s inconvenient to have two different operations (addition and multiplication) for different forms of transformation It would be desirable for all transformations to be expressed in a common, linear form (since matrix multiplications are linear transformations) Computer Vision : CISC4/689

Example: “Trick” of additional coordinate makes this possible Old way: New way: Computer Vision : CISC4/689

Computer Vision : CISC4/689 Translation Matrix We can write the formula using this “expanded” transformation matrix as: From now on, assume points are in this “expanded” form unless otherwise noted: Computer Vision : CISC4/689

Homogeneous Coordinates “Expanded” form is called homogeneous coordinates or projective space Change to projective space by adding a scale factor (usually but not always 1): Computer Vision : CISC4/689

Homogeneous Coordinates: Projective Space Equivalence is defined up to scale ¸ (non-zero for finite points) Think of projective points in P2 as rays in R3, where z coordinate is scale factor All Euclidean points along ray are “same” in this sense Computer Vision : CISC4/689

Leaving Projective Space Can go back to non-homogeneous representation by dividing by scale factor and dropping extra coordinate: This is the same as saying “Where does the ray intersect the plane defined by z = 1”? Analogy to perspective projection, where f=1 (image plane) and lambda is z of any point in the ray. For different lambda’s along the line, projected point is the same, thus Equivalence class Computer Vision : CISC4/689

Homogeneous Coordinates: Rotations, etc. A 2-D rotation, scaling, shear or other transformation normally expressed by a 2 x 2 matrix R is written in homogeneous coordinates with the following 3 x 3 matrix: The non-commutativity of matrix multiplication explains why different transformation orders give different results—i.e., RT  TR In homogeneous form In homogeneous form Computer Vision : CISC4/689

Example: Transformations Don’t Commute Computer Vision : CISC4/689

Example: Transformations Don’t Commute Computer Vision : CISC4/689

Example: Transformations Don’t Commute Computer Vision : CISC4/689

Example: Transformations Don’t Commute Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography Computer Vision : CISC4/689

2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography Translation components Computer Vision : CISC4/689

2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography Scale/rotation components Computer Vision : CISC4/689

2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography Shear/rotation components Shear/rotation off-diagonal (more about relationship between shearing and rotation at http://splorg.org/people/tobin/projects/israel/projects/paeth/rotation_by_shearing.html) Computer Vision : CISC4/689

2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography Homogeneous scaling factor Computer Vision : CISC4/689

Computer Vision : CISC4/689 2-D Transformations Full-generality 3 x 3 homogeneous transformation is called a homography When these are zero (as they have been so far), H is an affine transformation Computer Vision : CISC4/689