CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections

Pinhole camera model Pinhole model: –Captures pencil of rays – all rays through a single point –The point is called Center of Projection (COP) –The image is formed on the Image Plane –Effective focal length f is distance from COP to Image Plane

Dimensionality Reduction Machine (3D to 2D) 3D world2D image What have we lost? Angles Distances (lengths)

Projection can be tricky… Slide source: Seitz

Projection can be tricky… Slide source: Seitz

Projective Geometry What is lost? Length Which is closer? Who is taller?

Lengths can’t be trusted... B’ C’ A’

…but humans adopt! Müller-Lyer Illusion We don’t make measurements in the image plane

Projective Geometry What is lost? Length Angles Perpendicular? Parallel?

Parallel lines aren’t…

Projective Geometry What is preserved? Straight lines are still straight

Vanishing points and lines Parallel lines in the world intersect in the image at a “vanishing point”

Vanishing points and lines o Vanishing Point o Vanishing Line

Vanishing points and lines Vanishing point Vanishing line Vanishing point Vertical vanishing point (at infinity) Credit: Criminisi

Vanishing points and lines Photo from online Tate collection

Note on estimating vanishing points Use multiple lines for better accuracy … but lines will not intersect at exactly the same point in practice One solution: take mean of intersecting pairs … bad idea! Instead, minimize angular differences

Vanishing objects

Modeling projection The coordinate system –We will use the pin-hole model as an approximation –Put the optical center (Center Of Projection) at the origin –Put the image plane (Projection Plane) in front of the COP Why? –The camera looks down the negative z axis we need this if we want right-handed-coordinates –

Modeling projection Projection equations –Compute intersection with PP of ray from (x,y,z) to COP –Derived using similar triangles (on board) We get the projection by throwing out the last coordinate:

Homogeneous coordinates Is this a linear transformation? Trick: add one more coordinate: homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates no—division by z is nonlinear

Homogeneous coordinates Invariant to scaling Point in Cartesian is ray in Homogeneous Homogeneous Coordinates Cartesian Coordinates

Perspective Projection Projection is a matrix multiply using homogeneous coordinates: divide by third coordinate This is known as perspective projection The matrix is the projection matrix Can also formulate as a 4x4 divide by fourth coordinate

Orthographic Projection Special case of perspective projection –Distance from the COP to the PP is infinite –Also called “parallel projection” –What’s the projection matrix? Image World

Scaled Orthographic Projection Special case of perspective projection –Object dimensions are small compared to distance to camera –Also called “weak perspective” –What’s the projection matrix?

Spherical Projection What if PP is spherical with center at COP? In spherical coordinates, projection is trivial:  d  Note: doesn’t depend on focal length!

Projection matrix x: Image Coordinates: w(u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1) OwOw iwiw kwkw jwjw R,T

K Projection matrix Intrinsic Assumptions Unit aspect ratio Principal point at (0,0) No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Remove assumption: known optical center Intrinsic Assumptions Unit aspect ratio No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Remove assumption: square pixels Intrinsic Assumptions No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Remove assumption: non- skewed pixels Intrinsic AssumptionsExtrinsic Assumptions No rotation Camera at (0,0,0) Note: different books use different notation for parameters

Oriented and Translated Camera OwOw iwiw kwkw jwjw t R

Allow camera translation Intrinsic AssumptionsExtrinsic Assumptions No rotation

3D Rotation of Points, : Rotation around the coordinate axes, counter-clockwise: p p’p’p’p’  y z Slide Credit: Saverese

Allow camera rotation

Degrees of freedom 56

Vanishing Point = Projection from Infinity

Lens Flaws

Lens Flaws: Chromatic Aberration Dispersion: wavelength-dependent refractive index –(enables prism to spread white light beam into rainbow) Modifies ray-bending and lens focal length: f(λ) color fringes near edges of image Corrections: add ‘doublet’ lens of flint glass, etc.

Chromatic Aberration Near Lens Center Near Lens Outer Edge

Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’) straight lines curve around the image center

Radial Distortion Radial distortion of the image –Caused by imperfect lenses –Deviations are most noticeable for rays that pass through the edge of the lens No distortionPin cushionBarrel

Radial Distortion

Slide Credits This set of sides also contains contributions kindly made available by the following authors – Alexei Efros – Stephen E. Palmer – Steve Seitz – Derek Hoiem – David Forsyth – George Bebis – Silvio Savarese