Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2006 Lecture 4 Camera Calibration Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) What are Intrinsic Parameters?

Sebastian Thrun CS223B Computer Vision, Winter Perspective Projection, Remember? fZ X O -x

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) Intrinsic Parameters: –Focal Length f –Pixel size s x, s y –Distortion coefficients k 1, k 2 … –Image center o x, o y

Sebastian Thrun CS223B Computer Vision, Winter A Quiz Can we determine all intrinsic parameters by … exposing the camera to many known objects?

Sebastian Thrun CS223B Computer Vision, Winter Example Calibration Pattern

Sebastian Thrun CS223B Computer Vision, Winter Our Calibration target

Sebastian Thrun CS223B Computer Vision, Winter Harris Corner Detector

Sebastian Thrun CS223B Computer Vision, Winter Another Quiz (the last today) How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20

Sebastian Thrun CS223B Computer Vision, Winter Experiment 1: Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm Projective Perspective of Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter Experiment 2: Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm 500cm50cm100cm Projective Perspective of Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter Perspective Camera Model Object Space

Sebastian Thrun CS223B Computer Vision, Winter Calibration: 2 steps Step 1: Transform into camera coordinates Step 2: Transform into image coordinates

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (extrinsic) Homogeneous Coordinates

Sebastian Thrun CS223B Computer Vision, Winter Homogeneous Coordinates Idea: Most Operations Become Linear! Extract Image Coordinates by Z- normalization

Sebastian Thrun CS223B Computer Vision, Winter Advantage of Homogeneous C’s i-th data point

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (intrinsic) Pixel size Focal length Image center

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Transformation

Sebastian Thrun CS223B Computer Vision, Winter Plugging the Model Together!

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients)

Sebastian Thrun CS223B Computer Vision, Winter Q: Can We recover all Intrinsic Params? No

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters, Revisited Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients) Focal length, in pixel units Aspect ratio

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration via SVD

Sebastian Thrun CS223B Computer Vision, Winter Calibration via SVD N>=7 points, not coplanar

Sebastian Thrun CS223B Computer Vision, Winter Calibration via SVD

Sebastian Thrun CS223B Computer Vision, Winter Calibration via SVD A has rank 7 (without proof)

Sebastian Thrun CS223B Computer Vision, Winter Calibration via SVD Remaining Problem: See book

Sebastian Thrun CS223B Computer Vision, Winter Summary, SVD Solution Replace rotation matrix by arbitrary matrix Transform into linear set of equations Solve via SVD Enforce rotation matrix (see book) Solve for remaining parameters (see book) SVD solution: algebraic minimization, assume Gaussian noise in parameter space

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Calibration Examples: …

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Squares

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Mean Square Gradient descent:

Sebastian Thrun CS223B Computer Vision, Winter Summary Non-Linear Least Squares Solve nonlinear equations via gradient descent Assume Gaussian noise in image space, not parameter space

Sebastian Thrun CS223B Computer Vision, Winter SVD Versus LQ SVD Minimization of squared distance in parameter space Globally optimal Nonlin Least Squares Minimization of squared distance in Image space Locally optimal

Sebastian Thrun CS223B Computer Vision, Winter Q: How Many Images Do We Need? Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM  4+6K  M>3 and K  2/(M-3) 2 images with 4 points, but will 1 images with 5 points work? No, since points cannot be co-planar!

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

Sebastian Thrun CS223B Computer Vision, Winter Advanced Calibration: Nonlinear Distortions Barrel and Pincushion Tangential

Sebastian Thrun CS223B Computer Vision, Winter Barrel and Pincushion Distortion telewideangle

Sebastian Thrun CS223B Computer Vision, Winter Models of Radial Distortion distance from center

Sebastian Thrun CS223B Computer Vision, Winter Tangential Distortion cheap glue cheap CMOS chip cheap lense image cheap camera

Sebastian Thrun CS223B Computer Vision, Winter Image Rectification (to be continued)

Sebastian Thrun CS223B Computer Vision, Winter Summary Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion