1. Lecture 03 15/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

2. Perspective Projection Direct Linear Transformation Traiangulation

3. PERSPECTIVE PROJECTION

4. Perspective projection Scene point Image coordinates Camera frame Image plane Optical axis Focal length Thus far, in camera’s reference frame only.

5. Extrinsic: location and orientation of camera frame with respect to reference frame Intrinsic: how to map pixel coordinates to image plane coordinates Camera parameters Camera 1 frame Reference frame

6. Extrinsic camera parameters World reference frame Camera reference frame

7. Extrinsic: location and orientation of camera frame with respect to reference frame Intrinsic: how to map pixel coordinates to image plane coordinates Camera parameters Camera 1 frame Reference frame

8. Same thing, but written in terms of homogeneous coordinates From pinhole camera model Projection matrix for perspective projection

9. Intrinsic camera parameters Ignoring any geometric distortions from optics, we can describe them by: Coordinates of projected point in camera reference frame Coordinates of image point in pixel units Coordinates of image center in pixel units Effective size of a pixel (mm)

10. Camera parameters We know that in terms of camera reference frame: Substituting previous eqns describing intrinsic and extrinsic parameters, can relate pixels coordinates to world points: R i = Row i of rotation matrix and

11. Projection matrix This can be rewritten as a matrix product using homogeneous coordinates: The motion of the camera is equivalent to the inverse motion of the scene wx im wy im w = K[R t] XwYwZw1XwYwZw1 K=K= point in camera coordinates External parameters Internal parameters

12. Calibrating a camera Compute intrinsic and extrinsic parameters using observed camera data Main idea Place “calibration object” with known geometry in the scene Get correspondences Solve for mapping from scene to image: estimate M=K[R t]

13. This can be rewritten as a matrix product using homogeneous coordinates: = K[R t] XwYwZw1XwYwZw1 Let M i be row i of matrix M product M is single projection matrix encoding both extrinsic and intrinsic parameters P w in homog. Projection matrix wx im wy im w

14. This can be rewritten as a matrix product using homogeneous coordinates: = K[R t] XwYwZw1XwYwZw1 Let M i be row i of matrix M product M is single projection matrix encoding both extrinsic and intrinsic parameters P w in homog. Projection matrix wx im wy im w

15. Estimating the projection matrix For a given feature point

16. Estimating the projection matrix Expanding this first equation, we have:

17. Estimating the projection matrix

18. This is true for every feature point, so we can stack up n observed image features and their associated 3d points in single equation: Solve for m ij ’s (the calibration information) [F&P Section 3.1] ………… … P m Pm = 0

19. Summary: camera calibration Associate image points with scene points on object with known geometry Use together with perspective projection relationship to estimate projection matrix (Can also solve for explicit parameters themselves)

20. DIRECT LINEAR TRANSFORMATION

21. Direct Linear Transformation from 3D to 2D Given two cameras (a stereo pair) and a 3D object with known 3D coordinates we can: 1)For each camera compute the direct linear transformation from 3D to 2D Now we can move the stereo pair to a different place, take a pair of pictures and recover 3D information: 1)Given two camera matrices and a pair of matching points, we can triangulate to get its depth. 2)We’ll talk about triangulation later in class

22. When would we calibrate this way? Makes sense when geometry of system is not going to change over time …When would it change?

23. TRIANGULATION

24. 24 Triangulation Problem: Given some points in correspondence across two or more images (taken from calibrated cameras), {(u j,v j )}, compute the 3D location X

25. 25 Triangulation Method I: intersect viewing rays in 3D, minimize: – X is the unknown 3D point – C j is the optical center of camera j – V j is the viewing ray for pixel (u j,v j ) – s j is unknown distance along V j Advantage: geometrically intuitive CjCj VjVj X

26. 26 Triangulation Method II: solve linear equations in X – advantage: very simple Method III: non-linear minimization – advantage: most accurate (image plane error)