1. 29 April 2008Birkbeck College, U. London1 Geometric Model Acquisition Steve Maybank School of Computer Science and Information Systems Birkbeck College London, WC1E 7HX Edited version of the slides for the VVG Summer School, held at the University of Bath 21 September 2007

2. 29 April 2008Birkbeck College, U. London2 Geometric Model Acquisition Aim: make a 3D model of a scene from two or more images taken from different viewpoints. Why is it possible: the image differences depend in part on the shapes of the objects in the scene.

3. 29 April 2008Birkbeck College, U. London3 Two Images of the Same Scene http://vasc.ri.cmu.edu/idb/images/stereo/fruit SOURCE "University of Illinois, Bill Hoff“ DESCRIPTION "Fruit on table, digitized from 35mm."

4. 29 April 2008Birkbeck College, U. London4 Two Images of a Point in R 3 p c 1 c 2 q 1 q 2 Epipolar plane: Image 1 Image 2 object point optical centre optical centre

5. 29 April 2008Birkbeck College, U. London5 Corresponding Points Points in different images correspond, if they are projections of the same scene point p. In projective coordinates, projection is a matrix application,

6. 29 April 2008Birkbeck College, U. London6 Method for Finding Corresponding Points

7. 29 April 2008Birkbeck College, U. London7 Example 1 of Correlation Based Matching Points in lh image =(150,100), (250,150), (350,250), (450,350), (250,450) Correlations (ρ) = 0.750, 0.685, 0.912, 0.644, 0.691 Search area = (2d+1)x(2d+1) box, d=20.

8. 29 April 2008Birkbeck College, U. London8 What Do We Need for GAM? Description of image formation in the camera. Description of the relative positions of the cameras. Equations involving the measurements, the scene points and the relative positions of the cameras. Statistical description of the errors in the measurements.

9. 29 April 2008Birkbeck College, U. London9 Pinhole Camera Light tight box Small hole (optical centre) Viewing screen (image) Object Light rays Central perspective projection model for image formation (Brunelleschi, 15 th C.).

10. 29 April 2008Birkbeck College, U. London10 Camera Coordinate Frame (X,Y,Z) (0,0,0) (0,0,-f) Origin (0,0,0) at the pin hole. Focal length of the camera = f. Axes of image coordinate frame are parallel to X, Y axes of the CCF. Image point = (-Xf/Z, -Yf/Z) Z Y Xx y

11. 29 April 2008Birkbeck College, U. London11 Mathematical Version of the Camera Coordinate Frame (X,Y,Z) (0,0,0) (0,0,f) Origin (0,0,0) at the pin hole. Focal length of the camera = f. The image is in front of the pin hole! Image point = (Xf/Z, Yf/Z). The minus signs have gone. Z y xX Y Image plane

12. 29 April 2008Birkbeck College, U. London12 Relative Position of the Cameras The relative position of the cameras is described by an orthogonal matrix R and a translation vector t. R, t

13. 29 April 2008Birkbeck College, U. London13 Transformation of Coordinates If a point p has coordinates (X,Y,Z) T in the first CCF, then in the second CCF the same point p has coordinates R, t ● p

14. 29 April 2008Birkbeck College, U. London14 Properties of Orthogonal Matrices

15. 29 April 2008Birkbeck College, U. London15 Projection Ray Y X Z ● Any scene point projecting to (x, y, f) T is on the projection ray. CCF

16. 29 April 2008Birkbeck College, U. London16 Projection Rays of Corresponding Points 1 ● The projection rays of corresponding points intersect at a scene point. Geometric model acquisition is based on this single constraint. For an extreme example, see http://www.wisdom.weizmann.ac.il/~vision/VideoAnalysis/Demos/Traj2Traj/hall.htm

17. 29 April 2008Birkbeck College, U. London17 Projection Rays of Corresponding Points 2 ● The equations of the projection rays are known, but they hold in different coordinate systems.

18. 29 April 2008Birkbeck College, U. London18 Transformation of Coordinates

19. 29 April 2008Birkbeck College, U. London19 The Essential Matrix

20. 29 April 2008Birkbeck College, U. London20 Model Acquisition

21. 29 April 2008Birkbeck College, U. London21 Naïve Estimates of E

22. 29 April 2008Birkbeck College, U. London22 Better Way of Estimating E

23. 29 April 2008Birkbeck College, U. London23 Geometric Picture ●● First imageSecond image

24. 29 April 2008Birkbeck College, U. London24 Camera Calibration Ideal pixel coordinates Measured pixel coordinates Ideal CCF Camera calibration is a transformation from measured pixel coordinates to ideal pixel coordinates.

25. 29 April 2008Birkbeck College, U. London25 Calibration Matrix

26. 29 April 2008Birkbeck College, U. London26 Fundamental Matrix The fundamental matrix F is defined by

27. 29 April 2008Birkbeck College, U. London27 Properties of E and F det(E)=det(T t )det(R)=0 The matrix E is essential iff SingularValues(E) = (σ,σ,0) det(F)=det(K ~ )det(E)det(K)=0 The matrix F is fundamental iff det(F)=0.

28. 29 April 2008Birkbeck College, U. London28 Minimal Data

29. 29 April 2008Birkbeck College, U. London29 Books D.A. Forsyth and J. Ponce. Computer Vision: a modern approach. Prentice Hall, 2003. R.C. Gonzalez and R.E. Woods. Digital Image Processing. Second edition, Prentice Hall, 2002.