875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Fall 2011

Introductions 2

Class Recent methods in computer vision Broadness of the class depends on YOU! Initial papers in class are the best papers of CVPR(2010,2011), ICCV(2011), ECCV(2010) 3

Grade Requirements Presentation of 3 papers in class  30 min talk,  10 min questions Papers for selection must come from:  top journals: IJCV, PAMI, CVIU, IVCJ  top conferences: CVPR (2010,2011), ICCV (2011), ECCV (2010), MICCAI (2010, 2011)  approval for all other venues is needed Final project  evaluation, extension of a recent method from the above 4

Schedule Aug. 29 th, Introduction Aug. 31 st, Geometric computer vision, first paper selection Sept. 5 th, Labor day Sept. 7 th, Geometric computer vision, Robust estimation Sept. 12 th, Optimization Sept. 19 th, Classification Sept. 21 st, 1. round of presentations starts Oct 17 th, 2. round of presentations starts Oct 31 st, definition of final projects due Nov 7 th, 3. round of presentations starts Dec 5 th and 7 th, final project presentation 5

How to give a great presentation Structure of the talk:  Motivation (motivate and explain the problem)  Overview  Related work (short concise discussion)  Approach  Experiments  Conclusion and future work 6

How to give a great presentation Use large enough fonts  5-6 one line bullet items on a slide max Keep it simple No complex formulas in your talk Bad Powerpoint slides How to for presentations 7

How to give a great presentation Abstract the material of the talk  provide understanding beyond details Use pictures to illustrate  find pictures on the internet  create a graphic (in ppt, graph tool)  animate complex pictures 8

How to give a good presentation Avoid bad color schemes  no red on blue looks awful Avoid using laser pointer (especially if you are nervous) Add pointing elements in your presentation Practice to stay within your time! Don’t rush through the talk! 9

Projective and homogeneous points Given: Plane  in R 2 embedded in P 2 at coordinates w=1  viewing ray g intersects plane at v (homogeneous coordinates)  all points on ray g project onto the same homogeneous point v  projection of g onto  is defined by scaling v=g/l = g/w w=1 R3R3 O  (R 2 ) w x y

Affine and projective transformations Affine transformation leaves infinite points at infinity Projective transformations move infinite points into finite affine space Example: Parallel lines intersect at the horizon (line of infinite points). We can see this intersection due to perspective projection!

Homogeneous coordinates Homogeneous representation of lines equivalence class of vectors, any vector is representative Set of all equivalence classes in R 3  (0,0,0) T forms P 2 Homogeneous representation of points on if and only if The point x lies on the line l if and only if x T l = l T x = 0 Homogeneous coordinates Inhomogeneous coordinates but only 2DOF

Ideal points and the line at infinity Intersections of parallel lines Example Ideal points Line at infinity normal direction Note that in P 2 there is no distinction between ideal points and others

Pinhole Camera Model Camera obscura (Frankreich, 1830)

object optical axis image plane principal point (u,v) Pinhole Camera Model focal length, aspect ratio (f, af) Skew s aperture

Pinhole Camera Model Selbstkalibrierung bestimmt die intrinsischen Kameraparameter

Introduction to Computer Vision for Robotics Projective Transformation Projective Transformation maps M onto M p in P 3 space X Y O Projective Transformation linearizes projection

Introduction to Computer Vision for Robotics Projection in general pose Rotation [R] Projection center C M World coordinates Projection: mpmp

Introduction to Computer Vision for Robotics Projection matrix P Camera projection matrix P combines:  inverse affine transformation T cam -1 from general pose to origin  Perspective projection P 0 to image plane at Z 0 =1  affine mapping K from image to sensor coordinates

Homography X Y 0  Homography  plane to plane warping  purely rotating camera

Self-calibration for Rotating Cameras Agapito et al. Rotation invariant formulation Projection of the dual absolute conic into image i Projection of the dual absolute conic into image j  calibration through Choleski decomposition

Removing projective distortion select four points in a plane with know coordinates (linear in h ij ) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary

Freely Moving Camera X Y CjCj Z CiCi Epipolar Linie:  Computable from image correspondences

Example: motion parallel with image plane

Example: forward motion e e’e’

Introduction to Computer Vision for Robotics The Essential Matrix E F is the most general constraint on an image pair. If the camera calibration matrix K is known, then more constraints are available Essential Matrix E E holds the relative orientation of a calibrated camera pair. It has 5 degrees of freedom: 3 from rotation matrix R ik, 2 from direction of translation e, the epipole.

Estimation of P from E From E we can obtain a camera projection matrix pair: E=Udiag(0,0,1)V T P 0 =[I 3x3 | 0 3x1 ] and there are four choices for P 1 : P 1 =[UWV T | +u 3 ] or P 1 =[UWV T | -u 3 ] or P 1 =[UW T V T | +u 3 ] or P 1 =[UW T V T | -u 3 ] four possible configurations: only one with 3D point in front of both cameras

Kruppa Equations Kruppa-equation (Faugeras et al.`92) for constant camera calibration Dual absolute conic limited to epipolar-geometrie