1. Projective Geometry and Geometric Invariance in Computer Vision Babak N. Araabi Electrical and Computer Eng. Dept. University of Tehran Workshop on image and video processing Mordad 13, 1382

2. 2 Do parallel lines intersect? Perhaps!

3. 3 Fifth Euclid's postulate For any line L and a point P not on L, there exists a unique line that is parallel to L (never meets L) and passes through P. At first glance it would seem that the parallel postulate ought to be a theorem deducible from the other more basic postulates. For centuries mathematicians tried to prove it. Eventually it was discovered that the parallel postulate is logically independent of the other postulates, and you get a perfectly consistent system even if you assume that parallel postulate is false.

4. 4 Alternative postulates Non-Euclidean geometry; though not a typical one The projective axiom: Any two lines intersect (in exactly one point). More intuitive approach: Take each line of ordinary Euclidean geometry and add to it one extra object called a point at infinity. In addition, the collection of all the extra objects together is also called a line in projective plane (called the line at infinity).

5. 5 Why Projective Geometry? A more appropriate framework when dealing with projection related issues. Perspective projection: Photophraphy Human vision Perspective projection is a non-linear mapping with Euclidean coordinates, but a linear mapping with homogeneous coordinates

6. 6 Overview 2D Projective Geometry 3D Projective Geometry Application: Invariant matching From 2D images to 3D space

7. 7 Projective 2D Geometry

8. 8 Homogeneous coordinates Homogeneous representation of lines equivalence class of vectors, any vector is representative Set of all equivalence classes in R 3  (0,0,0) T forms P 2 Homogeneous representation of points on if and only if The point x lies on the line l if and only if x T l = l T x = 0 Homogeneous coordinates Inhomogeneous coordinates but only 2DOF

9. 9 Points from lines and vice-versa Intersections of lines The intersection of two lines and is Line joining two points The line through two points and is Example

10. 10 Ideal points and the line at infinity Intersections of parallel lines Example Ideal points Line at infinity tangent vector normal direction Note that in P 2 there is no distinction between ideal points and others

11. 11 A model for the projective plane exactly one line through two points exaclty one point at intersection of two lines

12. 12 Duality Duality principle: To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem. For instance, the basic axiom that "for any two points, there is a unique line that intersects both those points", when turned around, becomes "for any two lines, there is a unique point that intersects (i.e., lies on) both those lines" In projective geometry, points and lines are completely interchangeable!

13. 13 Conics Curve described by 2 nd -degree equation in the plane or homogenized or in matrix form with 5DOF: 6 parameters

14. 14 Five points define a conic For each point the conic passes through or stacking constraints yields

15. 15 Projective transformations A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Definition: A mapping h : P 2  P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=Hx Theorem: Definition: Projective transformation or 8DOF projectivity=collineation=projective transformation=homography

16. 16 Mapping between planes central projection may be expressed by x’=Hx (application of theorem)

17. 17 Removing projective distortion select four points in a plane with know coordinates (linear in h ij ) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary, better ways to compute

18. 18 More examples

19. 19 Transformation of lines and conics Transformation for lines Transformation for conics For a point transformation

20. 20 A hierarchy of transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

21. 21 Class I: Isometries (iso=same, metric=measure) orientation preserving: orientation reversing: special cases: pure rotation, pure translation 3DOF (1 rotation, 2 translation) Invariants: length, angle, area

22. 22 Class II: Similarities (isometry + scale) also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) 4DOF (1 scale, 1 rotation, 2 translation) Invariants: ratios of length, angle, ratios of areas, parallel lines

23. 23 Class III: Affine transformations non-isotropic scaling! (2DOF: scale ratio and orientation) 6DOF (2 scale, 2 rotation, 2 translation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas

24. 24 Class VI: Projective transformations Action non-homogeneous over the plane 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Invariants: cross-ratio of four points on a line ratio of ratio of area

25. 25 Action of affinities and projectivities on line at infinity Line at infinity becomes finite, allows to observe vanishing points, horizon, Line at infinity stays at infinity, but points move along line

26. 26 Decomposition of projective transformations upper-triangular, decomposition unique (if chosen s>0) Example:

27. 27 Overview transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l ∞ Ratios of lengths, angles. The circular points I,J lengths, areas.

28. 28 Number of invariants? The number of functional invariants is equal to, or greater than, the number of degrees of freedom of the configuration less the number of degrees of freedom of the transformation e.g. configuration of 4 points in general position has 8 dof (2/pt) and so 4 similarity, 2 affinity and zero projective invariants

29. 29 Projective geometry of 1D The cross ratio Invariant under projective transformations 3DOF (2x2-1)

30. 30 Projective 3D Geometry

31. 31 Projective 3D Geometry Points, lines, planes and quadrics Transformations П ∞, ω ∞ and Ω ∞

32. 32 3D points in R 3 in P 3 (4x4-1=15 dof) projective transformation 3D point

33. 33 Planes Dual: points ↔ planes, lines ↔ lines 3D plane Euclidean representation Transformation

34. 34 Planes from points (solve as right nullspace of ) Or implicitly from coplanarity condition

35. 35 Points from planes (solve as right nullspace of ) Representing a plane by its span

36. 36 Lines Example: X -axis (4dof)

37. 37 Points, lines and planes

38. 38 Hierarchy of transformations Projective 15dof Affine 12dof Similarity 7dof Euclidean 6dof Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π ∞ The absolute conic Ω ∞ Volume

39. 39 The plane at infinity The plane at infinity π  is a fixed plane under a projective transformation H iff H is an affinity 1.canonical position 2.contains directions 3.two planes are parallel  line of intersection in π ∞ 4.line // line (or plane)  point of intersection in π ∞

40. 40 Application in Invariant Matching

41. 41 How to define an invariant 2D identifier patch on an image? Consider a multilateral identifier patch. Lines are invariant under the perspective projection. 3 (4) reference points are required to obtain an affine (projective) invariant representation of the multilateral identifier patch. Weak-perspective/perspective camera models.

42. 42 Affine invariant representation of a planar point Three reference points. Ratio of areas are invariant. X at intersection of R 1 R 2 and R 3 A. (r 1,r 2 ) invariants defined by: R3R3 R1R1 R2R2 X A

43. 43 Projective invariant representation of a planar point Four reference points. Ratio of ratio of areas are invariant. X at intersection of R 1 R 4 and R 3 A. (r 1,r 2 ) invariants defined by: R3R3 R1R1 R2R2 X A R4R4

44. 44 Affine invariant parallelogram: Three reference points Select A to make a parallelogram with three reference points R 1, R 2, and R 3 R3R3 R2R2 R1R1 A

45. 45 Affine invariant parallelogram: Construction of identifier patch (1) B2B2 B1B1 R3R3 R2R2 R1R1 A Select B 1 on R 2 R 3

46. 46 Affine invariant parallelogram: Construction of identifier patch (2) C2C2 C1C1 B2B2 B1B1 R3R3 R1R1 Select C 1 on B 1 R 1

47. 47 Affine invariant parallelogram: Construction of identifier patch (3) D2D2 D1D1 C2C2 C1C1 R3R3 R1R1 Select D 1 on C 1 R 1

48. 48 From 2D to 3D

49. 49 Perspective projection Perspective projection of 6 (feature) points from 3D world (dorsal fin) to 2D image: p i =(x i,y i,z i )q i =(u i,v i )i=1,…,6 p1p1 p2p2 p3p3 p4p4 p5p5 p6p6 Focal point Image plane q1q1 q2q2 q3q3 q4q4 q5q5 q6q6

50. 50 3D and 2D Projective Invariants I 1, I 2, and I 3 are 3D projective invariants (under PGL(4)). i 1, i 2, i 3, and i 4 are 2D projective invariants (under PGL(3)). There is no general case view-invariant. 2D invariants impose constraints (in the form of polynomial relations) on 3D invariants. * PGL(n): Projective General Linear group of order n.

51. 51 Invariants of point sets: Algebraic invariants Transfer images to a reference frame. A unique projective/affine transformation between any two sets of 4/3 corresponding points. Coordinate of any other point is invariant in the reference frame. P1P1 P2P2 P4P4 P3P3 P4P4 P3P3 P2P2 P1P1 P1P1 P3P3 P2P2 P4P4 Reference frame Global invariant

52. 52 Invariant Relations Constraints (here for 6 points) typically have the following form This is a fundamental object-image relation for six arbitrary 3D points and their 2D perspective projections. Therefore having at least 6 points in correspondence over a number of images, 3D invariants (I k ’s) can be estimated and invariant relations can be used as geometric invariant models. n  6 points 3n-15 I k ’s 2n-8 i k ’s 2n-11 relations or

53. 53 Where are we going?

54. 54 Thanks for your Attention! Questions?