1. Linear cameras What is a perspective camera? Reguli Linear congruences Direct and inverse projections Multi-view geometry Présentations mercredi 26 mai de 9h30 à midi, salle U/V Planches : –http://www.di.ens.fr/~ponce/geomvis/lect6.ppthttp://www.di.ens.fr/~ponce/geomvis/lect6.ppt –http://www.di.ens.fr/~ponce/geomvis/lect6.pdfhttp://www.di.ens.fr/~ponce/geomvis/lect6.pdf

2. П1П1 Chasles’ absolute conic: x 1 2 +x 2 2 +x 3 2 = 0, x 4 = 0. Kruppa (1913); Maybank & Faugeras (1992) Triggs (1997); Pollefeys et al. (1998,2002)  ,  u 0, v 0 The absolute quadric u 0 = v 0 = 0 The absolute quadratic complex  2 =  2,  = 0 u0u0 v0v0 k l f x’ ¼ P ( H H -1 ) x H = [ X y ]

3. Relation between K, , and  *

4. » = u 1 » 1 + u 2 » 2 + u 3 » 3 Line bundles c x 33 11 22 

5. » = u 1 » 1 + u 2 » 2 + u 3 » 3 y = u 1 y 1 + u 2 y 2 + u 3 y 3 y2y2 c r x y y1y1 33 11 22  y3y3 Line bundles

6. » = X u, where X 2 R 6 £ 3, u 2 R 3 y = Y u, where Y 2 R 4 £ 3, u 2 R 3 y2y2 c r x y y1y1 33 11 22 y3y3 Line bundles 

7. u = Y z y y = Y z y [(c Ç x) Æ r] y2y2 c r x y y1y1 33 11 22 y3y3 Line bundles  Note:

8. u = Y z y y = Y z y [(c Ç x) Æ r] y2y2 c r x y y1y1 33 11 22 y3y3 Line bundles  Note: (c Ç x) Æ r = [c x – x c ] r TT

9. u = Y z y y = Y z y [(c Ç x) Æ r] = P x when z = c y2y2 c r x y y1y1 33 11 22 y3y3 Line bundles 

10. c r x y  u ¼ P x  ¼ P * p ¼ P T   ¼ P T y Perspective projection z  p’p’

11. c r x y  y ¼ P x  ¼ P * p ¼ P T   ¼ P T y Perspective projection z  p’p’ Note: y and u have are identified here

12. П1П1 Chasles’ absolute conic: x 1 2 + x 2 2 + x 3 2 = 0, x 4 = 0. The absolute quadratic complex:  T diag(Id,0)  = | u | 2 = 0.

13. Perspective projection c r x x ’  c r x x ’  x’ ¼ P x  ’ ¼ P *  p’ ¼ P T  ’  ¼ P T x’ x’ ¼ P x  ¼ P *  p’ ¼ P T   ¼ P T The AQC general equation:  T  = 0, with  = X *T X * Proposition:  T  ’ ¼ û ¢ û’ Proposition : P  P T ¼  ’ p  y ’  ’ y ’  ’ Proposition : P  * P T ¼  * Triggs (1997); Pollefeys et al. (1998) e p T = H p p T e x = H -1 p x e  = p 

14. Relation between K, , and  *

15. What is a camera? (Ponce, CVPR’09) x c ξ r y x

16. x c ξ r y c

17. x c ξ r y x c ξ

18. x c ξ r y x c ξ ξ

19. x c ξ r y x ξ r y Linear family of lines x ξ x c ξ ξ ξ

20. Lines linearly dependent on 2 or 3 lines (Veblen & Young, 1910) Then go on recursively for general linear dependence © H. Havlicek, VUT

21. What a camera is Definition: A camera is a two-parameter linear family of lines – that is, a degenerate regulus, or a non-degenerate linear congruence.

22. Rank-3 families: Reguli Line fields ≡ epipolar plane images (Bolles, Baker, Marimont, 1987) Line bundles

23. Rank-4 (nondegenerate) families: Linear congruences Figures © H. Havlicek, VUT

24. x ξ y r x y r ξ Hyperbolic linear congruences Crossed-slit cameras (Zomet et al., 2003) Linear pushbroom cameras (Gupta & Hartley, 1997)

25. © E. Molzcan © Leica Hyperbolic linear congruences

26. © T. Pajdla, CTU Elliptic linear congruences Linear oblique cameras (Pajdla, 2002) Bilinear cameras (Yu & McMillan, 2004) Stereo panoramas / cyclographs (Seitz & Kim, 2002)

27. Parabolic linear congruences Pencil cameras (Yu & McMillam, 2004) Axial cameras (Sturm, 2005)

28. Plücker coordinates and the Klein quadric line screw the Klein quadric  = x Ç y = uvuv [ ] x y  Note: u. v = 0 P5P5

29. Pencils of screws and linear congruences line s P5P5 the Klein quadric Reciprocal screws: (s | t) = 0 Screw ≈ linear complex: s ≈ { ± | ( s | ± ) = 0 }

30. line s P5P5 the Klein quadric t l Pencils of screws and linear congruences Reciprocal screws: (s | t) = 0 Screw ≈ linear complex: s ≈ { ± | ( s | ± ) = 0 } Pencil of screws: l = { ¸ s + ¹k t ; ¸k¹ 2 R } The carrier of l is a linear congruence

31. P5P5 e h p Reciprocal screws: (s | t) = 0 Screw ≈ linear complex: s ≈ { ± | ( s | ± ) = 0 } Pencil of screws: l = { ¸ s + ¹k t ; ¸k¹ 2 R } The carrier of l is a linear congruence Pencils of screws and linear congruences

32. x ±2±2 Hyperbolic linear congruences » ±1±1

33. x »1»1 p1p1 ±1±1 ±2±2 p2p2 » = (x T [ p 1 p 2 T ]x) » 1 + (x T [ p 1 p 2 T ]x) » 2 + (x T [ p 1 p 2 T ]x) » 3 + (x T [ p 1 p 2 T ]x) » 4 »2»2 »3»3 »4»4 »

34. x »1»1 p1p1 ±1±1 ±2±2 p2p2 Hyperbolic linear congruences » = (y T [ p 1 p 2 T ] y) » 1 + (y T [ p 1 q 2 T ] y) » 2 + (y T [ q 1 p 2 T ] y) » 3 + (y T [ q 1 q 2 T ] y) » 4 y = u 1 y 1 + u 2 y 2 + u 3 y 3 = Y u »2»2 »3»3 »4»4 » y

35. x »1»1 p1p1 ±1±1 ±2±2 p2p2 Hyperbolic linear congruences » = (u T [ ¼ 1 ¼ 2 T ]u) » 1 + (u T [ ¼ 1 ρ ½ 2 T ]u) » 2 + (u T [ ρ ½ 1 ¼ 2 T ]u) » 3 + (u T [ρ ½ 1 ρ ½ 2 T ]u) » 4 = X û, where X 2 R 6 £ 4 and û 2 R 4 »2»2 »3»3 »4»4 » y

36. x ξ ± a2a2 p1p1 z p2p2 p a1a1 Parabolic linear congruences ± s ° T » = X û, where X 2 R 6 £ 5 and û 2 R 5

37. Elliptic linear congruences x » y » = X û, where X 2 R 6 £ 4 and û 2 R 4

38. x »1»1 y1y1 »2»2 y2y2 Epipolar geometry ( » 1 | » 2 ) = 0 or û 1 T F û 2 = 0, where F = X 1 T X 2 2 R 4 £ 4 Feldman et al. (2003): 6 £ 6 F for crossed-slit cameras Gupta & Hartley (1997): 4 £ 4 F for linear pushbroom cameras

39. Trinocular geometry D i ( » 1, » 2, » 3 ) = 0 or T i (û 1, û 2, û 3 ) = 0, for i = 1,2,3,4  1  1  1  2  2  2  3  3  3  4  4  4  5  5  5  6  6  6 δ η φ x

40. The essential map (Oblique cameras, Pajdla, 2002) x ξ Ax x ! ξ = x Ç Ax B H P E Canonical forms of essential maps A ( = matrices with quadratic minimal poylnomial) (Batog, Goaoc, Ponce, 2009) Alternative geometric characterization of linear congruences

41. A new elliptic camera? (Batog, Goaoc, Ponce, 2010)

42. SMOOTH SURFACES AND THEIR OUTLINES Elements of Differential Geometry What are the Inflections of the Contour? Koenderink’s Theorem The second fundamental form Koenderink’s Theorem Aspect graphs More differential geometry A catalogue of visual events Computing the aspect graph http://www.di.ens.fr/~ponce/geomvis/lect6.ppt http://www.di.ens.fr/~ponce/geomvis/lect6.pdf

43. Smooth Shapes and their Outlines Can we say anything about a 3D shape from the shape of its contour?

44. What are the contour stable features?? foldscuspsT-junctions Shadows are like silhouettes.. Reprinted from “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992).  1992 Kluwer Academic Publishers. Reprinted from “Solid Shape,” by J.J. Koenderink, MIT Press (1990).  1990 by the MIT.

45. Differential geometry: geometry in the small A tangent is the limit of a sequence of secants. The normal to a curve is perpendicular to the tangent line.

46. What can happen to a curve in the vicinity of a point? (a) Regular point; (b) inflection; (c) cusp of the first kind; (d) cusp of the second kind.

47. The Gauss Map It maps points on a curve onto points on the unit circle. The direction of traversal of the Gaussian image reverts at inflections: it folds there.

48. The curvature C C is the center of curvature; R = CP is the radius of curvature;  = lim  s = 1/R is the curvature.

49. Closed curves admit a canonical orientation..  > 0 <0<0  = d  / ds à derivative of the Gauss map!

50. Twisted curves are more complicated animals..

51. A smooth surface, its tangent plane and its normal.

52. Normal sections and normal curvatures Principal curvatures: minimum value  maximum value  Gaussian curvature: K =  1 1 2 2

53. The differential of the Gauss map dN (t)= lim  s ! 0 Second fundamental form: II( u, v) = u T dN ( v ) (II is symmetric.) The normal curvature is  t = II ( t, t ). Two directions are said to be conjugated when II ( u, v ) = 0.

54. The local shape of a smooth surface Elliptic point Hyperbolic point Parabolic point K > 0 K < 0 K = 0 Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers.

55. The parabolic lines marked on the Apollo Belvedere by Felix Klein

56. N. v = 0 ) II( t, v )=0 Asymptotic directions: The contour cusps when when a viewing ray grazes the surface along an asymptotic direction. II(u,u)=0

57. The Gauss map The Gauss map folds at parabolic points. Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers. K = dA’/dA

58. Smooth Shapes and their Outlines Can we say anything about a 3D shape from the shape of its contour?

59. Theorem [Koenderink, 1984]: the inflections of the silhouette are the projections of parabolic points.

60. Koenderink’s Theorem (1984) K =   rc Note:  > 0. r Corollary: K and  have the same sign! c Proof: Based on the idea that, given two conjugated directions, K sin 2  =  u  v

61. What are the contour stable features?? foldsT-junctionscusps How does the appearance of an object change with viewpoint? Reprinted from “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992).  1992 Kluwer Academic Publishers.

62. Imaging in Flatland: Stable Views

63. Visual Event: Change in Ordering of Contour Points Transparent ObjectOpaque Object

64. Visual Event: Change in Number of Contour Points Transparent ObjectOpaque Object

65. Exceptional and Generic Curves

66. The Aspect Graph In Flatland

67. The Geometry of the Gauss Map Gauss sphere Image of parabolic curve Moving great circle Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers.

68. Asymptotic directions at ordinary hyperbolic points The integral curves of the asymptotic directions form two families of asymptotic curves (red and blue)

69. Asymptotic curves Parabolic curve Fold Asymptotic curves’ images Gauss map Asymptotic directions are self conjugate: a. dN ( a ) = 0 At a parabolic point dN ( a ) = 0, so for any curve t. dN ( a ) = a. dN ( t ) = 0 In particular, if t is the tangent to the parabolic curve itself dN ( a ) ¼ dN ( t )

70. The Lip Event Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers. v. dN (a) = 0 ) v ¼ a

71. The Beak-to-Beak Event Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers. v. dN (a) = 0 ) v ¼ a

72. Ordinary Hyperbolic Point Flecnodal Point Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” By S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers.

73. Red asymptotic curves Red flecnodal curve Asymptotic spherical map Red asymptotic curves Red flecnodal curve Cusp pairs appear or disappear as one crosses the fold of the asymptotic spherical map. This happens at asymptotic directions along parabolic curves, and asymptotic directions along flecnodal curves.

74. The Swallowtail Event Flecnodal Point Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” by S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers.

75. The Bitangent Ray Manifold: Ordinary bitangents....and exceptional (limiting) ones. limiting bitangent line unode Reprinted from “Toward a Scale-Space Aspect Graph: Solids of Revolution,” by S. Pae and J. Ponce, Proc. IEEE Conf. on Computer Vision and Pattern Recognition (1999).  1999 IEEE.

76. The Tangent Crossing Event Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” by S. Pae and J. Ponce, the International Journal of Computer Vision, 43(2):113-131 (2001).  2001 Kluwer Academic Publishers.

77. The Cusp Crossing Event After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992).  1992 Kluwer Academic Publishers.

78. The Triple Point Event After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992).  1992 Kluwer Academic Publishers.

79. X0 X1 E1 S1 S2 E3 S1 S2 Tracing Visual Events P 1 (x 1,…,x n )=0 … P n (x 1,…,x n )=0 F(x,y,z)=0 Computing the Aspect Graph Curve Tracing Cell Decomposition After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992).  1992 Kluwer Academic Publishers.

80. An Example

81. Approximate Aspect Graphs (Ikeuchi & Kanade, 1987) Reprinted from “Automatic Generation of Object Recognition Programs,” by K. Ikeuchi and T. Kanade, Proc. of the IEEE, 76(8):1016-1035 (1988).  1988 IEEE.

82. Approximate Aspect Graphs II: Object Localization (Ikeuchi & Kanade, 1987) Reprinted from “Precompiling a Geometrical Model into an Interpretation Tree for Object Recognition in Bin-Picking Tasks,” by K. Ikeuchi, Proc. DARPA Image Understanding Workshop, 1987.