1. CS 395/495-26: Spring 2004 IBMR: 2-D Conics: Introduction Jack Tumblin jet@cs.northwestern.edu

2. Intuition for Conics: 2D Ellipses Still a circle/ellipse after P 2 projective transform (Better!) every transformed conic is another conic! H1H1H1H1 H2H2H2H2 H3H3H3H3

3. Intuition for Conics: 2D Ellipses Cartesian axis Circle/Ellipse: a x 2 + b y 2 – r 2 = 0Cartesian axis Circle/Ellipse: a x 2 + b y 2 – r 2 = 0 P 2 axis Circle/Ellipse: a x 1 2 + b x 2 2 – r 2 x 3 2 = 0P 2 axis Circle/Ellipse: a x 1 2 + b x 2 2 – r 2 x 3 2 = 0 5 parameters (deg. of freedom):5 parameters (deg. of freedom): –center point (cx, cy) –major, minor axes (1/a, 1/b) –orientation  1/a 1/b 

4. Conics: Key Ideas –‘Conics’ == intersection of cone & plane: –Many possible shapes: circles, ellipses, parabola, hyperbola, degenerate lines & points In P 2, Conics have Dual Forms: –‘Point Conics’ defined by points on the curve –‘Line Conics’ defined by lines tangent to curve

5. Conics written as a Matrix –All points on any conic solve a 2D quadratic: ax 2 + bxy + cy 2 +dx +ey +f = 0 –In homogeneous coordinates: (mpy. all by x 3 2 ) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 x T Cx = 0 x T Cx = 0 –C is symmetric, 5DOF ( not 6; ignores x 3 scaling) –Can find any C from 5 homog. points... How? (SVD solves Null-space problems easily…) a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 (and how could conics help you find H ?...)

6. Familiar Conics: Circles, Ellipses... x T C x = 0 or General form of point conic:General form of point conic: ax 2 + bxy + cy 2 +dx +ey +f = 0 (or) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 Circle at d with radius r:Circle at d with radius r: 1x 1 2 + bx 1 x 2 + 1x 2 2 +dx 1 x 3 +ex 2 x 3 –r 2 x 3 2 = 0; C r = ‘Degenerate’ Cases (C rank<3): lines, parabolas, hyperbolae... Example:‘Degenerate’ Cases (C rank<3): lines, parabolas, hyperbolae... Example: ax 1 2 + bx 1 x 2 + 0x 2 2 +dx 1 +ex 2 + fx 3 2 = 0; a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 1 0 0 0 1 0 0 0 -r 2 a 0 0 0 0 0 0 0 f (Try it—plot points for x 3 = 1…)

7. Matrix C makes conic curves from points x : x T Cx = 0C is a ‘point conic’Matrix C makes conic curves from points x : x T Cx = 0C is a ‘point conic’ The tangent line L for point x is: L = C xThe tangent line L for point x is: L = C x P2 point conic transformed by H:P2 point conic transformed by H: C’ = H -T C H -1 (messy! requires an invertible H !) Point Conics: C L x Prove it! Use the two equations above…

8. Matrix C* makes conic curves from lines L : L T C* L = 0C is a ‘line conic’Matrix C* makes conic curves from lines L : L T C* L = 0C is a ‘line conic’ The tangent point x for line L is: x = C* LThe tangent point x for line L is: x = C* L Projective transform by H is:Projective transform by H is: C*’ = H C* H T (better -- no H inversion required) If C is non-singular (rank 3), then C* = C -1If C is non-singular (rank 3), then C* = C -1 ‘Dual’ or Line Conics: C L x Prove it! Use two equations above…

9. Conic Summary –Matrix C identifies all the points on a conic: x T Cx = 0 C is a ‘point conic’ –Given a point x on a conic curve, the homog. tangent line l is given by l = C x –Matrix C* identifies all lines tangent to a conic: l T C*l = 0C* is a ‘Dual Conic’ –Can you prove this? Try it! C* = C -1 (hint: use the boxed equations)

10. Degenerate Conics x T C x = 0 or General form of point conic:General form of point conic: ax 2 + bxy + cy 2 +dx +ey +f = 0 (or) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 Very special ‘degenerate’ case: ‘infinite radius circle’ as x 3  0, aka C  Very special ‘degenerate’ case: ‘infinite radius circle’ as x 3  0, aka C  1x 1 2 + bx 1 x 2 + 1x 2 2 +dx 1 x 3 +ex 2 x 3 +0x 3 2 = 0 C  = only solutions to x T C  x =0 are 2 complex x vectors: and a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 1 0 0 0 1 0 0 0 0 1 i 0 1 -i -i 0

11. Conics: Key Ideas –Transformed conics tell you how H will change angles: –one strange, degenerate conic C*  can measure angle –Angles between line pairs yield transformed C*  2D world space 2D image space H = H S H A H P C*  = 1 0 0 0 1 0 0 0 0 L m m’ L’ C*  ’ =      0

12. Conic Weirdness I ‘Circular Points’ I=, J= ; and C  = –‘Ideal points’; 2 points at infinity / horizon, but… –Complex : real part on x 1 axis, imag. on x 2 –“Where infinite circle hits horizon line L  [0,0,1]” –Where? imaginary! (infinite x,y axes, maybe?) ?WHY BOTHER? –To measure angles in projective space –To use angle correspondences to find H A,H P 1 i 0 1 -i -i 0 1 0 0 0 1 0 0 0 0

13. Conic Weirdness 2 ‘Circular Points’ I=, J= ; and C  = C  is a point conic, affected by H A H P only.C  is a point conic, affected by H A H P only. C*  is line conic, but is the same matrix (!?!)C*  is line conic, but is the same matrix (!?!) –Is same as this outer product: IJ T + JI T –Intersects L , the infinity line (proof: L  C*  L  =0) –Affected by H P and H A only, ( H S does nothing) ALL transforms of C*  have only 4DOF (pg 33 ) 1 i 0 1 -i -i 0 1 0 0 0 1 0 0 0 0

14. Conics: Angle Measuring Matrix H transforms C*  to another space:Matrix H transforms C*  to another space: C*  ’ = H C*  H T Angles: –define ‘world space’ C*  as –(for any two world-space homogeneous lines) L and m are perpendicular iff L T C*  m = 0 –Angle  between lines L and m given by 1 0 0 0 1 0 0 0 0 (L T C*  m) (L T C*  m) (L T C*  L)(m T C*  m) (L T C*  L)(m T C*  m) cos(  ) = (Remember, it is simpler to transform a line conic)

15. Conics: Angle Measuring Matrix H transforms C*  to another space:Matrix H transforms C*  to another space: C*  ’ = H C*  H T Angles: –ALSO true for transformed L, m and C* , so… –If you FIND C*  ’ in image space, you can measure 2 image-space lines L’, m’ and find the angle between them in world space! Use the same expression: –If you FIND C*  ’ in image space, you can measure 2 image-space lines L’, m’ and find the angle  between them in world space! Use the same expression: (L’ T C*  ’ m’) (L’ T C*  ’ m’) (L’ T C*  ’ L’) (m T C*  ’ m’) (L’ T C*  ’ L’) (m T C*  ’ m’) cos(  ) = (Remember, it is simpler to transform a line conic)

16. ? How can we find C*  ’ without H ? One Answer: measure perpendicular lines… –We know 2 world-space lines L and m are perpendicular iff L T C*  m = 0 –Also true for transformed L, m and C*  : (e.g. in image space) L’ T C*  ’ m’ = 0 Hmmm. L’ and m’ are known, C*  ’ unknown. Isn’t this a null-space problem? Once we have C*  ’ then what? See any other uses for conics in P2? 1 0 0 0 1 0 0 0 0

17. Polar Lines and Pole Points Line Conic C’s tangent line L t at point x t by:Line Conic C’s tangent line L t at point x t by: C x t = L t (given x t is on the conic: x t T C x t =0) xtxtxtxt LtLtLtLt C (You can skip this)

18. Polar Lines and Pole Points Line Conic C’s tangent line L t at point x t by:Line Conic C’s tangent line L t at point x t by: C x t = L t (given x t is on the conic: x t T C x t =0) But if x is NOT on the conic? try Cx p = L pBut if x is NOT on the conic? try Cx p = L p xpxpxpxp LpLpLpLp C (You can skip this)

19. Polar Lines and Pole Points Line Conic C’s tangent line L t at point x t by:Line Conic C’s tangent line L t at point x t by: C x t = L t (given x t is on the conic: x t T C x t =0) But if x is NOT on the conic? try Cx p = L pBut if x is NOT on the conic? try Cx p = L p ‘Polar line’ L p = conic at p 1, p 2 (find them?ugly!)‘Polar line’ L p = conic at p 1, p 2 (find them?ugly!) xpxpxpxp LpLpLpLp C p1p1p1p1 p2p2p2p2 (You can skip this)

20. Polar Lines and Pole Points Line Conic C’s tangent line L t at point x t by:Line Conic C’s tangent line L t at point x t by: C x t = L t (given x t is on the conic: x t T C x t =0) But if x is NOT on the conic? try Cx p = L pBut if x is NOT on the conic? try Cx p = L p ‘Polar line’ L p = conic at p 1, p 2 (to find them?ugly!)‘Polar line’ L p = conic at p 1, p 2 (to find them?ugly!) p 1, p 2 tangent lines meet at ‘Pole point’ x pp 1, p 2 tangent lines meet at ‘Pole point’ x p xpxpxpxp LpLpLpLp C p1p1p1p1 p2p2p2p2 (You can skip this)

21. Polar Lines and Pole Points Line Conic C’s tangent line L t at point x t by:Line Conic C’s tangent line L t at point x t by: C x t = L t (given x t is on the conic: x t T C x t =0) But if x is NOT on the conic? try Cx p = L pBut if x is NOT on the conic? try Cx p = L p ‘Polar line’ L p = conic at p 1, p 2 (to find them?ugly!)‘Polar line’ L p = conic at p 1, p 2 (to find them?ugly!) p 1, p 2 tangent lines meet at ‘Pole point’ x pp 1, p 2 tangent lines meet at ‘Pole point’ x p xpxpxpxp LpLpLpLp C p1p1p1p1 p2p2p2p2 Interesting, But why does book show it?

22. Question: Can you apply DLT? Choose pairs of lines that are known to be perpendicular in world space; measure those lines in image space.Choose pairs of lines that are known to be perpendicular in world space; measure those lines in image space. Solve for C*  ’ using L’ T C*  ’ m’ = 0Solve for C*  ’ using L’ T C*  ’ m’ = 0 We also know that HC*  H T = C*  ’We also know that HC*  H T = C*  ’ Can you solve for H? Zisserman tried, but book method is confused, contains errors, and may be incorrect…

23. END. STOP HERE

24. x’ Apply the 3x3 matrix H x’ = Hx Apply the 3x3 matrix H x’ = Hx Recall: Projective Transform H x3x3x3x3 x2x2x2x2 x1x1x1x1 x y x’ y’ (x,y) x (x’,y’) Homog. coords x’ = [x’,y’,1] T  2D image (x’,y’) 2D image (x,y)  Homog. coords [x,y,1] T = x

25. x’ Apply the 3x3 matrix H x’ = Hx Apply the 3x3 matrix H x’ = Hx Recall: Projective Transform H x3x3x3x3 x2x2x2x2 x1x1x1x1 x y x’ y’ (x,y) x (x’,y’) Homog. coords x’ = [x’,y’,1] T  2D image (x’,y’) 2D image (x,y)  Homog. coords [x,y,1] T = x Goal: Find H for ‘Image Rectification’ ‘World Plane’ ‘View Plane’

26. Recall: Rectification Undo parts of H x’ = H xwhere H = H S H A H P GOAL: Put world plane x’ into view plane x –Affine Rect.; (find only H P (2DOF)); –Metric Rect.;(find H A and H P (6DOF)); –Full Rect.;(find all: H S H A H P (8DOF)); METHODS: 1.Affine: ‘Vanishing Point’, Horizon line methods 2.Metric: Conics & Circular Points 3.Full: 4-point correspondence

27. Recall: The bits and pieces of H Recall: The bits and pieces of H H has 8 independent variables (DOF) Computer Vision Jargon (2D projective):Computer Vision Jargon (2D projective): Isometry --3DOF(2D translate t x,t y ; 2D rotate  z ; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s ,/s, s  /s) Projective--8DOF (changes x 3 ; 3D-rotation-like) Rectification “up to a Similarity” (finds H P, H A )

28. Conics: Key Ideas –Transformed conics tell you how H changes angles –one strange, degenerate conic C*  measures angle –Angles between line pairs yield transformed C*  –We can compute H A and H P from transformed C*  2D world space 2D image space H S H A H P C*  = C*  ’ = 1 0 0 0 1 0 0 0 0    L M m L     

29. Undoing H: Metric Rect. 1 C*  ’ =  C*  ’ =  First, ignore projective part: set v=0.First, ignore projective part: set v=0. Choose two pairs of perpendicular lines L,mChoose two pairs of perpendicular lines L,m Second, Use L T ’ C*  ’ m’ = 0 to solve for KSecond, Use L T ’ C*  ’ m’ = 0 to solve for K KK T  Kv       v T K  s 2D world space 2D image space H S H A H P L M m L KK T  0       0  s 0  s aka ‘Weak Perspective’

30. Undoing H: Metric Rectification C*  ’ has only 4DOF (before,after ANY H)C*  ’ has only 4DOF (before,after ANY H) All C*  ’ DOF are contained in (H A H P )All C*  ’ DOF are contained in (H A H P ) SVD can convert C*  ’ to (H A H P ) matrix!SVD can convert C*  ’ to (H A H P ) matrix! Task is then “find C*  ’, then use it to find H”Task is then “find C*  ’, then use it to find H” 2D world space 2D image space H S H A H P C*  = C*  ’ =    L M m L    0 1 0 0 0 1 0 0 0 0

31. Undoing H: Metric Rect. 1 H = H S H A H P =H = H S H A H P = Book shows how to write transformed C*  as (simple, tedious):Book shows how to write transformed C*  as (simple, tedious): C*  ’ = C*  ’ = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) sRtsRt0T10T1sRtsRt0T10T1 I0I0vTvvTvI0I0vTvvTv K0K00T10T1K0K00T10T1              KK T  Kv       v T K  0

32. Undoing H: Metric Rect. 1 H = H S H A H P =H = H S H A H P = Book shows how to write transformed C*  as (simple, tedious):Book shows how to write transformed C*  as (simple, tedious): C*  ’ = C*  ’ = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) sRtsRt0T10T1sRtsRt0T10T1 I0I0vTvvTvI0I0vTvvTv K0K00T10T1K0K00T10T1              KK T  Kv       v T K  0 !!!BUT DERIVATION & RESULT IS WRONG!!! SKIP ‘Metric Rectification I’ pg 35,36!

33. ‘Let SVDs Explain it All for You’ SVD finds H for you (?) (pg 35) Book’s Method for finding H A H P from C*  ’ : –C*  ’ = H C*  H T is symmetric; –Assume SVD(C*  ’) = USV T = (U S ) I ( S V T ) = ( H ) C*  ( H T ) = ( H ) C*  ( H T ) –BUT THIS DOESN’T WORK! –( Recall C*  = ) A = USV T ‘Find Input & Output Axes, Linked by Scale’ 1 0 0 0 1 0 0 0 0

34. Eigen -values,-vectors, Fixed pt & line Formalizes ‘invariant’ notion:Formalizes ‘invariant’ notion: –if x is ‘fixed’ for H, then Hx only scales x H x = x ( is a constant scale factor) –x is an ‘eigenvector’, is its ‘eigenvalue’ –again, SVD helps you find them. Elaborate topic (but not hard). Skip for now.Elaborate topic (but not hard). Skip for now. (You can skip this)

35. Book Errata Website: http://www.robots.ox.ac.uk/~az/HZbook/HZerrata.html pg. 35, eqn. 1.22 is dubious / erroneous; C*  ’ = HC*  H T = (H S H A H P ) C*  (H S H A H P ) T C*  ’ = HC*  H T = (H S H A H P ) C*  (H S H A H P ) T is the transformed line conic, but book uses C*  ’ = (H P H A H S ) C*  (H P H A H S ) T instead, AND has error: = But I get: C*  ’ = KK T  Kv       v T K  ? (book: ?=0; errata website: ?=v T KK T v) KK T  KK T v       v T KK T  v T KK T v

36. Dubious Book Aggravations: –Similarity transform H S leaves C*  unchanged: C*  = H S C*  H S T –H transforms C*  to image space C*  ’ by: C*  ’ = H C*  H T = (H S H A H P ) C*  ( H S H A H P ) T –Book claims to derive C*  ’ = (H P H A ) C*  ( H P H A ) T (!?!?!) Then errs in simplifying to: 1 0 0 0 1 0 0 0 0 KK T  Kv       v T K  0 *

37. SVDs and Conics Conics (both C and C*) are symmetric;Conics (both C and C*) are symmetric; SVD of any symmetric A is also symmetric: SVD(A) = USU T !Not Always!SVD of any symmetric A is also symmetric: SVD(A) = USU T !Not Always! Find conics’ singular values’ sign: s i =0,1, or –1, to classify conic type: (pg 40)Find conics’ singular values’ sign: s i =0,1, or –1, to classify conic type: (pg 40) S i values | Equation | Type. (1, 1, 1) (1, 1,-1) (1, 1, 0) (1,-1, 0) (1, 0, 0) x 2 + y 2 + w 2 = 0 x 2 + y 2 - w 2 = 0 x 2 + y 2 = 0 x 2 - y 2 = 0 x 2 = 0 imaginary-only circle/ellipse single real point (0,0,1) 2 lines: x+/- y 2 co-located lines: x=0

38. (recall) H = H S H A H P =(recall) H = H S H A H P = Tedious algebra * (pg35) shows symmetry:Tedious algebra * (pg35) shows symmetry: C*  ’ = H C*  H T = C*  ’ = H C*  H T = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) ?But what do K and V really control??But what do K and V really control? KK T  Kv       v T K  0 * K  0   0 T  1 Undoing H: Metric Rectification s R  t     0 T  1 I  0    v T  v *==error?!?

39. (recall) H = H S H A H P =(recall) H = H S H A H P = Tedious algebra (pg35) shows symmetry:Tedious algebra (pg35) shows symmetry: C*  ’ = H C*  H T = C*  ’ = H C*  H T = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) ?But what do K and V really control??But what do K and V really control? KK T  Kv       v T K  ? K  0   0 T  1 Undoing H: Metric Rectification s R  t     0 T  1 I  0    v T  v (book: ?=0; errata website: ?=v T KK T v)

40. Undoing H: Metric Rectification OK, then how do we find K and v?OK, then how do we find K and v? (book: ?=0; errata website: ?=v T KK T v) (book: ?=0; errata website: ?=v T KK T v) Choose known-perpendicular line pairs (L i, m i ), then compute by:Choose known-perpendicular line pairs (L i, m i ), then compute by: –Method 1a: (pg 36) Assume v=0, solve for K –Method 1b: (NOT in book) Assume K=I, solve for v –Method 2:Rearrange, solve for full C*  ’ then get H A H P using SVD. KK T  Kv       v T K  ? C*  ’ = H C*  H T =

41. Method 1a: (pg 36) Assume v=0, solve for K Choose 2  line pairs (L a,m a ) and (L b,m b )Choose 2  line pairs (L a,m a ) and (L b,m b ) Both pairs satisfy L T C*  ’ m = 0 or:Both pairs satisfy L T C*  ’ m = 0 or: (note x 3 term is ignored!)(note x 3 term is ignored!) ‘flatten’ to one equation:‘flatten’ to one equation: Undoing H: Metric Rect. 1a KK T  Kv       v T K  0 C*  ’ = H C*  H T =  KK T  0       0  0 0  0 s1 s2 0 s2 s3 0 0 0 0 m1m1m2m2m3m3m1m1m2m2m3m3 l 1 l 2 l 3 =0 l 1 m 1 l 2 m 1 +l 1 m 2 l 2 m 2 s1s1s2s2s3s3s1s1s2s2s3s3 = 0

42. Method 1a: (pg 36) Assume v=0, solve for K ‘Stack’ to combine both line pairs;‘Stack’ to combine both line pairs; Solve for s by SVD: find ‘input null space’ (Ax=0)Solve for s by SVD: find ‘input null space’ (Ax=0) From s make C*  ’ = KK T, then extract H A H P using SVD: recall C*  ’ = H C*  H T, it is symmetric…From s make C*  ’ = KK T, then extract H A H P using SVD: recall C*  ’ = H C*  H T, it is symmetric… Undoing H: Metric Rect. 1a KK T  Kv       v T K  0 C*  ’ = H C*  H T =  s1 s2 0 s2 s3 0 0 0 0 l a1 m a1 l a2 m a1 +l a1 m a2 l a2 m a2 s1s1s2s2s3s3s1s1s2s2s3s3 = 0 l b1 m b1 l b2 m b1 +l b1 m b2 l b2 m b2

43. Method 1b: (not in book) Assume K=I, solve for v Choose 2  line pairs (L a,m a ) and (L b,m b )Choose 2  line pairs (L a,m a ) and (L b,m b ) These satisfy L T C*  ’ m = 0 or:These satisfy L T C*  ’ m = 0 or: ‘flatten’ to one equation (messy), stack, solve for v 1,v 2‘flatten’ to one equation (messy), stack, solve for v 1,v 2 Extract H p from C*  ’ using SVDExtract H p from C*  ’ using SVD Undoing H: Metric Rect. 1b KK T  Kv       v T K  0 C*  ’ = H C*  H T =  I  v I  v       v T  0 v T  0 1 0 v1 0 1 v2 v1 v2 ? m1m1m2m2m3m3m1m1m2m2m3m3 l 1 l 2 l 3 =0