1. Shape Spaces Kathryn Leonard 22 January 2005 MSRI Intro to Image Analysis

2. Underlying questions a) How should we represent shape? b) What does it mean for shapes to be close?

3. Human recognition Asymmetric distances and triangle inequality. Distance measure switching.

4. Observations 1. If shapes are the same when they are “close”, then recognition happens in shape neighborhoods. 2. Shape neighborhoods make a mathematician want to consider spaces of shapes, and metrics on those spaces.

5. Some shape neighborhoods Image courtesy of David Mumford.

6. Mathematical setting Shape = region in R 2 bounded by some curve (or collection of curves) with some degree of smoothness. Choose favorite shape representation. Construct space of shapes consisting of representations of all possible shapes. Based on choice of representation, put a metric on shape space.

7. Some shape representation categories 1. Boundary curve representations 2. Feature vectors 3. Structural descriptions and “grammars”

8. Properties of shape space (Assume hereafter that our shapes are bounded by a single simple closed curve.) 1. Contractible (flow by kN). 2. Non-linear. 3. Locally looks like a linear space.

9. Shape model I: the boundary curve Consider nested spaces of curves, analogous to nested spaces of functions. Notation:  = boundary curve of a shape  = tangent angle function to   = curvature of . Nesting: …  {L 2   ’ }  {L 2   }  {  is measurable}  {  is measurable}  {big ambient space}

10. Metrics on boundary curves Image courtesy of David Mumford.

11. Shape model II: horizon functions & curvelets Return to the functional setting: a shape is now a binary function (on = inside, off = outside), with boundary of on/off regions defined by a curve. Properties: Linear function space. Curvelets give an optimal decomposition in the L 2 norm for space of horizon functions whose boundaries are C 2. Metric = L 2 metric.

12. Why curvelets are good 1. Localize in scale, location, and orientation. 2. Possess a norm-equivalence property. 3. Possess an anisotropic scaling property.

13. Curvelets.v. wavelets Images taken from M. Elad, D. Donoho & J.-L. Starck, “Redundant Multiscale Transforms and Their Application for Morphological Content.”

14. More evidence for curvelet goodness What is this a picture of? Here we see an image decomposed into small, oriented pieces, where the magnitudes of these pieces is preserved while the orientations are randomized.

15. More evidence for curvelet goodness What is this a picture of? Here the orientations of the oriented pieces is preserved while the magnitudes are randomized.

16. More evidence for curvelet goodness On left, random orientation, correct magnitude. On right, correct orientation, random magnitude Suggests orientation is key. Current work: shape matching between data sets. Images taken from Eero Simoncelli; created using steerable pyramids.

17. Shape model III: medial axis Pair (m,r), where m is a collection of curves (skeleton) and r is a scalar function (length of ribs). Equivalent definitions: 1. Closure of locus of centers of maximal circles contained inside shape + radii. 2. Shock set obtained my evolving curve in direction of normal + time to shock formation. (grassfire)

18. Some shapes and their axes

19. Properties of medial axis Nice relationship to boundary curve (can go back and forth using known formulas). Geometric information is preserved. Captures symmetries of curve. Allows for good parts matching. Matches some computations in our own brains.

20. Notation and properties of m.a. Within a differentiable branch of the medial axis: r’ = cos   ± =  m ± (  +  /2) Relationships between higher derivatives also exist.

21. Some metrics on space of m.a. 1. Since boundary curve is recoverable, can reinterpret curve metrics as metrics on the medial axis. 2. Can take curve metric and apply to medial axis. 3. Can define cost function on “moves” on the axis.  Discrete structure on shape space.

22. Discrete structure (Kimia)

23. Why medial axis is good 1. Captures symmetries of shape. 2. Preserves intrinsic geometric quantities. 3. Allows for structural decomposition into parts. 4. Translates nicely to discrete setting.

24. Shape model IV: diffeomorphisms of the plane Model a shape by the map of the plane taking the unit circle to the boundary of the shape (modulo rigid motion). Mathematical formalism: Diffeomorphisms of the plane form a group G, which acts on the space of curves transitively. Therefore, we may identify the space of shapes with the space G modulo rigid motions.

25. Defining a metric We may define paths in shape space by defining paths in G/H. A metric is therefore given by minimizing the length of paths between two shapes. Kicker: definition of metric must respect group action. Image from David Mumford.

26. Oversimplification of metric… Idea: Take a small deformation of a curve , i.e.,  +  = (  1 +  1,  2 +  2 ), where ||  || ≤ . This defines a vector field on  ; define energy to be minimized as energy of that vector field. Glue these infinitesimals together to get from one shape to another. Reinterpret as an element of G. Check that resulting metric is left-invariant. End result: G is contained in L 2, and we look for admissible paths there, minimizing an energy over such admissible paths.

27. Example of geodesic Image of Faisel Beg, courtesy of David Mumford.

28. Origins of idea 1. D’arcy Thompson Shapes from the same class should be able to be deformed into each other. Therefore, metrics should be geodesics--lengths of shortest deformation path between the two shapes. 2. Grenander’s pattern theory: Classes of interest are made up of primitives. Group acts on these primitives following some set of rules (probabilistic). To understand class, one must understand primitives and actions on primitives.

29. Shape model V: conformal self- maps of S 1 Riemann mapping theorem guarantees the existence of a conformal map from the unit circle to any shape boundary. Add point at infinity and do the same for the exterior of the shape…then inverting one and composing gives a diffeomorphism from S 1 into itself. This diffeomorphism is unique, up to Möbius transformations of S 1. Fixing two points makes unique. Again find a G/H setting, where G is the group of diffeomorphisms of S 1 and H is the group of Möbius transformations.

30. Constructing the diffeomorphism Images taken from E. Sharon & D. Mumford, “2D-Shape Analysis using Conformal Mapping.”

31. Oversimplification of metric Want to measure amount of movement on unit circle--standard approach is through summing length changes of tangent vectors associated to mapping. Defines a periodic function f(  ), can look at Fourier coefficients of f, {a n }. Metric is the given by  (n 3 -n)|a n |.

32. Useful starting points: Curvelets: www.acm.caltech.edu/~emmanuel Medial axis: www.lems.brown.edu (Ben Kimia) www.lems.brown.edu www.stat.ucla.edu/~sczhu www.cs.ucdavis.edu/amenta Diffeomorphisms: www.cmla.ens-cachan.fr/Utilisateurs/younes cis.jhu.edu/people/faculty/mim Conformal mappings: www.math.utk.edu/~kens www.dam.brown.edu/people/eitans or mumford www.dam.brown.edu/people/eitans

33. The End Thank you!