Shape reconstruction and inverse problems Lecture 9 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 1

Measurement Projection Shape space Measurement space

Reconstruction ? Find Given Shape space Measurement space 4

Reconstruction Shape space Measurement space 5

Given a (possibly noisy) measurement of unknown shape Inverse problems Given a (possibly noisy) measurement of unknown shape Reconstruct the shape by minimizing the distance between given measurement and measurement obtained from shape 6

Inverse problems 7

Many shapes have the same measurement! Ill-posedness Many shapes have the same measurement! Shape space Measurement space 8

Prior knowledge We know that the measurements come from deformations of the same object! 9

Regularization Deformations of the dog shape Shape space Measurement space 10

Regularization Prior Shape space Measurement space 11 Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 11

Inverse problems with intrinsic prior Error Regularization Prior is given on the intrinsic geometry of the shape (intrinsic prior) Error = distance between measurements Regularization = intrinsic distance from prior shape Prior shape is a deformation of the shape we need to reconstruct Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 12

Solution of inverse problems with intrinsic prior Optimization variable: shape , represented as a set of coordinates Possible initialization: prior shape Gradients of and w.r.t. are required Does it sound familiar? Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 13

? Extrinsic dissimilarity Intrinsic dissimilarity

Joint similarity as inverse problem Measurement space = shape space Identity projection operator = intrinsic distance on shape space = extrinsic distance on measurement space Prior = the shape itself 15

Computation of the regularization term Assume shape = deformation of prior with the same connectivity Trivial correspondence Compute L2 distortion of geodesic distances and gradient is a fixed (precomputed) matrix of geodesic distances on depends on the variables (must be updated on every iteration) A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

Computation of using Dijkstra’s algorithm Same approach as in joint similarity Compute and fix the path of the geodesic is a matrix of Euclidean distances between adjacent vertices is a linear operator integrating the path length along fixed path At each iteration, only changes Computation of is straightforward A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

Inconsistency of Dijkstra’s algorithm Dijkstra/Analytic 1.1 1.05 FMM/Analytic 1 Number of points Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Computation of using Fast Marching Standard FMM FMM with derivative propagation Distance update Distance update Distance derivative update Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Shape-from-X Silhouette Sparse points Image Shading Stereo 20

Denoising Measurement space = shape space Identity projection operator = intrinsic distance on shape space = extrinsic distance on measurement space Noisy measurement 21

Denoising Unknown shape Prior Measurement Reconstruction (without prior) Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 22

Denoising Unknown shape Prior Measurement Reconstruction (with prior) Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 23

Bundle adjustment Clean measurement Shape Noisy measurement 24 Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 24

Bundle adjustment Measurement space of 2D point clouds Projection operator (assuming known correspondence) Noisy measurement 25

Bundle adjustment Unknown shape Prior Measurement Reconstruction (without prior) Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 26

Bundle adjustment Unknown shape Prior Measurement Reconstruction (with prior) Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009 27

Shape-to-image matching Measured image Reconstruction Salzmann, Pilet, Ilic, Fua, 2007 28