1. CSE554Laplacian DeformationSlide 1 CSE 554 Lecture 8: Laplacian Deformation Fall 2012

2. CSE554Laplacian DeformationSlide 2 Review Alignment – Registering source to target by rotation and translation Rigid-body transformations Methods – Aligning principle directions (PCA) – Aligning corresponding points (SVD) – Iterative improvement (ICP) Combines PCA and SVD Input After PCA After ICP Source Target

3. CSE554Laplacian DeformationSlide 3 Non-rigid Registration Rigid alignment cannot account for shape variance Non-rigid deformation can give a better fit After non-rigid deformationRigid alignment Source Target

4. CSE554Laplacian DeformationSlide 4 Non-rigid Registration A minimization problem – Minimizing the distance between the deformed source and the target “Fitting term” – Minimizing the distortion to the source shape “Distortion term”

5. CSE554Laplacian DeformationSlide 5 Intrinsic vs. Extrinsic Intrinsic methods – Deforms points on the source curve/surface – App: boundary curve or surface matching Extrinsic methods – Deforms all points on and interior to the source curve/surface – App: image or volume matching

6. CSE554Laplacian DeformationSlide 6 Laplacian-based Deformation An intrinsic method – Simple, efficient, producing reasonable results Preserving local shape features – Widely used in graphics applications for interactive deformation Reference: “Laplacian surface editing”, by Sorkine et al., 2004 (citation ~ 500)

7. CSE554Laplacian DeformationSlide 7 Input – Source with n points: p 1,…,p n Let the first m points be “handles” – Target location of handles: q 1,…,q m Output – Deformed locations of source points: p 1 ’,…,p n ’ Setup p 1 =q 1 p 3 =q 3 q2q2 p2p2 An example with 3 target points, two of which are stationary (red) Source Deformed When deforming the source to fit a target shape: m=n and q i is the point on the target closest to p i.

8. CSE554Laplacian DeformationSlide 8 Overview Finding deformed locations p i ’ that minimize: – E f : fitting term Measures how close are the deformed source to the target – E d : distortion term Measures how much the source shape is changed

9. CSE554Laplacian DeformationSlide 9 Fitting Term Sum of squared distances to target handle locations q2q2 p2p2

10. CSE554Laplacian DeformationSlide 10 Distortion Term Q: How to measure shape? A: By “bumpiness” at each vertex – Laplacian: vector from the centroid of neighbors to the vertex Recall that in fairing, we reduced this vector to “smooth out” bumps A linear operator over point locations where N i are indices of neighboring vertices of p i

11. CSE554Laplacian DeformationSlide 11 Distortion Term Minimizing changes in Laplacians during deformation – Over all source points  i : Laplacian at p i before deformation

12. CSE554Laplacian DeformationSlide 12 Putting Together Finding deformed locations p i ’ that minimize: – A quadratic equation in terms of variables (p ix ’, p iy ’, p iz ’) q i,  i are constants L[] is a linear operator

13. CSE554Laplacian DeformationSlide 13 Quadratic Minimization A general form of quadratic minimization: – There are s variables: x=(x 1,…,x s ) T – Each a 1,…, a k is a length-s column vector (linear coefficients) – Each b 1,…, b k is a scalar (constant coefficients) – k should be greater than s (so that the problem is over-constrained)

14. CSE554Laplacian DeformationSlide 14 Quadratic Minimization To solve: Re-write in matrix form: whereis a k by s matrix is a length-k vector

15. CSE554Laplacian DeformationSlide 15 Quadratic Minimization The minimizer is where the partial derivatives are all zero – To solve for x in this equation: Taking matrix inverse (good for small s, but numerically unstable for large s) Using specialized linear system solver (LinearSolve in Mathematica, TNT/LAPACK in C)

16. CSE554Laplacian DeformationSlide 16 Quadratic Minimization Re-writing our minimization in the general form – In 2D, there are 2n variables: x = (p 1x ’,…, p nx ’, p 1y ’,…, p ny ’ ) T In 3D, there are 3n variables – We will next re-write each quadratic term in 2D as (a i x-b i ) 2 Can be extended easily to 3D

17. CSE554Laplacian DeformationSlide 17 Quadratic Minimization The a i and b i in the fitting term – There are 2m quadratic terms – In the first set of m terms: For i=1,…,m, b i =q ix, a i contains all zero, except its (i)th entry is 1. – In the second set of m terms: For i=1,…,m, b i+m =q iy, a i+m contains all zero, except its (i+n)th entry is 1

18. CSE554Laplacian DeformationSlide 18 Quadratic Minimization The a i and b i in the fitting term – There are 2m quadratic terms – Example with 3 vertices and 2 fitting constraints (n=3; m=2):

19. CSE554Laplacian DeformationSlide 19 Quadratic Minimization The a i and b i in the distortion term: – There are 2n quadratic terms – The first set of n terms: For i=1,…,n, a i is all zero except the (i)th entry is 1, the (j)th entries are -1/|N i | for all j  N i, and b i =  ix – The second set of n terms: For i=1,…,n, a i+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are -1/|N i | for all j  N i, and b i+n =  iy

20. CSE554Laplacian DeformationSlide 20 Quadratic Minimization The a i and b i in the distortion term: – There are 2n quadratic terms – Example with 3 vertices (n=3):

21. CSE554Laplacian DeformationSlide 21 Summary Compute Laplacians (  i ) Construct coefficients ( a i, b i ) – Stick them into matrices (A,B) Solve (x) –

22. CSE554Laplacian DeformationSlide 22 Results Deformed A small deformation

23. CSE554Laplacian DeformationSlide 23 Results Deformed A larger deformation

24. CSE554Laplacian DeformationSlide 24 Results Stretching Deformed

25. CSE554Laplacian DeformationSlide 25 Results Deformed Shrinking

26. CSE554Laplacian DeformationSlide 26 Results Rotation Deformed

27. CSE554Laplacian DeformationSlide 27 Discussion Limitations – Local features are “skewed”, and they don’t scale with the model Reason: Laplacian changes with rotation or scale – Two bumps that differ by rotation or scale have different Laplacians Which will be penalized by our distortion term

28. CSE554Laplacian DeformationSlide 28 A Better Distortion Term Not penalizing rotation and scaling of local features – Transforming the original Laplacian vectors before comparing to the deformed Laplacians T i is a matrix that describes how the local shape around p i is deformed

29. CSE554Laplacian DeformationSlide 29 Key Questions How to represent transformations as matrices? How to compute T i ? We will focus in the derivations of the 2D case – 3D results will be briefly presented at the end

30. CSE554Laplacian DeformationSlide 30 Transformation Matrices (2D) Homogeneous coordinates – A 2D point: (x,y,1) A 2D vector: (x,y,0) – A 3D point: (x,y,z,1) A 3D vector: (x,y,z,0)

31. CSE554Laplacian DeformationSlide 31 Transformation Matrices (2D) Translation – Cartesian coordinates: vector addition – Homogeneous coordinates: matrix product

32. CSE554Laplacian DeformationSlide 32 Transformation Matrices (2D) Isotropic scaling – Cartesian coordinates: vector scaling – Homogeneous coordinates: matrix product

33. CSE554Laplacian DeformationSlide 33 Transformation Matrices (2D) Rotation – Cartesian coordinates: matrix product – Homogeneous coordinates: matrix product

34. CSE554Laplacian DeformationSlide 34 Transformation Matrices (2D) Summary of elementary similarity transformations – To combine transformations: take the product of these matrices Translation by vector v Scaling by scalar s Rotation by angle 

35. CSE554Laplacian DeformationSlide 35 Similarity Transforms (2D) General similarity transformations – The product of any set of elementary matrices can be written this way – Any choice of (a, w, t x, t y ) can be written as a sequence of rotation, isotropic scaling and translation Note that a and w can’t be both zero

36. CSE554Laplacian DeformationSlide 36 Computing T i (2D) Suppose we know the deformed locations p i ’ Compute T i as the similarity transform that best fits the neighborhood of p i to that of p i ’

37. CSE554Laplacian DeformationSlide 37 Computing T i (2D) Suppose we know the deformed locations p i ’ Compute T i as the similarity transform that best fits the neighborhood of p i to that of p i ’ This is a quadratic minimization problem for entries of T i – E.g., a, w, t x, t y

38. CSE554Laplacian DeformationSlide 38 Computing T i (2D) The matrix form of the minimization is: where is a 2|N i |+2 by 4 matrix, and N i ={i1, i2,…} are indices of neighboring vertices of p i

39. CSE554Laplacian DeformationSlide 39 Computing T i (2D) By quadratic minimization: – Linear expressions of variables (p ix ’, p iy ’)

40. CSE554Laplacian DeformationSlide 40 Distortion Term (2D) Two parts of each distortion term: – Transformed Laplacian: – Laplacian of the deformed locations: where is a 2 by 2|N i |+2 matrix

41. CSE554Laplacian DeformationSlide 41 Distortion Term (2D) Putting together: – They form 2n quadratic terms (a i x-b i ) 2 for x = (p 1x ’,…, p nx ’, p 1y ’,…, p ny ’ ) T All b i are zero Each a i can be extracted from H where are its rows and

42. CSE554Laplacian DeformationSlide 42 Results (2D) New distortion term Old distortion term

43. CSE554Laplacian DeformationSlide 43 Results (2D) New distortion term Old distortion term

44. CSE554Laplacian DeformationSlide 44 Results (2D) New distortion term Old distortion term

45. CSE554Laplacian DeformationSlide 45 Results (2D) New distortion term Old distortion term

46. CSE554Laplacian DeformationSlide 46 Registration Use nearest neighbors as corresponding target locations – Assuming the source is already close to the target Iterative closest point (ICP) – 1. For each point on the source, assign its closest point on the target as its corresponding point. Compute Laplacian-based deformation. A threshold on the closest distance can be used to throw away unlikely correspondences – 2. Repeat step (1) until a termination criteria is met. Maximum iteration or minimum RMSD improvement

47. CSE554Laplacian DeformationSlide 47 Result After rigid alignment1 iteration of Laplacian 7 iterations of LaplacianOverlaying all curves

48. CSE554Laplacian DeformationSlide 48 Result Weighting the distortion term small w medium w large w

49. CSE554Laplacian DeformationSlide 49 Similarity Transforms (3D) Elementary transformation matrices – To perform a sequence of transformations: take the product of these matrices Translation by vector v Scaling by scalar s Rotation by angle  around X axis

50. CSE554Laplacian DeformationSlide 50 Similarity Transforms (3D) General similarity transformations in 3D – Approximates the product of a set of elementary matrices Up to a small rotation angle May introduce skewing for large rotations

51. CSE554Laplacian DeformationSlide 51 Computing T i (3D) Assuming known deformation, by quadratic minimization: – Linear expressions of the deformed points p i ’ C is a 3|N i |+3 by 7 matrix where

52. CSE554Laplacian DeformationSlide 52 Distortion Term (3D) Constructing transformed Laplacian: where