1. 1 GEOMETRIE Geometrie in der Technik H. Pottmann TU Wien SS 2007

2. 2 GEOMETRIE Overview Registration algorithms Registration with known correspondences in 2D and 3D Registration with unknown correspondences

3. 3 GEOMETRIE Registration problems Rigid registration; matching 2D or 3D systems with Euclidean congruence transformation: Merging partial scans of the same object (3D model acquisition) Comparing measurement data of an object to the CAD model of the same object (industrial inspection)

4. 4 GEOMETRIE registration 0° 90° 180° 270° registation = merge individual scans into a single data set.

5. 5 GEOMETRIE CAD model of a technical object Industrial inspection

6. 6 GEOMETRIE Industrial inspection Image of a manufactured object (synthetic data) Point cloud obtained by laser scanning (synthetic data) Point cloud with transparent surface (for visualization)

7. 7 GEOMETRIE Industrial Inspection Input to the registration algorithm Output of the registration algorithm

8. 8 GEOMETRIE Color coding of deviations Industrial inspection

9. 9 GEOMETRIE Nonrigid registration There are many applications, e.g. in image processing, where one wants to merge images or data sets with help of nonrigid transformations Panorama images Merging images of different modality or images of the same nonrigid objects at different times (brain surgery, before and during the surgery…) … We will discuss here only affine registration.

10. 10 GEOMETRIE Simple registration problems Registration problems of 2 systems with known correspondences possess an explicit solution: Probably the simplest registration problem is affine registration of 2 systems in R n with known correspondences; amounts to the solution of a linear system A little harder is Euclidean registration of 2 systems with known correspondences; there is still an explicit solution; eigenvalue problem

11. 11 GEOMETRIE Harder problems The following problems cannot be solved explicitly and need to be treated with help of iterative optimization algorithms: Simultaneous Euclidean registration of N systems with known correspondences Euclidean or affine registration of 2 systems without correspondences Simultaneous Euclidean or affine registration of N systems without correspondences

12. GEOMETRIE 12 Registration of two systems with known correspondences

13. 13 GEOMETRIE Affine registration Given: two point sets X={x 1,…,x N }, Y={y 1,…,y N } in R n ; x i corresponds to y i with confidence value w i >0 Find: optimal alignment by an affine map x‘=a+R. x via minimization of F(a,R)= i w i (x i ‘-y i ) 2

14. 14 GEOMETRIE Affine registration The objective function is quadratic in the n(n+1) unknowns a,R: F(a,R)= i w i (x i ‘-y i ) 2 =  i w i (a+R. x i -y i ) 2 Thus, we just have to solve a linear system. Euclidean registration is not so simple because of the constraints on R; in either case the following property holds:

15. 15 GEOMETRIE barycenter is mapped to barycenter For registration with an affine map or a special case of it (similarity, congruence transformation), the minimizer of F maps the barycenter s X of X to the barycenter s Y of Y: s X =(1/W) i w i x i s Y =(1/W) i w i y i, W=  i w i

16. 16 GEOMETRIE Barycenter to barycenter Proof: partial derivative of F with respect to a yields:  i w i (a+R. x i -y i )=0 Exactly if s X is mapped to s Y, i.e., s Y =a+R. s X, this equation holds:  i w i (s Y -R. s X +R. x i -y i )=  i w i (s Y -y i ) + R.  i w i (x i -s X )=0. Note:  i w i (s Y -y i )=0,  i w i (x i -s X )=0

17. 17 GEOMETRIE Euclidean registration Euclidean registration in R 2 is still simple, see course notes. In the following, we study Euclidean registration in R 3. We give an explicit solution (B.K.P. Horn, 1987) and an iterative solution algorithm; the latter will then be extended to N systems.

18. 18 GEOMETRIE Euclidean registration; explicit solution At first, we map the barycenters of X and Y onto each other and choose this point as origin; the transformed point sets are again denoted by X and Y; we set w i =1. The remaining problem: compute a rotation x‘=R. x, such that F(R)=  i (R. x i -y i ) 2 is minimized

19. 19 GEOMETRIE Euclidean registration; explicit solution We find: F(R)=  i (R. x i -y i ) 2 =  i (x i T. R T -y i T ). (R. x i -y i )= (use R T. R=I)  i (x i T. x i +y i T. y i -2y i T. R. x i )  min  i (x i T. x i +y i T. y i ) is given and positive, F is positive; hence, the minimizer must be a maximizer of f=  i y i T. R. x i

20. 20 GEOMETRIE Euclidean registration; explicit solution f=  i y i T. R. x i max: Using the quaternion representation of the rotation, R. x i =a*x i a f can be shown to be a quadratic form in a (now back to matrix notation) f=a T. M. a (see course notes) Hence, we have to maximize f under the constraint a T. a=1 eigenvalue problem, normalized eigenvector to the largest eigenvalue of M is the solution.

21. 21 GEOMETRIE Iterative solution Basic idea: in each iteration step linearize the motion, i.e. estimate the actual displacement of a point x i with help of its velocity vector and minimize objective function with the linearized constraint After motion estimation, do not displace the points x i with v(x i ), but with an appropriate rigid body motion

22. 22 GEOMETRIE Iterative solution; step 1 With linearized motion v(x)=c+c  x, the objective function is quadratic in (c,c): F=  i (x i ‘-y i ) 2 =  i (x i +c+c  x i -y i ) 2 (c,c) are computed as solutions of a linear system.

23. 23 GEOMETRIE Iterative solution Transformation which maps x i to x i + v(x i ) is an affine map and not a rigid Euclidean transformation  use the uniform helical motion that is uniquely determined by the velocity vector field represented by

24. 24 GEOMETRIE Iterative solution: step 2 Applying a rotation about the axis of the helical motion through an angle of  = arctan(|c|) and a translation parallel to this axis by the distance p brings the points x i close to the tips x i + v(x i ) of the velocity vectors used in the objective function F iteration  converges to local minimum

25. 25 GEOMETRIE Recall: axis and pitch of a helical motion From the vector the axis and the pitch p of the underlying helical motion are calculated by: a … direction vector of axis A … moment vector of axis A [independent of the choice of q since q  a = (q+ a)  a ] A a o q

26. 26 GEOMETRIE Optimization perspective Constrained minimization of a quadratic function in R 12 Constraint manifold is 6- dimensional manifold M 6, describing rigid body motions. Proposed algorithm is a projected gradient algorithm and requires stepsize control! M6M6 Minimizer for affine registration Minimizer for Euclidean registration T6T6 Step 1

27. GEOMETRIE 27 Simultaneous registration of N systems with known correspondences

28. 28 GEOMETRIE Registration with correspondences Assume that partial scans from an object have been taken; these are partially overlapping clouds of points From additionally taken ditigal photos, correspondences shall be known (with some confidence value) Merge the clouds into a single cloud taking the correspondences into account

29. 29 GEOMETRIE tunnel data: initial position

30. 30 GEOMETRIE tunnel data: final position

31. 31 GEOMETRIE initial position of two point clouds

32. 32 GEOMETRIE final position of two point clouds

33. 33 GEOMETRIE initial position of point clouds

34. 34 GEOMETRIE final position of point clouds

35. 35 GEOMETRIE Registration of N systems Given point clouds, viewed as rigid systems  i ; use only those points in a cloud which correspond to overlapping region with another cloud Input: pairs (x i,y i ) of corresponding points; knowledge of indices j,k of the sytems to which the points belong: x i  j, y i  k Confidence value w i for the match x i  y i

36. 36 GEOMETRIE Relative kinematics Assume that we know an approximate solution (=initial position for the iterative algorithm; e.g. found interactively) Fix one system, and call it  0 Linearize the motions; velocity field of motion  l / 0 is of the form v l0 (x i )=c l +c l  x i

37. 37 GEOMETRIE Simultaneous registration If points (x i,y i ) are displaced with their corresponding velocity vectors, their distance after displacement is given by Q(x i,y i )=(x i +v j0 (x i )-y i -v k0 (y i )) 2 = (x i -y i +c j +c j  x i - c k -c k  x i ) 2 This is a quadratic function in the unknowns c j,c j,c k,c k.

38. 38 GEOMETRIE Simultaneous registration Iterative algorithm, which performs two steps in each iteration Minimize F= i w i Q(x i,y i ); this is a quadratic function in the unknowns (c l,c l ) of the velocity fields; solve the arising linear system For all moving systems  l, perform motion  l / 0, indicated by the velocity field, as discussed for two systems. Use an appropriate common stepsize for all these motions

39. GEOMETRIE 39 Registration of 2 systems without correspondences

40. 40 GEOMETRIE Industrial inspection

41. 41 GEOMETRIE A Registration Problem Without Correspondences Given: a large number of 3D data points obtained by some 3D measurement device (laser scan, light sectioning, …) from a technical object CAD model of this workpiece Goal: register point cloud to the CAD model check for manufacturing errors, classify deviations, color coding of results, error estimates, … Possible solutions: ICP Alternative algorithms

42. 42 GEOMETRIE The Iterative Closest Point (ICP) algorithm Besl and McKay, 1992: Iteration to each point x i of the data point cloud compute closest point y i on the CAD model Consider y i as corresponding to x i and solve the registration problem with these correspondences (calculation of rigid motion in each iteration step is an eigenvalue problem); call new points x i ’ again x i yiyi xixi

43. 43 GEOMETRIE ICP algorithm algorithm converges monotonically to a local minimum basically a double projection; local linear convergence uses squared distance function to footpoints (should only be used in the far field) Rigid body motions Deformations which map point cloud precisely onto the CAD model surface

44. 44 GEOMETRIE Initial position and improvements If data point cloud covers the entire CAD surface, we can align the barycenters and the inertia axes (principal component analysis; eigenvectors of the covariance matrix; see fitting of planes) Otherwise, the problem of finding an initial position is hard and not yet well solved Various improvements of ICP (use other correspondences, speed up footpoint computation, make algorithm symmetric,…)

45. 45 GEOMETRIE Alternative registration algorithm: TDM Chen and Medioni; use squared distances to tangent plane at footpoint (TDM: Tangent Distance Minimization) TDM uses a much better approximation of the squared distance function in the near field! We can use the approach based on the linearization of the motion (linear velocity vector field)

46. 46 GEOMETRIE Registration with instantaneous kinematics xixi nini didi v(xi)v(xi) x i... data points velocity vector field v(x i ) of a one parameter motion: determine distance d i to surface and surface normal n i in the corresponding surface point y i. minimize quadratic in unknowns yiyi sum of squared distances of to tangent planes at closest points

47. 47 GEOMETRIE TDM TDM is the same as a Gauss-Newton algorithm for the solution of a nonlinear least squares problem Gauss-Newton converges quadratically for a zero residual problem and works well for small residual problems In Computer Vision seen as a variant of ICP; from the optimization perspective, this is not just a variant of ICP

48. 48 GEOMETRIE Comparison of ICP and TDM

49. 49 GEOMETRIE Comparison of ICP and TDM

50. 50 GEOMETRIE Comparison of TDM and ICP

51. 51 GEOMETRIE Registration with SDM Goal: compute motion minimizing Use local quadratic approximants of the squared distance function to the model shape Use linearization of the motion (estimate displacement by a velocity vector field)

52. 52 GEOMETRIE Registration with SDM In each iteration, minimize local quadratic model of the objective function with Due to the linearized motion we get quadratic convergence only for a zero residual problem. TDM

53. 53 GEOMETRIE SDM2 A registration algorithm which has local quadratic convergence even for a larger residual can be formulated with help of a second order approximant of a rigid body motion

54. 54 GEOMETRIE Local convergence, small residual

55. 55 GEOMETRIE Local convergence; large residual

56. 56 GEOMETRIE Local convergence; partial match