1. Geometrically Stable Sampling for the ICP Algorithm
Natasha Gelfand
Stanford University
Leslie Ikemoto
UC Berkeley
Szymon Rusinkiewicz
Princeton University
Marc Levoy
9/17/2018

2. Two Flavors of ICP Point-point error Point-plane error
faster to converge pi qi ni pi qi
ICP picks a set of corresponding points and computes the transformation that aligns two meshes by minimizing the error metric derived from the distance between them
Point-point minimizes the distance between the points
Point-plane minimizes the distance between points on one mesh and tangent planes at the corresponding points at the other
Point-plane is a better approximation of true point-surface error when meshes are close to each other
Converges better because input surfaces can slide against each other
9/17/2018

3. Source of Failures Complex error landscape Many local minima
Usually have to start from a good guess
Point selection and error metric in minimization are key to convergence
Some shapes are particularly “difficult”
Noise
ICP is a local search algorithm, so it’s guaranteed to find the closest local minimum from its starting position.
The error landscape is often complex
Especially in the presence of noise in the input, there can be many local minima
Landscape depends on the choice of error metric
Point-point metric can have artificial local minima even if the true distance between the surfaces does not have a local minimum
Point-plane error does not create such artificial local minima
Point-plane error may have difficulties when features are small
And when there is noise in the input data
9/17/2018

4. Requirements for Error Metric
Planar and spherical areas with no features should not contribute to distance metric
“Lock and key” features should pull the surfaces to correct relative pose
Robust even as features get small and featureless regions noisy
Point-plane error metric tends to perform well in alignment since it allows two meshes to “slide” against each other
For example, two planes moving against each other should not incur a distance penalty, but they would with a point-point error metric.
In the point-plane error metric this would be a plateau in the error landscape
This is where the point-point error would lock the meshes, but point-plane has a plateau in the error landscape
If there are features on the meshes, they should pull them into alignment
But, since this in effect creates a shallow error landscape, it can suffer from slow convergence in the presence of noise
Small contributions from noisy regions, but since they are large compared to the features, the contributions add up and prevent the features from pulling the meshes together
9/17/2018

5. Solution Detect when parts of the input patch are prone to sliding
Use covariance analysis
Concentrate samples in “lock and key” areas to improve convergence
Use geometrically stable point-selection strategy
Similar to normal-space sampling [Rusinkiewicz & Levoy]
Detect when input has no “lock and key” features
Do not align that scan pair at all
Improves convergence of global registration [Ikemoto et.al]
Evenly distributing samples creates a shallow error landscape around the global minimum in the presence of geometry that can slide
Shallow landscape means lock and key features may be overwhelmed
Find which parts of the input are the lock and key features and sample there
Would also be nice if this method told us that two meshes cannot be aligned uniquely since their overlap doesn’t have any locks
Useful in global registration b/c we may get the alignment wrong due to sliding and it will then propagate through the entire set of scans
9/17/2018

6. Minimization Equation
Find (R, t) that minimize
Linearizing rotations:
find r=[rx ry rz]T, t=[tx ty tz]T
that minimize ni qi pi
9/17/2018

7. Analyzing the Metric (1)
Change in error from each point-pair:
For some transformations e=0:
“torque”
“force”
translation
perpendicular
to “force” P Q P Q
9/17/2018

8. Analyzing the Metric (2)
Change in error from each point-pair:
For some transformations e=0:
“torque”
“force”
rotation
perpendicular
to “torque” P Q Q P
9/17/2018

9. Covariance Matrix Aligning transform is given by Cx=b, where
C encodes the change in error when surfaces are moved from optimal alignment
Now that we looked at the contribution from a single point-pair, what happens when we put them all together?
Stack all torque-force vectors into 6x1 matrix F
Covariance of F gives how much the error changes when a transform is applied to the point set
Shows the error landscape around the current set of point-pairs
Directions in which the error landscape is shallow corresponds to sliding transformations
9/17/2018

10. Sliding Directions
Eigenvectors of C with small eigenvalues correspond to sliding transformations
3 small eigenvalues
2 translation
1 rotation
3 small eigenvalues
3 rotation
2 small eigenvalues
1 translation
1 rotation
Some examples of common shapes that exhibit sliding in one or more directions
1 small eigenvalue
1 rotation
1 small eigenvalue
1 translation
9/17/2018

11. Sample Selection
Goal of sampling is to produce covariance matrix C with a good condition number
Equally constrain all eigenvectors
Find points that well sample the small features on P
Since P and Q are similar in the overlap region, use both points and normals from P to compute C
Okay, we showed that we can detect if the given input meshes have large correlated featureless areas
Now we want to sample the features that will pull the meshes together
These are the same features that make the landscape deep around the global minimum
All transformations (eigenvectors) equally constrained
Analyze which features prevent P from sliding against itself
Since P and Q are similar where they overlap
Do not compute correspondences, use both points and normals from P
9/17/2018

12. Implementation (1)
Estimate C from sparse uniform sampling of input mesh
C = XLXT
6x6 matrix
6 eigenvectors
form basis for
the space of
transforms
9/17/2018

13. Implementation (2) For each point form
For each eigenvector x j compute
Magnitude of the constraint pi exerts on x j
Sort points in decreasing “order of influence” for each eigenvector p2 p3 p4 p1 p7 p5 p9 p8 p6 … x1 x2 x3 x4 x5 x6
Sorting is speeded up by binning
9/17/2018

14. Implementation (3)
Let be an estimate of how well eigenvector x j is constrained by already chosen points
Choose next pi from from list with the smallest corresponding t j
Most unconstrained eigenvector
Compute closest point qi in Q
Once enough points are chosen, solve for alignment and iterate
Notice that this algorithm is only an approximation: we never re-compute the matrix C
Although we do not look at Q when computing the covariance matrix and sorting the points we need to perform an overlap test before updating the totals
Test by computing for each point p, its closest point q \in Q and checking that it’s not on the boundary
9/17/2018

15. Sample Selection Small eigenvalues 9/17/2018
This patch was aligned to a copy of itself
Three unstable transformations: translation in XY plane and rotation around Z
It’s not guaranteed that two translations and rotation are going to be the small eigenvectors, any basis spanning that subspace may be picked, here we re-projected into translations are rotation to make the illustration simpler
9/17/2018

16. Sample Selection Large eigenvalues 9/17/2018
Only a few of the points in the corners need to be picked to ensure that the mesh does not rotate out of the plane
9/17/2018

17. Results (Planar Patches)
Without noise, any sampling works fine
Similar to normal-space sampling for planar regions
For the planes example
Condition number under uniform sampling = 66.1
Condition number after stable sampling = 3.7 (30% of the points were selected)
9/17/2018

18. Results (Planar Patches)
25 iterations to converge
Naïve implementation about 3 times slower per iteration than uniform sampling
uniform
sampling
stable
sampling
FUR:
25 iterations to converge while sampling 10% of the points
Each iteration is 5 second for stable sampling and 1.5 for uniform
But our implementation can be improved, most obviously by caching closest points from the stability computation stage into the alignment stage
9/17/2018

19. Results (Spherical Patches)
Normal-space sampling cannot align this
Because the sliding is rotational, normal space sampling will not align this
In the sphere, all normals are well distributed so normal-space and uniform are going to behave similarly
Initial condition number = 26.1
Final condition number = 4.1
9/17/2018

20. Conclusions
Point-plane error metric can have shallow error landscape when registering shapes with small features
Noise adds many small local minima
These conditions can be detected by analyzing the covariance matrix of the minimization equation
Eigenvectors with small eigenvalues give sliding directions
Stable sampling improves these landscapes by equally constraining all eigenvectors
Selects point samples inside the features
Allows ICP to align “difficult” input
9/17/2018

21. Limitations: Noise
The algorithm can fail if input data is particularly noisy
Smoothing can help
One of the failure modes of the algorithm is in the presence of noise
The algorithm can favor noisy areas because they will look like they have large influence on the covariance matrix
Noise looks like good features to sample
Smoothing can help, but only is the size of the noise is smaller than the size of the features
The groove in this picture is 10mm wide and the mean height of the noise is 0.2 mm
9/17/2018

22. Limitations: Noise Success depends on relative size of features
If the size of the noise is size of the features are similar, we’ll smooth both away and the sampling will look random
The groove in this picture is 1 mm wide and the mean height of the noise is 0.2 mm
The groove disappears after 6 iterations of simple smoothing by averaging with the neighbors
9/17/2018

23. Extensions and Future Work
Use stability analysis to weigh mesh pairs in global registration
A hierarchical method for aligning warped meshes [Ikemoto et al]
Stable sampling for global registration
Simultaneous covariance analysis of the entire set of scans
Weighing eigenvectors based on the area outside the overlap
Large vs. small leverage on the entire scan
9/17/2018

24. Extensions and Future Work
Use of stability analysis as shape descriptor
Segmentation into areas with similar sliding transformations
9/17/2018