1. 3D Global Registration

2. The Problem Given: n scans around an objectGiven: n scans around an object Goal: align them allGoal: align them all First attempt: ICP each scan to one otherFirst attempt: ICP each scan to one other

3. The Problem Want method for distributing accumulated error among all scansWant method for distributing accumulated error among all scans

4. Approach #1: Avoid the Problem In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan)In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan) Align all other scans to this “anchor”Align all other scans to this “anchor” Disadvantage: not always practical to obtain anchor scanDisadvantage: not always practical to obtain anchor scan

5. Approach #2: The Greedy Solution Align each new scan to all previous scansAlign each new scan to all previous scans Disadvantages:Disadvantages: – Order dependent – Doesn’t spread out error

6. Approach #3: The Brute-Force Solution While not converged:While not converged: – For each scan: For each point:For each point: –For every other scan Find closest pointFind closest point – Minimize error w.r.t. transforms of all scans Disadvantage:Disadvantage: – Solve ( np )  ( np ) matrix equation, where n is number of scans and p is number of points per scan

7. Approach #3a: Slightly Less Brute-Force While not converged:While not converged: – For each scan: For each point:For each point: –For every other scan Find closest pointFind closest point Minimize error w.r.t. transform of this scanMinimize error w.r.t. transform of this scan Faster than previous method (matrices are p  p )Faster than previous method (matrices are p  p )

8. Graph Methods Many globalreg algorithms create a graph of pairwise alignments between scansMany globalreg algorithms create a graph of pairwise alignments between scans Scan 1 Scan 5 Scan 4 Scan 3 Scan 2 Scan 6

9. Pulli’s Algorithm Perform pairwise ICPs, record sample (e.g. 200) of corresponding pointsPerform pairwise ICPs, record sample (e.g. 200) of corresponding points For each scan, starting w. most connectedFor each scan, starting w. most connected – Align scan to existing set – While (change in error) > threshold Align each scan to othersAlign each scan to others All alignments during globalreg phase use precomputed corresponding pointsAll alignments during globalreg phase use precomputed corresponding points

10. Sharp et al. Algorithm Perform pairwise ICPs, record only optimal rotation/translation for eachPerform pairwise ICPs, record only optimal rotation/translation for each Decompose alignment graph into cyclesDecompose alignment graph into cycles While (change in error) > toleranceWhile (change in error) > tolerance – For each cycle: Spread out error equally among all scans in the cycleSpread out error equally among all scans in the cycle – For each scan belonging to more than 1 cycle: Assign average transform to scanAssign average transform to scan

11. Lu and Milios Algorithm Perform pairwise ICPs, record optimal rotation/translation and covariance for eachPerform pairwise ICPs, record optimal rotation/translation and covariance for each Least squares simultaneous minimization of all errors (covariance-weighted)Least squares simultaneous minimization of all errors (covariance-weighted) Requires linearization of rotationsRequires linearization of rotations – Worse than the ICP case, since don’t converge to (incremental rotation) = 0

12. Open Questions in Global Registration Best way to do correctly-weighted globalreg without linearizing rotations?Best way to do correctly-weighted globalreg without linearizing rotations? How to prevent bias (if many scans in one area, few scans in another)?How to prevent bias (if many scans in one area, few scans in another)? Robust outlier detectionRobust outlier detection – For graph methods, pairwise ICP often automated – One bad ICP can throw off the entire model

13. Bad ICP in Globalreg One bad ICP can throw off the entire modelOne bad ICP can throw off the entire model Correct Globalreg Globalreg Including Bad ICP

14. Huber’s Automatic Modeling System Start with unaligned cansStart with unaligned cans Generate candidate pairwise alignments using spin imagesGenerate candidate pairwise alignments using spin images Incrementally add scans, checking consistencyIncrementally add scans, checking consistency – ICP error in overlapping regions – Free space consistency – Occupied space consistency

15. Spin Images Johnson and HebertJohnson and Hebert “Signature” that captures local shape“Signature” that captures local shape Similar shapes  similar spin imagesSimilar shapes  similar spin images

16. Computing Spin Images Start with a point on a 3D modelStart with a point on a 3D model Find (averaged) surface normal at that pointFind (averaged) surface normal at that point Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangentsDefine coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents Express other points (within some distance) in terms of the new coordinatesExpress other points (within some distance) in terms of the new coordinates

17. Computing Spin Images Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal:Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal:

18. Computing Spin Images

19. Spin Image Parameters Size of neighborhoodSize of neighborhood – Determines whether local or global shape is captured – Big neighborhood: more discriminatory power – Small neighborhood: resistance to clutter Size of bins in histogram:Size of bins in histogram: – Big bins: less sensitive to noise – Small bins: captures more detail, less storage

20. Using Spin Images for Identification Range Image Model in Database

21. Using Spin Images for Identification Detected Models

22. Using Spin Images for Alignment Not robustNot robust

23. Verification – Overlap Distance Views 1 and 2: 48%, 2.1 mm Views 1 and 9: 15%, 3.1 mm

24. Verification – Visibility Consistency consistent surfaces S1S1S1S1 S2S2S2S2 C1C1C1C1 C2C2C2C2 surfaces are close free space violation S1S1S1S1 S2S2S2S2 C1C1C1C1 C2C2C2C2 S 2 blocks S 1 occupied space violation S1S1S1S1 S2S2S2S2 C1C1C1C1 C2C2C2C2 S 1 is not observed

25. Automatically Modeling the Angel top view connectivity after local verification horizontal slice 1 4 9 5 6 8 11 12 14 15 13 10 2 3 7 1 4 9 5 6 8 11 121415 13 10 2 3 7 initial model (bad matches in red)

26. Inferring New Pose Relations 1 4 9 5 6 8 11 12 14 15 13 10 2 3 7 30% overlap 1% overlap 1 9 5 6 8 11 121415 13 10 2 3 4 7

27. horizontal slice The Final Angel Model top view