Segmentation into Planar Patches for Recovery of Unmodeled Objects Kok-Lim Low COMP Computer Vision 4/26/2000

2 The Big Picture Work by Marjan Trobina –“From planar patches to grasps: a 3-D robot vision system handling unmodeled objects.” Ph.D. thesis, ETHZ, Overview of whole system acquire range images from 2 or more views segment into planar patches generate object hypotheses compute grasping points robot arm

3 Setup

4 Segmentation into Planar Patches Objectives –extract planar patches from range images in a robust way –just sufficient info for recovery of unmodeled objects for computing grasping points Can be viewed as data compression –transforming range image to just a few parameters

5 Related Work 1) Edge-based segmentation –detect surface discontinuities –find closed edge chains –not robust against noisy data 2) Split-and-merge paradigm –tessellation of image –using quadtrees or Delaunay triangles

6 Related Work 3) Clustering –map data to feature space –find clusters –points in feature space have no image-space connectivity 4) Region growing –grow region until approximation error too large –order dependent, result usually far from optimal 5) Recover-and-select paradigm

7 Recover-and-Select Paradigm (RS) Originally proposed by Leonardis Very robust against statistical noise and outliers Consists of 2 intertwined stages: model recovery model selection iterate until remaining models are completely grown output input

8 Overview of RS Algorithm Model recovery –regularly place seeds –grow statistically consistent seeds independently –fit a plane (model) to each region –stop growth of region if planar fit is lousy –stop growth of region if no compatible points can be added Model selection –select some recovered (plane) models –minimize number of selected models while keeping approximation error low

9 Planar Fitting Planar patch parameterized by a 1, a 2, a 3 in f(X, Y) = a 1 X + a 2 Y + a 3 = Z Distance function from range data point r(m) to planar patch d 2 (m) = ( r(m) – f(m) ) 2

10 Planar Fitting Approximation error for set D of n points is For each set of points D, minimize  by Linear Least Squares method to obtain plane parameters a 1, a 2, a 3.

11 Model Recovery Place seeds in regular grid of 7x7 windows Define model acceptance threshold T Grow seed if its  < T (statistically consistent) Define compatibility constraint C Add adjacent point m to patch if d 2 (m) < C (compatible) Stop when   T or no compatible point can be added Output is a set of overlapping planar patch models

12 Model Selection Select smallest number of models while keeping approximation error small Objective function (to be maximized) for model s i F(s i ) = K 1 n i – K 2  i – K 3 N i where n i = | D |  i = approximation error of model s i N i = number of parameters in model s i

13 Model Selection Objective function for M models wherep = [ p 1...p M ] and p i = 0 or 1 c ii = K 1 n i – K 2  i – K 3 N i c ij = ( –K 1 | D i  D j | + K 2  ij ) / 2 Use greedy algorithm to find vector p so that F(p) is near to maximum

14 Example

15 Result

16 Result

17 Multiresolution Recover-and-Select (MRS) In RS, many seeds are grown and then discarded MRS uses hierarchical approach to reduce waste Basic idea –build an image pyramid –apply standard RS on coarsest image –selected patches are projected to the next finer level and used as seeds for the new level –start new seeds on the unprojected regions in the next finer level Speedup of 10 to 20 times

18 Result

19 Viewpoint Invariant Segmentation Range images from different viewpoints A planar patch extracted from different views should have same parameters and error measure Modifications to model recovery stage: –project data points into direct 3-D space prior to the segmentation –minimizing the orthogonal distance to the plane

20 Viewpoint Invariant Segmentation Planar patch now parameterized by a 1, a 2, a 3, a 4 in f(X, Y, Z) = a 1 X + a 2 Y + a 3 Z + a 4 = 0 Distance function from 3-D range data point M to planar patch d  2 (M) = f(M) 2

21 Viewpoint Invariant Segmentation Approximation error  = 1 where 1 is the smallest eigenvalue of the covariance matrix of the n points in the patch The normal (a 1, a 2, a 3 ) of the planar patch is the eigenvector with eigenvalue 1

22 Postprocessing create explicit patch boundary description post-processing to clean edge classify patches as “true planes” or “curved patches”, and fit points on curved planes with quadrics classify adjacency relation as concave or convex e.t.c.

23 Generating Object Hypotheses Objects are unmodeled Group planar patches into Single-View Object Hypotheses (SVOHs) Combine SVOHs into Global Object Hypotheses (GOHs) Prefer oversegmentation to undersegmentation — avoid grasping 2 objects at the same time

24 Generating SVOHs A SVOH is a set of connected patches, such that for any 2 patches, there exists at least one path that does not contain any concave relation

25 Establishing GOHs A GOH is a set of SVOHs, such that for any SVOH i there is at least one SVOH j (from a different view) such that SVOH i and SVOH j have at least one pair of patches s k (from SVOH i ) and s l (from SVOH j ) which fulfills the same-surface predicate Rough idea of “same-surface predicate” –when 2 patches satisfy the same-surface predicate, they are on the same plane or on the same curved surface and they are intersecting each other

26 Result

27 References Marjan Trobina –“From Planar Patches to Grasps: A 3-D Robot Vision System Handling Unmodeled Objects.” Ph.D. thesis, ETHZ, 1995 A. Leonardis –“Image Analysis Using Parametric Models: Model- Recovery and Model-Selection Paradigm.” Ph.D. thesis, University of Ljubljana, 1993 A. Leonardis –“Recover-and-Select on Multiple Resolutions.” Technical report LRV-95, Computer Vision Lab, University of Ljubljana, 1995

28 References Frank Ade, Martin Rutishauser and Marjan Trobina –“Grasping Unknown Objects.” ETHZ, 1995 Martin Rutishauser, Markus Stricker and Marjan Trobina –“Merging Range Images of Arbitrarily Shaped Objects.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1994