Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Search-Classify Approach for Cluttered Indoor Scene Understanding Liangliang Nan 1, Ke Xie 1, Andrei Sharf 2 1 SIAT, China 2 Ben Gurion University, Israel.

Similar presentations


Presentation on theme: "A Search-Classify Approach for Cluttered Indoor Scene Understanding Liangliang Nan 1, Ke Xie 1, Andrei Sharf 2 1 SIAT, China 2 Ben Gurion University, Israel."— Presentation transcript:

1 A Search-Classify Approach for Cluttered Indoor Scene Understanding Liangliang Nan 1, Ke Xie 1, Andrei Sharf 2 1 SIAT, China 2 Ben Gurion University, Israel

2 Digitalization of indoor scenes Indoor scenes from Google 3D Warehouse

3 Acquisition of indoor scenes

4 Goal Scene understanding

5 Challenges Clutter –Densely populated –Arbitrary arrangements Partial representation –Occlusions Complex geometry

6 Classification & Segmentation Two interleaved problems –What are the objects? –Where are the objects? Chicken-egg problem –Classification needs segmentation –Segmentation needs a prior

7 Our solution Search –Propagate / accumulate patches Classify –Query classifier to detect object

8 Related Work Indoor scenes ( This Session ) – [Fisher et al. 2012] [Shao et al. 2012] [Kim et al. 2012] Semantic relationship – [Fisher et al. 2010, 2011] Recognition using depth + texture (RGB-D) – [Quigley et al.2009], [Lai and Fox 2010] Outdoor classification – [Golovinskiy et al. 2009] Semantic labeling – [Koppula et al. 2011] Controlled region growing process

9 Our search-classify idea

10 Method overview Training Search-Classify

11 Point cloud features –Height-size ratio of BBox –Aspect ratio of each layer –Bottom-top, mid-top size ratio –Change in COM along horizontal slabs BhBh BdBd BwBw

12 Classifier Handle missing data –Occlusion Random decision forest –Efficient multi-class classifier Trained with both scanned and synthetic data –Manually segmented and labeled –510 chairs –250 tables –110 cabinets –40 monitors etc. [Shotton et al. 2008, 2011]

13 Search-Classify Starts from seeds –Random patch triplets –Remove seeds with low confidence Accumulating neighbor patches –Highest classification confidence Stop condition –Steep decrease in classification confidence Seed

14 Segmented - classified objects problems –Overlap, outliers, ambiguities etc. Refinement –Outliers = patches with large distance Segmentation refinement by template fitting

15 Template deformation Different styles for each class Predefined scalable parts Templates can deform [Xu et al. 2010]

16 Template deformation Different styles for each class Predefined scalable parts Templates can deform [Xu et al. 2010]

17 Fitting via template deformation ConfidenceFitting errorBest fitting Best matching template –One-side Euclidean distance from points to template

18 Results and discussion

19

20 Scalability test with varied object density 0 (25) 1 (45) 5 (60)

21 Results and discussion Comparison Lai et al Ours

22 Limitation Upward assumption –Features –Template fitting

23 Future work Contextual information

24 Thank you


Download ppt "A Search-Classify Approach for Cluttered Indoor Scene Understanding Liangliang Nan 1, Ke Xie 1, Andrei Sharf 2 1 SIAT, China 2 Ben Gurion University, Israel."

Similar presentations


Ads by Google