1. Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot
Yinxiao Li and Stanley T. Birchfield
Department of Electrical and Computer Engineering
Clemson University
{ yinxial, stb

2. Motivation Goal: Segment the floor in a single corridor image Why?
obstacle avoidance
mapping
autonomous exploration and navigation

3. Motivation Goal: Segment the floor in a single corridor image Why?
obstacle avoidance
mapping
autonomous exploration and navigation

4. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

5. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

6. Previous Work Free space detection Floor detection Limitations
Combination of color and histogram (Lorigo 1997)
Optical flow (Stoffler 2000, Santos-Victor 1995)
Stereo matching (Sabe 2004)
Apply planar homographies to optical flow vectors (Kim 2009, Zhou 2006)
Stereo homographies (Fazl-Ersi 2009)
Geometric reasoning (Lee 2009)
Limitations
Multiple images for motion
Multiple cameras for stereo
Require different colors for floor and wall
Computational efficiency
Calibrated cameras
Assume ceiling visible
Our contribution: Segment the floor in real time using a single image captured by a low-height mobile robot

7. Challenging problem: Reflections
Also:
variety of poses (vanishing point, ceiling not always visible)
sometimes wall and floor color are nearly the same 7

8. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

9. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold
David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

10. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold

11. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold

12. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold

13. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold

14. Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
Connect farthest endpoints
Repeat
Find maximum distance between the original curve and the simplified curve
Split curve at this point
Until max distance is less than threshold

15. Modified Douglas-Peucker Algorithm
dallowed
original algorithm: dallowed is constant modified algorithm: dallowed is given by half-sigmoid function
original
modified

16. Detecting Line Segments (LS)

17. Detecting Line Segments (LS)
Compute Canny edges

18. Detecting Line Segments (LS)
Compute Canny edges
Modified Douglas-Peucker algorithm to detect line segments
Vertical LS: slope within ±5° of vertical direction
Horizontal LS: Slope within ±45° of horizontal direction
dallowed

19. Detecting Line Segments (LS)
Compute Canny edges
Modified Douglas-Peucker algorithm to detect line segments
Vertical LS: slope within ±5° of vertical direction
Horizontal LS: Slope within ±45° of horizontal direction
3. Pruning line segments (320x240)
Vertical LS: minimum length 60 pixels
Horizontal LS: minimum length 15 pixels
Vanishing point (height selection): The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines. It is used for throwing away horizontal line segments coming from windows, ceiling lights, etc.
dallowed

20. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

21. Score Model is assigned to each horizontal line Weights
Structure Score
Bottom Score
Homogeneous Score
is assigned to each horizontal line

22. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T1 (How to set threshold T1?)
|I(x,y)| > T1

23. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T (gradient magnitude) (How to set threshold T1?)
I(x,y) < T (image intensity) (How to set threshold T2?)
|I(x,y)| > T1 and I(x,y) < T2

24. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T (gradient magnitude) (How to set threshold T1?)
I(x,y) < T (image intensity) (How to set threshold T2?)
We applied SVM to 800 points in 200 images:
|I(x,y)| > T1 and I(x,y) < T2

25. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T (gradient magnitude) (How to set threshold T1?)
I(x,y) < T (image intensity) (How to set threshold T2?)
Approximate using two thresholds:
|I(x,y)| > T1 and I(x,y) < T2

26. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T (gradient magnitude)
I(x,y) < T (image intensity)
Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2)
I(x,y) < TLC

27. Score Model – Structure
Given a typical corridor image
|I(x,y)| > T (gradient magnitude)
I(x,y) < T (image intensity)
Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2)
Compute the chamfer distance between the line segments and structure blocks

28. Score Model – Structure
Comparison of different threshold methods
Ridler-Calvard
Otsu
Our method is able to remove the spurious pixels on the floor caused by reflections or shadows
Method
R-C
Otsu
Ours
Correctness
62%
66%
82%
Our method

29. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

30. Score Model – Bottom
Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary

31. Score Model – Bottom
Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary

32. Score Model – Bottom
Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary
Compute the distance of each horizontal line segment to the “bottom” wall-floor boundary
(red circled horizontal line segments indicates a positive contribution)

33. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

34. Score Model – Homogeneous
Idea: The floor tends to have larger regions (due to decorations, posters, windows on the wall).
Algorithm:
Color-based segmentation of the image (using Felzenszwalb- Huttenlocher’s minimum spanning tree algorithm)
For each horizontal line segment, compute
size of region
just below
segment
size of largest
segment

35. Score Model – Homogeneous
Idea: The floor tends to have larger regions (due to decorations, posters, windows on the wall).
Algorithm:
Color-based segmentation of the image (using Felzenszwalb- Huttenlocher’s minimum spanning tree algorithm)
For each horizontal line segment, compute
size of region
just below
segment
size of largest
segment

36. Segmenting the floor Normalize the scores and sum
Threshold the final score:
Connect the remaining line segments and extend the endpoints to the edges of the image

37. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogenous Score
Experimental Results
Conclusion

38. Evaluation Criterion Wall- floor Boundary
Green line: Detected wall-floor boundary
Red line: Ground truth

39. Evaluation Criterion Wall- floor Boundary Area Error Rate
Green line: Detected wall-floor boundary
Red line: Ground truth
Area
Blue shade area: misclassified area
Red shade area: ground truth floor area
Error Rate
Segmentation is considered successful if rerr < 10%

40. 89.1% success on database of 426 images:
Sample Results
89.1% success on database of 426 images:

41. Images downloaded from the internet:
Sample Results
Images downloaded from the internet:

42. Sample Results
Some successful results on failure examples from Lee et al. 2008
Original Image
Lee’s
Ours
D. C. Lee, M. Hebert, and T. Kanade. “Geometric Reasoning for Single Image Structure Recovery”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

43. Sample Results Failure examples Textured wall Dark image Bright lights
Checkered floor
Dark
image
Bright lights

44. Sample Results
Video clip 1

45. Sample Results
Video clip 2

46. Outline Previous Work Detecting Line Segments
Score Model for Evaluating Line Segments
Structure Score
Bottom Score
Homogeneous Score
Experimental Results
Conclusion

47. Conclusion Summary Future work
Edge-based approach to segmenting floors in corridors
Correctly handles specular reflections on floor
Nearly 90% of the corridor images in our database can be correctly detected.
Speed: approximately 7 frames / sec
Future work
Speed up the current algorithm
Improving score model by add more visual cues (highly textured floors, low resolution, or dark environment)
Use floor segmentation for mapping

48. Acknowledgement Clemson Computer Vision Group reviewers Zhichao Chen
Vidya Murali
Anonymous reviewers

49. Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot
Thanks! Questions?
Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot