1. Multi Scale CRF Based RGB-D Image Segmentation Using Inter Frames Potentials Taha Hamedani Robot Perception Lab Ferdowsi University of Mashhad The 2 nd RSI International Conference on Robotics and Mechatronics (ICROM 2014) Oct 2014

2. Outline  Image segmentation Definition  Kinect Range Data and its Noises  Conditional Random Field (CRF)  Region Extraction based on Geometrical Features  Energy Function  Experimental Result  Future Work 2

3. Image Segmentation  Partitioning the image in to the similar and disjoint regions  These regions have the similar features such as RGB, orientation, texture 3

4. Microsoft Kinect Sensor  Estimate Depth data by structure light method  Project pseudo Random Pattern IR  Estimation based on comparison with position of received IR ray 4

5. 2.5 Dimension Data and Noises 5

6. Conditional Random Field (CRF) According to Hammersley-Clifford Theorem 6

7. Multi scale CRF  Build a Pyramid of variant resolutions of image  Compute unary and pairwise potentials for each level of pyramid based on coarsest level of pyramid 7

8. Energy minimization  Solve energy minimization problem as a top down strategy  Projection  relaxation 8

9. Inter level Cliques  Consider inter level cliques beside singleton and doubleton cliques in each level (Kato) 9

10. Region Extraction  Extract the main region of the scene based on geometrical features  RGB edges (canny edge detector)  Depth edges (cosine of angle between normals of two neighbor pixels)  Sum two edges  Morphological opening in order to construct regions of the scene 10

11. Region Extraction  A simple scene with three side wall 11

12. Region Extraction  More complex scene ( NYU V2) 12

13. Inter Frame Cliques  Consider inter frame cliques between current pixel and previous frame labeling 13

14. Energy Function  Define our energy function based on these new cliques and regions 14

15. Experimental Result  Data set  New York v2 (NYUV2)  Release in 2012  RGB-D images from Kinect  464 images of 26 different indoor scenes  Annotated for 1000 classes 15

16. Experimental Result 16

17. Experimental Result  Hausdorff Distance 17

18. Future Work  Using previous frame data as a more effective manner  Information updating after each minimization iteration such as Normal vector correction 18

19. Question ? Thanks for your attention ? 19