1. Sean M. Ficht

2.  Problem Definition  Previous Work  Methods & Theory  Results

3.  Track and follow specific person with a mobile robot  Cluttered environments  Brief occlusion  Long occlusion  Cooperative user

4.  Helper robot  Carry items for a person Example: Hospital situation

5.  Problem Definition  Previous Work  Methods & Theory  Results

6.  Person following with a mobile robot Appearance based Optical flow based Stereo vision based

7.  Segmentation of image  Classification  Detection  Limitations Sidenbladh, Kragic, and Christensen; ICRA; 1999 Tarokh and Ferrari; Journal of Robotic Systems; 2003 Schlegel, Illman, Jaberg, Schuster, and Worz; British Machine Vision Conference; 2005

8.  Calculate optical flow  Use to segment image  Limitations Chivilo, Mezzaro, Sgorbissa, and Zaccaria; IROS; 2004 Piaggio, Fornaro, Piombo, Sanna, and Zaccaria; IEEE ISIC/CIRA/ISAS joint conference; 1998

9.  Find features  Segment from background  Use to track  Limitations Zhichao and Birchfield; IROS; 2007

10.  Problem Definition  Previous Work  Methods & Theory  Results

11.  Kinect  Provides a depth image  Provides a RGB color image  Packaged solution

12.  Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

13.  HOG person detector (OpenCV) HOG descriptor o Cells -> Block -> Window o 4 cells in a block o 105 blocks in a window o 64x128 window Training Dalal and Triggs, CVPR, 2005

14.  Gradient of the Image  Binning of pixels in cells  Grouping of cells into blocks  Normalization

15.  Kernel convolution Magnitude = (g x 2 + g y 2 ) Angle = arctan(g y /g x )  Directional change in intensity

16.  Bins apply to each cell  Nine separate bins  Gradient magnitude added to bin

17.  Cells grouped into blocks  4 cells per block  Blocks overlap one another

18. 

19.  HOG person detector HOG descriptor o Cells -> Block -> Window o 4 cells in a block o 105 blocks in a window o 64x128 window Training

20.  Support Vector Machine (SVM) classifier  Binary classifier  Trained on images

21.  Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

22.  Color Histogram  Segmentation by depth to create template

23.  Represents distribution of colors  10 bins for each color channel 1000 element color histogram  Pixel classification  2 bin example Bin 1: 0-127 Bin 2: 128-255

24.  Average depth  Threshold (0.3 meters)  Template used to make color histogram

25.  Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

26.  System State  Motion model  Observation model  Expected state  Resample

27.  Hybrid state space  X and Y in image coordinates Scaled according to depth  Z in depth coordinates

28. 

29. 

30. 

31. 

32. 

35.  Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

36.  Input: tracking information from tracking algorithm  Uses tracking information to make movement decisions  Executes movement and returns to tracking algorithm

38.  Problem Definition  Previous Work  Methods & Theory  Results

39.  No occlusion Other people present (different depth) Other people present (similar depth) Pose change  Brief occlusion  Long occlusion

42.  Initial template

46. Initial template Non-occluded target Occluded target

47.  Average between 73% and 74%

50.  Problem Follow a person in different scenarios  System RGB-D sensor Generic detector Specific appearance model Particle filter Robot control architecture  Performance Performed in three separate test scenarios Rapid side to side target motion trade-off Large target scale changes

51.  Train a new HOG detector to handle scale issues  Using more particles  KLT features for trajectory histories  Adaptive appearance model