Sean M. Ficht

 Problem Definition  Previous Work  Methods & Theory  Results

 Track and follow specific person with a mobile robot  Cluttered environments  Brief occlusion  Long occlusion  Cooperative user

 Helper robot  Carry items for a person Example: Hospital situation

 Problem Definition  Previous Work  Methods & Theory  Results

 Person following with a mobile robot Appearance based Optical flow based Stereo vision based

 Segmentation of image  Classification  Detection  Limitations Sidenbladh, Kragic, and Christensen; ICRA; 1999 Tarokh and Ferrari; Journal of Robotic Systems; 2003 Schlegel, Illman, Jaberg, Schuster, and Worz; British Machine Vision Conference; 2005

 Calculate optical flow  Use to segment image  Limitations Chivilo, Mezzaro, Sgorbissa, and Zaccaria; IROS; 2004 Piaggio, Fornaro, Piombo, Sanna, and Zaccaria; IEEE ISIC/CIRA/ISAS joint conference; 1998

 Find features  Segment from background  Use to track  Limitations Zhichao and Birchfield; IROS; 2007

 Problem Definition  Previous Work  Methods & Theory  Results

 Kinect  Provides a depth image  Provides a RGB color image  Packaged solution

 Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

 HOG person detector (OpenCV) HOG descriptor o Cells -> Block -> Window o 4 cells in a block o 105 blocks in a window o 64x128 window Training Dalal and Triggs, CVPR, 2005

 Gradient of the Image  Binning of pixels in cells  Grouping of cells into blocks  Normalization

 Kernel convolution Magnitude = (g x 2 + g y 2 ) Angle = arctan(g y /g x )  Directional change in intensity

 Bins apply to each cell  Nine separate bins  Gradient magnitude added to bin

 Cells grouped into blocks  4 cells per block  Blocks overlap one another



 HOG person detector HOG descriptor o Cells -> Block -> Window o 4 cells in a block o 105 blocks in a window o 64x128 window Training

 Support Vector Machine (SVM) classifier  Binary classifier  Trained on images

 Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

 Color Histogram  Segmentation by depth to create template

 Represents distribution of colors  10 bins for each color channel 1000 element color histogram  Pixel classification  2 bin example Bin 1: Bin 2:

 Average depth  Threshold (0.3 meters)  Template used to make color histogram

 Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

 System State  Motion model  Observation model  Expected state  Resample

 Hybrid state space  X and Y in image coordinates Scaled according to depth  Z in depth coordinates











 Detection and Tracking Generic detector Specific appearance model Integrating particle filter  Robot Control

 Input: tracking information from tracking algorithm  Uses tracking information to make movement decisions  Executes movement and returns to tracking algorithm

 Problem Definition  Previous Work  Methods & Theory  Results

 No occlusion Other people present (different depth) Other people present (similar depth) Pose change  Brief occlusion  Long occlusion

 Initial template

Initial template Non-occluded target Occluded target

 Average between 73% and 74%

 Problem Follow a person in different scenarios  System RGB-D sensor Generic detector Specific appearance model Particle filter Robot control architecture  Performance Performed in three separate test scenarios Rapid side to side target motion trade-off Large target scale changes

 Train a new HOG detector to handle scale issues  Using more particles  KLT features for trajectory histories  Adaptive appearance model