研 究 生：周暘庭 Ｑ３６９９４４７７ 電腦與通信工程研究所 通訊與網路組 指導教授 ：楊家輝 Mean-Shift-Based Color Tracking in Illuminance Change

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 2 Introduction Color and Illuminance Mean-Shift Tracking Through Illuminance Space Experimental Results Conclusions Outline  Mean-shift in image space  Mean-shift in illuminance space

Introduction

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 4 Intuitive Description (1/7) Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 5 Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region Intuitive Description (2/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 6 Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region Intuitive Description (3/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 7 Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region Intuitive Description (4/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 8 Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region Intuitive Description (5/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 9 Distribution of identical billiard balls Region of interest Center of mass Mean Shift vector Objective : Find the densest region Intuitive Description (6/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 10 Distribution of identical billiard balls Region of interest Center of mass Objective : Find the densest region Intuitive Description (7/7)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 11 Introduction(1/2) Tracking often requires real-time processing, so high-speed processing is essential.  The mean-shift algorithm has a low calculation cost and offers high-speed execution. Tracking is difficult when lighting changes because the RGB values from the image changes with the lighting.

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 12 Introduction(2/2) This paper presents a novel approach for color tracking that is robust to lighting changes for robot vision. We use two interleaved mean-shift procedures to track the spatial location and illumination intensity of a blob in an image.  Mean-shift in image space  Mean-shift in illuminance space

Color and Illuminance

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 14 The illuminance of known color can be measured by observing the RGB values obtained by a CCD camera. In RGB color space, it is difficult to distinguish.  Because we cannot create a threshold criterion that specifies how the color space should be divided up into a handful of color classes. Using HIS color system, it is difficult to distinguish a moving object and light change. In this paper, we augment the RGB color space to make an RGB-illuminance space. Color and Illuminance(1/4) (a) shows various color patches (Blue, Black, Green, Pink, Purple, White, Yellow) under illuminance ranging from 10 to 1400 lx. (b) is the experimental setup.

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 15 Color and Illuminance(2/4) 2.1 RGB-illuminance space RGB-illuminance space: (a) color distribution in RB-illuminance space, (b) RB value of each color class at 100 lx, and (c) RB value at 200 lx. In the RGB-illuminance space, reference-based searching such as the k-NN method for color clustering can work.  But it takes a lot of time due to the number of reference patterns for each illuminance.

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 16 Color and Illuminance(3/4) 2.2 Color-illuminance model : illuminance intensity : R,G,B color values ( candidate ) : unknowns computed by the least- squares method

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 17 Color and Illuminance(4/4) 2.3 Iris adjustment : the iris ( F-number ) : iris diameter : focus length : RGB value ( model )

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 18

Mean-Shift Tracking Through Illuminance Space

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 20 Flow Diagram Input EvAdjust Iris Position tracking Light tracking Output No Yes Find the most similar part

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 21 Mean-Shift tracking(1/8) Mean-Shift in image space K : Gaussian Kernel function w : weight b(x) : feature value of the pixel x u : the color bin index m : total number of features q u : Pb. of feature u in model p u (x) : Pb. of feature u in candidate x 0 : current location x 0 What’s this ? x0x0 xixi △x△x  The mean-shift algorithm is a simple nonparametric method for seeking the nearest mode of a sample distribution.

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 22 Similarity Function: Target Model Target Candidate (u) ( b(x) ) Mean-Shift tracking(2/8)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering Linear approx. (around y 0 ) : Mean-Shift tracking(3/8)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 24 Mean-Shift tracking(4/8)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 25 Mean-Shift tracking(5/8) Mean-Shift in illuminance space for single-color tracking Step1 : Mean-shift in image space Similarity between model I and candidate I ( From cosine )

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 26 Mean-Shift tracking(6/8) Mean-Shift in illuminance space for single-color tracking Step2 : Mean-Shift in illuminance space

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 27 Mean-Shift tracking(7/8) Mean-Shift in illuminance space for single-color tracking Step3 : Iteration Iterate by interleaving steps 1 and 2 until

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 28 Mean-Shift tracking(8/8) Mean-Shift in illuminance space for multiple-color tracking w loc (x i ) c1 is the 1st maximum value in multiple colors c at x i and w loc (x i ) c2 is 2nd one.

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 29 Flow Diagram Input EvAdjust Iris Position tracking Light tracking Output No Yes Find the most similar part

Experimental Results

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 31 Experimental Results(1/2)

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 32 Experimental Results(2/2) (a) Smooth light change(b) Rapid light change

Conclusions

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 34 We proposed a tracking method using two interleaved mean-shift procedures to track the mode in illuminance space. We demonstrated that our method enables real- time color tracking that is robust to changes in illumination, where the illuminance ranges from 50 to 1200 lx. Conclusions

National Cheng Kung University, Tainan, Taiwan Institute of Computer and Communication Engineering 35 Thanks for your attention!!