Max-Confidence Boosting With Uncertainty for Visual tracking WEN GUO, LIANGLIANG CAO, TONY X. HAN, SHUICHENG YAN AND CHANGSHENG XU IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 24, NO. 5, MAY

Outline Introduction Max-Confidence Boosting MCB-Based Tracking Method Experimental Results Conclusions 2

Introduction We present a novel visual tracking method called max- confidence boosting (MCB). Our method can be viewed as a generalization of semi- supervised learning. One key difference is that our tracker works on “soft” labels, or namely indeterministic labels. We argue that indeterministic labels should be preferred over deterministic definite labels in visual tracking due to two reasons. 3

Introduction First reason: There are inherent ambiguities in visual detection. Sometimes the visual appearance of foreground and background may be similar. Sometimes the context information might be helpful in detection although they are not physically part of the tracking subject. 4

Introduction Second reason: Traditional trackers are based on deterministic labels, which may cause drift due to error accumulation. If the tracker position is not precise in some certain position, the appearance model will update according to the imperfect labeled data. Small classification errors on background patch will start to accumulate and eventually cause the tracker to drift away. 5

Introduction (i) A novel algorithm for learning from both indeterministic data and unlabeled data. (ii)A max-confidence boosting learning framework based ensemble tracker, which reduces the accumulating errors by utilizing both patches in current frame and those from previous frames. (iii) Max-confidence boosting is a generalized AdaBoost. The implications are significant because the improvement allows the well-known Adaboost algorithm to be applied to a wide variety of semi-supervised learning problems which are currently a popular and active subject for research. 6

Introduction── Adaboost It sequentially combines a set of weak classifiers by adaptively adjusting the weights of the training samples, and then yields a strong classifier as a linear combination of the selected weak classifiers. For the image patches that are incorrectly classified, they will also be fed into AdaBoost for online training. A small classification error on background patch will start to accumulate and eventually cause the tracker to drift away. 7

Outline Introduction Max-Confidence Boosting MCB-Based Tracking Method Experimental Results Conclusions 8

Max-Confidence Boosting 9

10

Max-Confidence Boosting The cost function based on all the training data is which we call Prediction Confidence. Our new boosting algorithm based on the prediction confidence is named as Max-Confidence Boosting (MCB). 11

Max-Confidence Boosting 12

Max-Confidence Boosting 13

Max-Confidence Boosting 14

Max-Confidence Boosting 15

Max-Confidence Boosting 16

Max-Confidence Boosting 17

Max-Confidence Boosting Framework 18

Outline Introduction Max-Confidence Boosting MCB-Based Tracking Method Experimental Results Conclusions 19

MCB-Based Tracking Method We apply the proposed MCB algorithm to obtain a discriminative tracker, which is robust to the changes of foreground and background, even under the scenarios with large camera motions and partial occlusions. We treat the patches in the new frame as unlabeled data, and train a classifier based on the labeled patches with indeterministic labels from previous frames together with the unlabeled ones in the new frame. 20

MCB-Based Tracking Method MCB treats the features of patches in the current frame as unlabeled data. For the ambiguous patch, such as the patches at the border of the tracked objects, the confidences on these patches are low and they do not contribute too much (if not at all) to update the classifier. Therefore, the classification errors on the patches of the current frame will not propagate to the subsequent frame. The error accumulation is avoided. 21

22

Outline Introduction Max-Confidence Boosting MCB-Based Tracking Method Experimental Results Conclusions 23

24

25

Experimental setup We will compare the tracking performance of the MCB based tracker with Avidan’s ensemble tracking (ADB), online boosting tracker (ONB), as well as multiple instance learning tracker (MIL). 26

Experimental setup For initialization, we manually select the object in the first frame, in which an inner boundingbox represents foreground and a triple outside boundingbox represents background. For unlabeled sequences, we manually label the tracking object as groundtruth and make a qualitative evaluation for all the tracking methods. 27

28

29

Partial Occlusion and Illumination variation 30 basketball sequence skating sequence

31

32

Cluttered Background 33 supermarket sequence snow sequence

34

Abrupt Motion and Similar Unicolor 35 hunting sequence

Computation time Our methods is approximate to the ADB and longer than the ONB, but our tracker gives the better results than them. Although the MIL tracker gives some better results, it depends on the initial parameters too much and costs too much computational complexity. 36 skating sequence

Outline Introduction Max-Confidence Boosting MCB-Based Tracking Method Experimental Results Conclusions 37

Conclusions In this paper, we proposed a novel tracker based on Max- Confidence Boosting (MCB) algorithm. MCB generalizes conventional AdaBoost algorithm from the data with deterministic labels to indeterministic labels. The tracking results on the challenge sequences validate the effectiveness of our new algorithms. 38

39 Thanks for your listening!