1. A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The State University of New Jersey, Piscataway, NJ, USA

2. Outline Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion

3. Generic Object Recognition Variations in scale, orientation and visibility Variability within Specificity Object of interest has to be recognized in context of multiple other objects and cluttered background

4. Recognition of Object Categories has recently gained a lot of attention in computer vision. 3D model based method traditionally handle specific object instance, e.g. with the goal of recovering object pose. Appearance template search-based methods, e.g., Schneiderman et al 00,Viola et al 02

5. Part-based object recognition Part-based methods [Fischler73, Lowe99, Fergus03, Fergus05]: Objects are modeled as constellation, a collection of parts or local features with distinctive appearance and spatial position. the recognition is based on inferring object class based on similarity in parts’ appearance and their spatial arrangement.

6. Bag-of-features: Zhang et al (1995), Mohr et al (1997), Lowe (1999), Matas et al (2002), Mikolajczyk et al(2003), Bray et al (2004). Ignore spatial arrangement of parts Very successful result But.. None of the features are completely affine invariant Correspondences based on very closely related features Not conducive for recognition of object categories

7. Part Structure Vidal-Naquet (2001), Zisserman et al (2003), Perona et al ( 2003), Schiele et al ( 2004), Triggs et al (2005), Serre et al (2005) Limitations: Combinatorial combination of parts and locations Number of parts considered are limited Huge chunk of image information is discarded

8. Part Based approaches for object recognition Phases in part based recognition: Part extraction [kadir01]: extract salient features from the image. Part selection [Dorko04]: select parts that best characterize the target object. Select parts by classification likelihood. Select parts by mutual information. Object Model [Fergus03, Fergus05]: construct a model of the target object using the selected parts. Object recognition [Fergus03, Fergus05]: having selected parts from a test image, perform its recognition using object model.

9. Discriminative Part Selection - Problem Description Given : A collection of parts from positive and negative images Find : Parts that consistently appear in the positive images but rarely in the negative images Requirements : Focus on part similarities across positive images Ignore similarity between parts within an image Avoid ubiquitous parts - those appear in all classes A collection of parts from the negative classes Parts from the positive class images

10. Our contributions Part selection: A statistical approaches for unsupervised selection of discriminative parts: Finds the parts that best discriminate between the positive and negative classes. Object model and recognition: A probabilistic Bayesian approach for recognition where object model does not need a reference part. We investigated PCA and 2D PCA for image part representation in our experiment and did a comparison.

11. Outline Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion

12. Statistical selection of image parts Given: image parts from positive training images. Find: parts that constantly appear in multiple instances of the positive evaluation images but only rarely appear in the negative evaluation images. We use each image part from training images to build classifier for positive and negative evaluation images. This classifier is built based on whether the image part appears in the evaluation image or not. A collection of parts from the negative classes Parts from the positive class images

13. Statistical selection of image parts This classifier is built based on whether the image part appears in the evaluation image or not. Similar to boosting in building week classifier, but we filter out uncharacteristic image parts and only keep the image parts on which better classifiers are built. These selected parts should be informative of the object and the model built from these parts is likely to achieve better recognition performance.

14. Statistical selection of image parts Build classifier: The appearance of image part v in evaluation image V can be implied by the Euclidean distance between v and the closest image part in V: We can use D(V,v) and a threshold t to classify evaluation data: Image V is positive if the distance between image part v and the image V is less than a threshold t, which implies v appears in V: Image V is negative if otherwise.

15. Statistical selection of image parts Measure the performance of the classifier: Performance is measure by the misclassified evaluation image: Select image parts: We select the image parts from which we can build a classifier with a classification error rate less than a threshold θ.

16. Experiments Image parts selection from the statistical methods:

17. Outline Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion

18. Parts based probabilistic model We model the probability of an object with its centroid, given the observed image parts from that image. If the probability is larger than a threshold, it indicates the presence of the object class in the image. Assuming independence between image parts v k and using Bayes’ rule, the probability model of object class O with its centroid C, given the m observed image parts (k=1,…,m), can be formulated as:

19. Parts based probabilistic model P(v k |O,C) can be modeled as mixture-Gaussians learnt from the selected image parts from the training data, which is clustered into n clusters A i, ( i =1..n ) according to their appearance and 2D offset of the centroid to them:

20. Parts based probabilistic model With further assumption of the independence between P(O) and P(C) and both part v k and the centroid C from one cluster following normal distribution, we can estimate the terms for P(v k |O,C) : Here and denote the sample mean for v k and C respectively. and denote the sample covariance for v k and C respectively. Other terms can be approximated using the statistics from each of the cluster A i.

21. Parts based probabilistic model Recognition: While performing recognition, the P(v k ) can be ignored so we have The recognition can be viewed as: for each of the image part v k, it will vote for the possible object centroid according the clusters it is close to: vkvk cluster i cluster j cluster k Weighted vote for centroid Test Image

22. Outline Background and motivations Statistical selection of image part Parts based probabilistic model Experiments Conclusion

23. Experiments Dataset: Dataset from Caltech database with four class objects: motorbikes, airplane, face, car rear end against background. trainevaluatetest positive negativepositivenegative Each class17550 225

24. Experiments Image part detection and representation: Use Kadir and Brady’s region based feature detector [Kadir01] for detecting informative image partes. Normalize and rescale image parts to 11×11 pixels and represent them as 121 dimension intensity vector. Both PCA and 2D PCA were applied on vectors to get a more compact 18 dimension intensity representation.

25. Experiments Experiment setting: Extract 100 image parts from each of the training image and evaluation image. Applied the statistical methods for removing the image parts from the background. Statistical method: build a simple classifier from each image part in training images and select the ones which led to a classifier with classification error rate less than 24%. Compute the probability of the centroid of a possible object in the image as the indicator of its presence. Use 2D offset between the image part and the centroid of the image as spatial information for the image part. Use k-means algorithm to cluster selected parts into 70 clusters and calculate the statistics for them.

26. Experiments Image parts selection from the statistical methods:

27. Experiments Objects detected in the image:

28. Experiments Equal ROC performance using different methods: datasetNo selection Statistical Method with 2D PCA Statistical Method with PCA [Fergus03][Opelt04] Airplane54.2%95.8%94.4%90.2%88.9% Motorbike67.8%93.7%94.9%92.5%92.2% Face62.7%97.3%98.4%96.4%93.5% Car(rear)65.6%98.0%96.7%90.3%n/a

29. Conclusion: We have presented a statistical method for selecting informative image parts for part-based object detection and class recognition. This method yields competitive recognition rates and surpasses the performance of many existing methods. It is a general method suitable for selecting a set of features in other application. Future work: Integrate information regarding the spatial arrangement between image parts; Develop the framework as a general method with various hyper-parameters automatically determined.

30. Main References [Dorko 04] G. Dorko and C. Schmid. Object Class Recognition Using Discriminative Local Features. Submitted to PAMI04. [Fergus 03] R. Fergus, P. Perona and A. Zisserman. Object class recognition by unsupervised scale-invariant learning. In CVPR(2) page 264-271, 2003. [Fergus 05] R. Fergus, P. Perona and A. Zisserman. A Sparse Object Category Model for Efficient Learning and Exhaustive Recogntion. [Fischler 73] M. Fischler and R.Elschlager. The representation and matching of pictorial structures, 1973. IEEE Transaction on Computer c-22(1): 67-92 [Kadir 01] T. Kadir and M. Brady. Scale, saliency and image description. IJCV 2001 [Lowe 99] D. G. Lowe. Object recogntion from local scale-invariant features. In Proc. Of the International Conference on Computer Vision ICCV Corfu, pages 1150-1157, 1999 [Opelt 04] A. Opelt, M. Fussenegger, A. Pinz, P. Auer. Weak hypotheses and boosting for generic object detection and recognition. ECCV(2) 71-84, 2004 [Schneiderman 00] H.Schneiderman and T.Kanade. A statistical method 3d object detection applied to faces and cars pages 45-51, 2000 [Viola 02] P.Viola and M. Jones Robust real time object detection. International Journal of Computer Vision 2002. [Wolfson 97] H.J. Wolfson and I. Rigoutsos. Geometric hashing: An overview. IEEE Computational Science & Engineer, 4(4):10-21,/1997.