Presentation is loading. Please wait.

Presentation is loading. Please wait.

Object Detection Using Semi- Naïve Bayes to Model Sparse Structure Henry Schneiderman Robotics Institute Carnegie Mellon University.

Similar presentations


Presentation on theme: "Object Detection Using Semi- Naïve Bayes to Model Sparse Structure Henry Schneiderman Robotics Institute Carnegie Mellon University."— Presentation transcript:

1 Object Detection Using Semi- Naïve Bayes to Model Sparse Structure Henry Schneiderman Robotics Institute Carnegie Mellon University

2 Object Detection Find all instances of object X (e.g. X = human faces)

3 Examples of Detected Objects

4 Sparse Structure of Statistical Dependency Chosen variable

5 Sparse Structure of Statistical Dependency Chosen coefficient

6 Sparse Structure of Statistical Dependency Chosen coefficient

7 Detection using a Classifier “Object is present” (at fixed size and alignment) “Object is NOT present” (at fixed size and alignment) Classifier

8 Proposed Model: Semi-Naïve Bayes Kononenko (1991), Pazzini (1996), Domingos and Pazzini (1997), Rokach and Maimon (2001) e.g. S 1 = (x 21, x 34, x 65, x 73, x 123 ) S 2 = (x 3, x 8, x 17, x 65, x 73, x 111 ) subsets input variables

9 Goal: Automatic subset grouping S 1 = (x 21, x 34, x 65, x 73, x 123 ) S 2 = (x 3, x 8, x 17, x 65, x 73, x 111 )... S n = (x 14, x 16, x 17, x 23, x 85, x 101, x 103, x 107 )

10 Approach: Selection by Competition x 1 x 2 x 3... x m S 1 S 2... S q Generate q candidate subsets log [p 1 (S 1 |  1 ) / p 1 (S 1 |  2 )] log [p 2 (S 2 |  1 ) / p 2 (S 2 |  2 )]... log [p q (S q |  1 ) / p q (S q |  2 )] Train q log likelihood function Select combination of n candidates log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )] log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )]... log [p jn (S jn |  1 ) / p jn (S jn |  2 )] H(x 1,…,x r ) = log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )]+log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )] log [p jn (S jn |  1 ) / p jn (S jn |  2 )] q functions n functions, n << q

11 Approach: Selection by Competition x 1 x 2 x 3... x m S 1 S 2... S q Generate q candidate subsets log [p 1 (S 1 |  1 ) / p 1 (S 1 |  2 )] log [p 2 (S 2 |  1 ) / p 2 (S 2 |  2 )]... log [p q (S q |  1 ) / p q (S q |  2 )] Train q log likelihood function Select combination of n candidates log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )] log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )]... log [p jn (S jn |  1 ) / p jn (S jn |  2 )] H(x 1,…,x r ) = log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )]+log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )] log [p jn (S jn |  1 ) / p jn (S jn |  2 )]

12 Generation of Subsets “modeling error for assuming independence” q is size of the subset

13 Generation of Subsets Selection of variables - “discrimination power” q is size of the subset

14 Pair-Wise Measurement pair-wise measurements Pair-affinity

15 Visualization of C(x,*) (frontal faces) x x x

16 Measure over a Subset Subset-affinity

17 Generation of Candidate Subsets x 1 x 2 x x m S 1 S S p Heuristic search and selective evaluation of D(S i ) C(x 1, x 2 ) C(x 1, x 3 ) C(x m-1, x m )

18 subset size vs. modeling power Model complexity limited by number of training examples, etc. Examples of limited modeling power –5 modes in a mixture model –7 projection onto principal components

19 Approach: Selection by Competition x 1 x 2 x 3... x m S 1 S 2... S q Generate q candidate subsets log [p 1 (S 1 |  1 ) / p 1 (S 1 |  2 )] log [p 2 (S 2 |  1 ) / p 2 (S 2 |  2 )]... log [p q (S q |  1 ) / p q (S q |  2 )] Train q log likelihood function Select combination of n candidates log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )] log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )]... log [p jn (S jn |  1 ) / p jn (S jn |  2 )] H(x 1,…,x r ) = log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )]+log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )] log [p jn (S jn |  1 ) / p jn (S jn |  2 )]

20 Log-likelihood function = Table S i = (x i1, x i2,..., x iq ) vector quantization table look-up

21 Sub-Classifier Training by Counting P i (f i |  1 ) P i (f i |  2 ) fifi fifi

22 Example of VQ x i1 x i2 x i3... x iq c 1 c 2 c 3 z 1 z 2 z 3 f = z 1 m 0 + z 2 m 1 + z 3 m 2 quantization to m levels projection on to 3 principal components

23 Approach: Selection by Competition x 1 x 2 x 3... x m S 1 S 2... S q Generate q candidate subsets log [p 1 (S 1 |  1 ) / p 1 (S 1 |  2 )] log [p 2 (S 2 |  1 ) / p 2 (S 2 |  2 )]... log [p q (S q |  1 ) / p q (S q |  2 )] Train q log likelihood function Select combination of n candidates log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )] log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )]... log [p jn (S jn |  1 ) / p jn (S jn |  2 )] H(x 1,…,x r ) = log [p j1 (S j1 |  1 ) / p j1 (S j1 |  2 )]+log [p j2 (S j2 |  1 ) / p j2 (S j2 |  2 )] log [p jn (S jn |  1 ) / p jn (S jn |  2 )]

24 h 1 (S 1 ) h 2 (S 2 )... h P (S P ) E 1,  1 E 1,  2 E 2,  1 E 2,  2... E p,  1 E p,  2 Evaluate on training data ROC 1 ROC 2... ROC P Evaluate ROCs Order top Q log-likelihood functions h j1 (S j1 ) h j2 (S j2 )... h jQ (S jQ ) Candidate log-likelihood functions

25 h j1 (S j1 ) + h 1 (S 1 )... h jQ (S jQ ) + h p (S p ) Form pQ pairs of log-likelihood functions E j1,  1 + E 1,  1 E j1,  2 + E 1,  2... E jQ,  1 + E p,  1 E jQ,  2 + E p,  2 Sum Evaluations ROC 1... ROC QP Evaluate ROCs Order top Q pairs of log-likelihood functions h k1,1 (S k1,1 ) + h k1,2 (S k1,2 )... h kQ,1 (S kQ,1 ) + h kQ,2 (S kQ,2 )... Repeat for n iterations

26 H 1 (x 1, x 2,..., x r )... H Q (x 1, x 2,..., x r ) Cross-Validation Selects Classifier Q Candidates: H 1 (x 1, x 2,..., x r ) = h k1,1 (S k1,1 ) + h k1,2 (S k1,2 ) h k1,n (S k1,n )... H Q (x 1, x 2,..., x r ) = h kQ,1 (S kQ,1 ) + h kQ,2 (S kQ,2 ) h Q,n (S kQ,n ) H * (x 1, x 2,..., x r ) Cross-validation

27 Example subsets learned for telephones

28 Evaluation of Classifier “Object is present” (at fixed size and alignment) “Object is NOT present” (at fixed size and alignment) Classifier

29 1) Compute feature values f 2 = #3214 f 1 = #5710 f n = #723

30 2) Look-Up Log-Likelihoods P 1 ( #5710 |  1 ) P 1 ( #5710 |  2 ) f 2 = #3214 f 1 = #5710 f n = #723 = 0.53 P 2 ( #3214 |  1 ) P 2 ( #3214 |  2 ) = 0.03 log P n ( #723 |  1 ) P n ( #723 |  2 ) = 0.23 log

31 3) Make Decision P 1 ( #5710 |  1 ) P 1 ( #5710 |  2 ) = 0.53 P 2 ( #3214 |  1 ) P 2 ( #3214 |  2 ) = 0.03 log P n ( #723 |  1 ) P n ( #723 |  2 ) = 0.23 log  > <

32 Detection using a Classifier “Object is present” (at fixed size and alignment) “Object is NOT present” (at fixed size and alignment) Classifier

33 View-based Classifiers Face Classifier #1 Face Classifier #2 Face Classifier #3

34 Detection: Apply Classifier Exhaustively Search in position Search in scale

35 Decision can be made by partial evaluation P 1 ( #5710 |  1 ) P 1 ( #5710 |  2 ) = 0.53 P 2 ( #3214 |  1 ) P 2 ( #3214 |  2 ) = 0.03 log P n ( #723 |  1 ) P n ( #723 |  2 ) = 0.23 log  > <

36 Detection Computational Strategy Computational strategy changes with size of search space Apply log [p 1 (S 1 |  1 ) / p 1 (S 1 |  2 )] exhaustively to scaled input image Apply log [p 2 (S 2 |  1 ) / p 2 (S 2 |  2 )] reduced search space Apply log [p 3 (S 3 |  1 ) / p 3 (S 3 |  2 )] further reduced search space

37 Repeat for N 2 Candidates Compute M 2 feature valuesLook-up M 2 log-likelihood values Candidate-Based Evaluation

38 Repeat for N 2 Candidates Compute N 2 + M 2 +2MN feature valuesLook-up M 2 log-likelihood values Feature-Based Evaluation

39 Create candidate subsets Train candidate log-likelihood functions Select log-likelihood functions Retrain selected log-likelihood functions using Adaboost Training images of object Cross-validation images Training images of non- object Determine detection threshold Automatically select non-object examples for next stage Images that do not contain object Increment stage Cascade Implementation Adaboost using confidence-rated predictions [Shapire and Singer, 1999] Bootstrapping [Sung and Poggio, 1995]

40 Boosting... H(x 1, x 2,..., x r ) = h j1 (S j1 ) + h j2 (S j2 ) h jn (S jn ) x 1 x 2 x 3... x m S 1 S 2... S p Generate p candidate subsets; p >> n h 1 (S 1 ) h 2 (S 2 )... h p (S p ) Train p candidate sub-classifiers h j1 (S j1 ) h j2 (S j2 )... h jr (S jn ) Select combination of n candidates Retrain with Adaboost using confidence-rated predictions [Shapire and Singer, 1999]

41 Face, eye, ear detection

42 Frontal Face Detection Recognition rate85.2%89.7%92.1%93.7%94.2% False detections (this method) False Detections [Viola and Jones, CVPR, 2001] MIT-CMU Frontal Face Test Set [Sung and Poggio, 1995; Rowley, Baluja and Kanade, 1997] –180 ms 300x200 image –400 ms 300x500 image Top Rank Video TREC 2002 Face Detection Top Rank 2002 ARDA VACE Face Detection algorithm evaluation AMD Athalon 1.2GHz

43 Face & Eye Detection for Red-Eye Removal from Consumer Photos CMU Face Detector

44 Eye Detection Experiments performed independently at NIST Sequested data set: 29,627 mugshots Eyes correctly located (radius of 15 pixels) 98.2% (assumed one face per image) Thanks to Jonathon Phillips, Patrick Grother, and Sam Trahan for their assistance in running these experiments

45 Realistic Facial Manipulation: Earring Example With Jason Pinto

46 Telephone Detection

47 Cart, pose 1

48 Cart, pose 2

49 Cart, pose 3

50 Door Handle Detection

51 Summary of Classifier Design Sparse structure of statistical dependency in many image classification problem Semi-naïve Bayes Model Automatic learning structure of semi-naïve Bayes classifier: –Generation of many candidate subsets –Competition among many log-likelihood functions to find best combination CMU on-line face detector:


Download ppt "Object Detection Using Semi- Naïve Bayes to Model Sparse Structure Henry Schneiderman Robotics Institute Carnegie Mellon University."

Similar presentations


Ads by Google