1. Rapid Object Detection using a Boosted Cascade of Simple Features
(moving or acting with great speed)
(increase the strength or value of Sth)
Rapid Object Detection using a Boosted Cascade of Simple Features
Original Author
Paul Viola & Michael Jones
In: Proc. Conf. Computer Vision and Pattern Recognition. Volume 1.,
Kauai, HI, USA (2001) 511–518
Speaker: Jing Ming Chiuan (井民全)

2. Outline Introduction The Boost algorithm for classifier learning
Feature Selection
Weak learner constructor
The strong classifier
A tremendously difficult problem
Result
Conclusion

3. Discard the background regions
What had we done?
A machine learning approach for visual object detection
Capable of processing images extremely rapidly
Achieving high detection rates
Three key contributions
A new image representation  Integral Image
A learning algorithm( Based on AdaBoost[5])
A combining classifiers method  cascade classifiers
Speed up the feature evaluation
Select a small # of visual features from a larger set
yield an efficient classifiers
Discard the background regions
of the image

4. A demonstration on face detection
A frontal face detection system
The detector run at 15 frames per second without resorting to image differencing or skin color detection
384 x 288 on a PentiumIII 700 MHz
Working only with a single grey scale image
Image difference in video sequences

5. The broad practical applications for a extremely fast face detector
User Interface, Image Databases, Teleconferencing
The system can be implemented on a small low power devices.
Compaq iPaq  2 frame/sec

6. Training process for classifier
The attentional operator is trained to detect examples of a particular class --- a supervised training process
Face classifier is constructed
In the domain of face detection
< 1% false negative
<40% false postivie

7. Cascaded detection process
The sub-windows are processed by a sequence of classifiers
each slightly more complex than the last
Any classifier rejects the sub-window,
no further processing is performed
The process is essentially that of a degenerate decision tree

8. Our object detection framework
Feature Evaluation
Haar Basis Functions
Integral Image
Haar Basis Functions
Original Image
Haar Basis Functions
In order to computing
features rapidly at many
scales
Large # of features
Cascaded Classifiers Structure
Feature Selection
Small set of critical features
Modified Ada Boost Procedure

9. Feature Selection The simple features are used
The detection process is based on the feature rather than the pixels directly.
Two Reasons:
The ad-hoc domain knowledge is difficult to learn using a finite quantify of training data.
The feature based system operates much faster
The simple features are used
The Haar basis functions which have been used by Papageorgiou et al.[9]

10. Three kinds of features Feature Selection
Two-Rectangle Feature
The difference between the sum of pixels
within two rectangular regions
The region have the same size and shape
And are horizontally or vertically adjacent
The base resolution is 24x24
The exhaustive set of rectangle is large, over 180,000.
Three-Rectangle Feature
the sum within two outside rectangle subtracted from the sum in a center rectangle
The difference between the diagonal pairs of rectangles
Four-Rectangle Feature

11. The recurrences pair for one pass computing
A intermediated representation for rapidly computing the rectangle features
Integral Image
The integral image
The original image
The recurrences pair for one pass computing i
The cumulative row sum 1 2 5 3 4 6 7 8 9 + 3 1 ii s 9 1 3 8 4 10 21 11 25 45 1 2 5 4 6 11 14 20 4 + +

12. Calculating any rectangle sum with integral image
2  A + B
3  A + C
4  A + B + C + D
Rectangle Sum
D =

13. Learning Classification Functions
Feature Set
Learning Process
Face
A variant AdaBoost procedure
non-
Face
Weak
Learner 1
Learner 2
The final strong classifier
Training set
Positive
Negative
The final strong classifier
AdaBoost learning algorithm
 Is used to do the feature selection task 24
Over 180,000 rectangle features associate with each sub-image 24

14. The Boost algorithm for classifier learning
Image
Step 1: Giving example images
The Boost algorithm for classifier learning
Positive =1
Negative=0
Step 2: Initialize the weights
Weak learner constructor
For t = 1, … , T
1. Normalize the weights,
2. For each feature j, train a classifier hj which is restricted to using a single
feature
3. Update the weights:

15. Weak learner constructor 圖示解說
Training set
Normalized the weights
miss
correct
Update the weights
Over 180,000 features
for each subimage
Features
Errors

16. Training the weak learner 圖解說明
If fj(x) >
X is a face
False positive
False negative ex
X (Training set)
Face examples
Non-Face examples

17. AdaBoosting
Place the most weight on the examples must often misclassified by the preceding weak rules
Forcing the base learner to focus its attention on the “hardest” examples

18. The Boost algorithm for classifier learning
Step 1: Giving example images
Step 2: Initialize the weights
Weak learner constructor
For t = 1, … , T
1. Normalize the weights,
2. For each feature j, train a classifier hj which is restricted to using a single
feature
3. Update the weights:
Selected the weaker classifiers
Final strong classifier

19. The Big Picture on testing process
Stage 1
Ada Boosting Learner
Stage 2
Pass
False (Reject)
Ada Boosting Learner
Stage 3
Pass
False (Reject)
Ada Boosting Learner
Feature Select
& Classifier
Feature set
100% Detection Rate
50% False Positive
False (Reject)
Reject as many negatives as possible  (minimize the false negative)

20. A tremendously difficult problem
How to determine
The number of classifier stages
The number of features in each stages
The threshold of each stage

21. Ada Boosting Learner Pass Ada Boosting Learner False (Reject)
Training example
Stage 1
Stage 2
Ada Boosting Learner
Pass
Ada Boosting Learner
100% Detection Rate
50% False Positive
Feature Select
& Classifier
False (Reject)
False (Reject)
face
Non-face

22. Result
A 38 layer cascaded classifier was trained to detect frontal upright faces
Training set:
Face: 4916 hand labeled faces with resolution 24x24.
Non-face: 9544 images contain no face.
(350 million subwindows within these non-face images)
Features
The first five layers of the detector: 1, 10, 25, 25 and 50 features
Total # of features in all layer  6061

23. Result Each classifier in the cascade was trained
Face : the vertical mirror image  9832 images
Non-face sub-windows: 10,000 (size=24x24)

24. Outline Result Speed of the final Detector Image Processing
Scanning the Detector
Integration of Multiple Detector
Experiments on a Real-World Test Set

25. Speed of the final Detector Result
The speed is directly related to the number of features evaluated per scanned sub-window.
MIT+CMU test set
An average of 10 features out of a total 6061 are evaluated per sub-window.
On a 700Mhz PentiumIII, a 384 x 288 pixel image in about .067 seconds (using a staring scale of 1.25 and a step size of 1.5)

26. Image Processing Result
Minimize the effect of different lighting-conditions
Variance normalized
reference:

27. Scanning the Detector Result
The final detector is scanned across the image at multiple scale and locations
Good results are obtained using a set of scales a factor of 1.25 apart
Locations are obtained by shifting the window some pixels
If the current scale is s, the window is shifted by
Scale is achieved by scaling the detector itself rather than the image
is the rounding operation

28. Integration of Multiple Detector Result
Multiple detections will usually occur around each face and some types of false positives.
A post-process to detected sub-windows in order to combine overlapping detections into a single detection
Two detections are in the same subset if their bounding regions overlap

29. Experiments on a Real-World Test Set Result
The MIT+CMU frontal face test set consists
of 130 images with 507 labeled frontal faces

30. Experiments on a Real-World Test Set Result
Our detector
Detection rates for various numbers of false positives on the MIT+ CMU test set
containing 130 images and 507 faces.

31. ROC curve for the face detector on MIT+CMU test set
75,081,800 sub-windows
scanned
ROC curve for the face detector on MIT+CMU test set
Correct detection rate
False Positive
The detector was run using a step size of 1.0 and starting scale of 1.0

32. A simple voting scheme to further improve results Result
Running three detectors
The 38 layer one described above plus two similarly trained detectors
Output the majority vote of three detectors
The improvement would be greater if the detectors were more independent.

33. Output of our face detector from the MIT+CMU test set

34. Conclusion
A object detection approach minimizes computation time while achieving high detection rate
This paper brings together new algorithms, representations and insights which are quite generic
The detector is approximately 15 times faster than previous approach

35. Conclusion
The database set includes faces under very wide range of conditions including: illumination, scale, pose, and camera variation