Presentation on theme: "Cascaded Classifier for Automatic Crater Detection Henry Z. Lo Advisor: Wei Ding Domain Scientist: Tomasz Stepinski Knowledge Discovery Lab University."— Presentation transcript:
Cascaded Classifier for Automatic Crater Detection Henry Z. Lo Advisor: Wei Ding Domain Scientist: Tomasz Stepinski Knowledge Discovery Lab University of Massachusetts Boston
Overview Introduction: o Cascading classifier. o Experimental road map. Experiments: o Tests on feature sets. o Tests on positive example training set content. o Tests on negative example training set size. o Tests on negative example training set content. Discussion: o Implications of results. o Unresolved issues. o Future directions.
Cascading Classifier Architecture: o Layers of Adaboost classifiers. o Each layer trained on the FP of previous layer. o Input must be accepted by all, sequentially, to be considered a crater. o Rejection can happen at any stage.
Cascading Classifier Features: o Exclusively uses Haar-like features. o Can be calculated in constant time. o Contrast based. o Scanned over entire subwindow.
Cascading Classifier Implementation: o Used OpenCV implementation. o Free and open source. o Many variables: Number of layers. "Minimum hit rate" - false positive rate. "Max false alarm" - false negative rate. 3 feature sets.
Experimental Road Map Tweak for performance: o OpenCV parameters. o Features. o Training set. The following OpenCV parameters improve performance: o Minimum hit rate. o Max false alarm. o Number of layers. Still need to tweak features and training sets for: o Training time. o Generalizability. L
Experimental Road Map
Each of these factors will be tested individually for effect on precision, recall, and F1. We avoid studying interaction effects for simplicity. In the future, we will investigate how to combine different features and test sets for optimal result. Experimental Road Map
We use tile 3-24 for both training and testing. This tile was chosen for its relatively smooth surface. Future studies will test on other tiles as well. Experimental Road Map
Feature Set Variation
OpenCV offers 3 different feature sets: o CORE: 1a, 1b, 2a, 2c. o BASIC: CORE + 2b, 2d, 3a o ALL: all features Since ALL is a superset of CORE and BASIC, it should perform best.
Feature Set Variation In recall, CORE and BASIC outperformed ALL. In precision and F1, the exact opposite was true.
Inclusion of tilted features beneficial to performance. More features than those given may provide further benefit. It is not obvious how to create Haar features in OpenCV. Postponing creation of specialized Haar features.
Ground Truth Windows
Positive examples contained tightly cropped craters. No crater rims or surrounding area. Experimented with including area around craters. Range: 1x crater radius - 2x crater radius, in steps of
Ground Truth Windows As the subwindow increased, precision and F1 increased. However, recall suffered.
Negative Example Set Size
All classifiers tested were trained on 300 negative examples. By providing the classifier with more negative examples, we give it more information. Performance should increase with more negative examples. Tested classifiers trained on 300, 400, 500, 600, and 700 negative examples.
Negative Example Set Size F1 and precision increase with more negative examples. Recall decreases.
Negative Example Manipulation
The idea is to put some false positives back into the training set. This will teach the classifier using its own mistakes. However, selecting the false positives is rather difficult, as we will see later.
Result Implications Window scaling has the most noticeable effect on F1, recall, and precision. Next most important is the feature set used. The number of negative training examples is the least important; however, this may be due to the small range of values being tested.
Future Directions Once optimal features and training sets are found, we can manipulate OpenCV variables. Recall that the classifier may be improved by the following: o More layers in the classifier. o Setting the minimum hit rate (recall). o Setting the max false alarm rate (precision). Time complexity of classifier training requires further study.
Future Directions Further exploration of cascaded classification algorithm: o Testing classifier on other tiles. Exploration of other object detection algorithms. o Neural networks.