1. Analysis of Classification Algorithms In Handwritten Digit Recognition Logan Helms Jon Daniele

2. Classification Algorithms Template Matching Naïve Bayes Classifier Neural Network

3. Benchmarks 1.Gradient-Based Learning Applied to Document Recognition by LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. 2.Comparison of Machine Learning Classifiers for Recognition of Online and Offline Handwritten Digits by Omidiora, E., Adeyanju, I., Fenwa, O.

4. MNIST Training Set:60,000 samples Test Set:10,000 samples Accuracy: Number of correctly guessed test samples/ 10,000

5. NAÏVE BAYES CLASSIFIER

6. Naïve Bayes Classifier Each pixel value (on/off) is independent of any other pixel value​ Each pixel has a probability associated with being on or off in any given digit class​ The probability of each pixel is used to determine the probability of an unknown digit being classified in one of the known classes

7. Naïve Bayes Classifier Training set: 60000 digits​ Test set: 10000 digits​ Success rate:​ Abysmal: 08.13% correct classification rate​ Benchmark:​ WEKA: Multimodal Naive Bayes: 83.65%​ 08.13% <<<<<<<<< 83.65%

8. Naïve Bayes Classifier Challenges:​ Pixel probabilities change according to the shape of the digit​ Are pixels the best feature set by which to compare different digits?​ Input size:​ 28x28 digit image results 786 pixels​ Requirements for matrix manipulation

9. Naïve Bayes Classifier Improvements​ Discarding extraneous pixel data​ Pixel values are mainly contained in a 20x20 matrix​ Using a binary pixel value vs a range of pixel values (0-255)​ Edge detection​ Incorporate feature extractor(s) and evaluate images based on those features

10. NEURAL NETWORK

11. Neural Network Type: Feed Forward Training: Back-propagation algorithm Response Function: Architectures: NameInput LayerHidden LayerOutput Layer NN30078430010 NN1000784100010

12. Training

13. NN300 Training time:~17 hours (~52 mins/epoch) Learning rate: EpochRate 1, 20.0005 3, 4, 50.0002 6, 7, 80.0001 9, 10, 11, 120.00005 13, 14, 15, 16, 17, 18, 19, 200.00001

14. NN1000 Training time:~2.5 days (~3 hrs/epoch) Learning rate: EpochRate 1, 20.0005 3, 4, 50.0002 6, 7, 80.0001 9, 10, 11, 120.00005 13, 14, 15, 16, 17, 18, 19, 200.00001

15. Results After 20 epochs NetworkAccuracy Benchmark 195.30% NN30075.82% Benchmark 275.12% NetworkAccuracy Benchmark 195.50% NN1000-

16. Benchmark 1 95.30%On MNIST test set as is. 96.4%Generated more training data by using artificial distortions 98.4%When using deslanted images

17. Future Work Further training of NN300 with the MNIST test set has increased accuracy to 84.01% Experiment with hidden neuron count and multiple hidden layers Research other types of neural networks