1. ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

2. Deep Convolutional Neural Networks Led by Geoffrey Hinton, University of Toronto Published in 2013 Based on the datasets from ImageNet LSVRC-2010 Contest Using graphic cards to train the neural network

3. ImageNet LSVRC-2010 Contest 1.2 million high-resolution images 1,000 different classes 50,000 validation images, 150,000 testing images Top-1 error 47.1% best in contest, 45.7% best published Top-5 errors 28.2% best in contest, 25.7% best published

4. Convolutional Neural Networks (CNN)

5. Architecture

6. Novel/Unusual features in architecture ReLU (Rectified Linear Units ) Nonlinearity Standard way: f(x) = tanh(x) or f(x) = (1 + e -x ) -1 (logistic function) ReLU: f(x) = max(0, x) 6 times faster than hyperbolic function Multiple GPUs Larger memory Parallel computing Control of communication

7. Overfitting Occurs when a statistical model describes random error or noise instead of the underlying relationship Exaggerate minor fluctuations in the data Will generally have poor predictive performance

8. Reducing overfitting Data Augmentation 1.Image translation and horizontal reflection Randomly extracting patches Four corner and one center patches with reflection for testing 2.Altering the intensities of the RGB channels in training images Approximately captures an important property of natural images reduces the top-1 error rate by over 1%

9. Reducing Overfitting Dropout Zero the output of each hidden neuron with probability 0.5. No longer contribute to forward pass and backward propagation Neural network samples a different architecture every time Reduce complex co-adaptations of neurons Used in two fully-connected layers

10. Result Two NVIDIA GTX 580 3GB GPUs 6 days of training 90 cycles 60 million parameters 37.5% top-1 error (45.7% second best) 17% top-5 error (28.2% second best)

11. Questions?