1. Artificial Neural Networks By: Steve Kidos

2. Outline Artificial Neural Networks: An Introduction Frank Rosenblatt’s Perceptron Multi-layer Perceptron Dot Example Learning and Cost Functions Learning Paradigms Implementations Applications Google Example

3. Artificial Neural Networks: An Introduction Artificial Neural Networks are modeled after the biological brain Consists of interconnected nodes, or “neurons” Uses parallel, nonlinear processing Neural networks are adaptive and experience-based Most importantly, a learning system

4. Frank Rosenblatt’s Perceptron In 1957 Frank Rosenblatt designed and implemented the first neural network, a single-layer perceptron Each input neuron is connected to every output neuron with each connection having a weight Learning is accomplished through modifying connection weights

5. Multi-layer Perceptrons Single-layer Perceptrons have limited functionality, a point Marvin Minsky drove home within the AI community Multi-layer Perceptrons are much more robust and are able to perform functions a single-layer perceptron cannot Allows for a much greater degree of complexity, allowing the network to deal with more complex problems

6. Dot Example 1 st Generation No training Essentially random behavior Sometimes they get lucky 100 th Generation Successful at finding white dot Very efficient

7. Learning and Cost Functions In the Dot example, there are a limited number of white dots per generation. This represents the cost. Each neural network has an associated cost, where the more white dots consumed, the less cost they have Dots with least cost survive to next generation “Mutated” dots are created from fit dots and have slightly different weights Learning is accomplished through mutations and cost (genetic algorithm)

8. Learning Paradigms Supervised learning, or “learning with a teacher”, uses pre-defined sets of input-output and corrects the network based on it’s output Reinforcement learning generates a cost associated with the network’s actions. The network learns by trying to minimize cost. Differs from supervised in that there is no explicit “expected” output. Dot example used this Unsupervised learning is generally self-organizing and only very specific networks use this method. It doesn’t have a teacher or reinforcement. It seeks to find structure in unlabeled data

9. Implementations of Neural Networks Multi-layer Feedforward Perceptron Self-Organizing Map Recurrent Networks Modular Neural Networks

10. Applications Function Approximation and Regression Pattern Recognition and Classification Robotics (controlling movement) NLP

11. Google’s Neural Network Used for visual classification Required 16,000 processors over 1,000 computers to process 10,000,000 images grabbed from YouTube Originally used to recognize cats, now being used to recognize speech, faces, and associate related text and images Visual classification outperforms humans in some instances

12. End

13. Sources http://www.cse.unsw.edu.au/~billw/mldict.html http://www.learnartificialneuralnetworks.com/ http://arxiv.org/ftp/cs/papers/0308/0308031.pdf http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vo l4/cs11/report.htmlhttp://www.doc.ic.ac.uk/~nd/surprise_96/journal/vo l4/cs11/report.html http://www.technologyreview.com/news/429442/g oogle-puts-its-virtual-brain-technology-to-work/http://www.technologyreview.com/news/429442/g oogle-puts-its-virtual-brain-technology-to-work/