1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2010 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall10/ann/ Lecture 1 September 13, 2010

2. S. Mandayam/ ANN/ECE Dept./Rowan University http://www.youtube.com/watch?v=gy5g33S0Gzo March 17, 2010

3. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan What is artificial intelligence? Course introduction Historical development – the neuron model The artificial neural network paradigm What is knowledge? What is learning? The Perceptron Widrow-Hoff Learning Rule The “Future”….?

4. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Intelligence Systems that think like humans Cognitive modeling Systems that think rationally Logic Systems that act like humans Natural language processing Knowledge representation Machine learning Systems that act rationally Decision theoretic agents

5. S. Mandayam/ ANN/ECE Dept./Rowan University Course Introduction Why should we take this course? PR, Applications What are we studying in this course? Course objectives/deliverables How are we conducting this course? Course logistics http://engineering.rowan.edu/shreek/fall10/ann/

6. S. Mandayam/ ANN/ECE Dept./Rowan University Course Objectives At the conclusion of this course the student will be able to: Identify and describe engineering paradigms for knowledge and learning Identify, describe and design artificial neural network architectures for simple cognitive tasks

7. S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

8. S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

9. S. Mandayam/ ANN/ECE Dept./Rowan UniversityHistory/People 1940’sTuringGeneral problem solver, “Turing test” 1940’sShannonInformation theory 1943McCulloch and PittsMath of neural processes 1949HebbLearning model 1959RosenblattThe “Perceptron” 1960WidrowLMS training algorithm 1969Minsky and PapertPerceptron deficiency 1985RumelhartFeedforward MLP, backprop 1988Broomhead and LoweRadial basis function neural nets 1990’sVLSI implementations 1997IEEE 1451 2000HondaAsimo robot

10. S. Mandayam/ ANN/ECE Dept./Rowan University Neural Network Paradigm Stage 1: Network Training ArtificialNeuralNetwork Present Examples Indicate Desired Outputs Determine Synaptic Weights ArtificialNeuralNetwork New Data Predicted Outputs Stage 2: Network Testing “knowledge”

11. S. Mandayam/ ANN/ECE Dept./Rowan University ANN Model ArtificialNeuralNetwork x Input Vector y Output Vector f Complex Nonlinear Function f(x) = y “knowledge”

12. S. Mandayam/ ANN/ECE Dept./Rowan University Popular I/O Mappings ANN x y Single output y1y1 ANN x 1-out-of-c selector y2y2 ycyc y1y1 ANN x Coder y2y2 ycyc ANN x Associator y

13. S. Mandayam/ ANN/ECE Dept./Rowan University The Perceptron    (.) w k1 w k2 w km x1x1 x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k Output, y k ukuk Activation/ squashing function

14. S. Mandayam/ ANN/ECE Dept./Rowan University Activation Functions Threshold Sigmoid

15. S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

16. S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

17. S. Mandayam/ ANN/ECE Dept./Rowan University Error-Correction Learning   (.) w k1 (n) x 1 (n) x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k (n) Activation/ squashing function w k2 (n) w km (n)  Output, y k (n) Desired Output, d k (n) Error Signal e k (n) + -

18. S. Mandayam/ ANN/ECE Dept./Rowan University Learning Tasks Pattern Association Pattern Recognition Function Approximation Filtering Classification x1x1 x2x2 1 2 DB x1x1 x2x2 1 2

19. S. Mandayam/ ANN/ECE Dept./Rowan University Perceptron Training Widrow-Hoff Rule (LMS Algorithm) w(0) = 0 n = 0 y(n) = sgn [w T (n) x(n)] w(n+1) = w(n) +  [d(n) – y(n)]x(n) n = n+1 Matlab Demo

20. S. Mandayam/ ANN/ECE Dept./Rowan University The Age of Spiritual Machines When Computers Exceed Human Intelligence by Ray Kurzweil | Penguin paperback | 0-14-028202-5 |

21. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary