1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2006 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall06/ann/ Lecture 1 September 18, 2006

2. S. Mandayam/ ANN/ECE Dept./Rowan University Japan's humanoid robots Better than people Dec 20th 2005 | TOKYO From The Economist print edition Why the Japanese want their robots to act more like humans

3. S. Mandayam/ ANN/ECE Dept./Rowan University Why the Japanese want their robots to act more like humans HER name is MARIE, and her impressive set of skills comes in handy in a nursing home. MARIE can walk around under her own power. She can distinguish among similar-looking objects, such as different bottles of medicine, and has a delicate enough touch to work with frail patients. MARIE can interpret a range of facial expressions and gestures, and respond in ways that suggest compassion. Although her language skills are not ideal, she can recognise speech and respond clearly. Above all, she is inexpensive. Unfortunately for MARIE, however, she has one glaring trait that makes it hard for Japanese patients to accept her:

4. S. Mandayam/ ANN/ECE Dept./Rowan University Why the Japanese want their robots to act more like humans ………………….she is a flesh-and-blood human being from the Philippines. If only she were a robot instead.

5. S. Mandayam/ ANN/ECE Dept./Rowan University Harveian Oration In celebration of cerebration by Professor Colin Blakemore, presented at the Royal College of Physicians, London, UK, on Oct 18, 2005 www.thelancet.comwww.thelancet.com Vol 366 Dec 10, 2005

6. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan What is artificial intelligence? Course introduction Historical development – the neuron model The artificial neural network paradigm What is knowledge? What is learning? The Perceptron Widrow-Hoff Learning Rule The “Future”….?

7. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Intelligence Systems that think like humans Cognitive modeling Systems that think rationally Logic Systems that act like humans Natural language processing Knowledge representation Machine learning Systems that act rationally Decision theoretic agents

8. S. Mandayam/ ANN/ECE Dept./Rowan University Course Introduction Why should we take this course? PR, Applications What are we studying in this course? Course objectives/deliverables How are we conducting this course? Course logistics http://engineering.rowan.edu/shreek/fall06/ann/

9. S. Mandayam/ ANN/ECE Dept./Rowan University Course Objectives At the conclusion of this course the student will be able to: Identify and describe engineering paradigms for knowledge and learning Identify, describe and design artificial neural network architectures for simple cognitive tasks

10. S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

11. S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

12. S. Mandayam/ ANN/ECE Dept./Rowan UniversityHistory/People 1940’sTuringGeneral problem solver, “Turing test” 1940’sShannonInformation theory 1943McCulloch and PittsMath of neural processes 1949HebbLearning model 1959RosenblattThe “Perceptron” 1960WidrowLMS training algorithm 1969Minsky and PapertPerceptron deficiency 1985RumelhartFeedforward MLP, backprop 1988Broomhead and LoweRadial basis function neural nets 1990’sVLSI implementations 1997IEEE 1451

13. S. Mandayam/ ANN/ECE Dept./Rowan University Neural Network Paradigm Stage 1: Network Training ArtificialNeuralNetwork Present Examples Indicate Desired Outputs Determine Synaptic Weights ArtificialNeuralNetwork New Data Predicted Outputs Stage 2: Network Testing “knowledge”

14. S. Mandayam/ ANN/ECE Dept./Rowan University ANN Model ArtificialNeuralNetwork x Input Vector y Output Vector f Complex Nonlinear Function f(x) = y “knowledge”

15. S. Mandayam/ ANN/ECE Dept./Rowan University Popular I/O Mappings ANN x y Single output y1y1 ANN x 1-out-of-c selector y2y2 ycyc y1y1 ANN x Coder y2y2 ycyc ANN x Associator y

16. S. Mandayam/ ANN/ECE Dept./Rowan University The Perceptron    (.) w k1 w k2 w km x1x1 x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k Output, y k ukuk Activation/ squashing function

17. S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

18. S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

19. S. Mandayam/ ANN/ECE Dept./Rowan University Error-Correction Learning   (.) w k1 (n) x 1 (n) x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k (n) Activation/ squashing function w k2 (n) w km (n)  Output, y k (n) Desired Output, d k (n) Error Signal e k (n) + -

20. S. Mandayam/ ANN/ECE Dept./Rowan University Learning Tasks Pattern Association Pattern Recognition Function Approximation Filtering Classification x1x1 x2x2 1 2 DB x1x1 x2x2 1 2

21. S. Mandayam/ ANN/ECE Dept./Rowan University Perceptron Training Widrow-Hoff Rule (LMS Algorithm) w(0) = 0 n = 0 y(n) = sgn [w T (n) x(n)] w(n+1) = w(n) +  [d(n) – y(n)]x(n) n = n+1 Matlab Demo

22. S. Mandayam/ ANN/ECE Dept./Rowan University The Age of Spiritual Machines When Computers Exceed Human Intelligence by Ray Kurzweil | Penguin paperback | 0-14-028202-5 |

23. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary