S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University Lecture 2 September 20, 2010

S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan Recall: Neural Network Paradigm Recall: Perceptron Model Learning Processes Rules Paradigms Tasks Perceptron Training Algorithm Widrow-Hoff Rule (LMS Algorithm) Lab Project 1

S. Mandayam/ ANN/ECE Dept./Rowan University Recall: Neural Network Paradigm Stage 1: Network Training ArtificialNeuralNetwork Present Examples Indicate Desired Outputs DetermineSynapticWeights ArtificialNeuralNetwork New Data Predicted Outputs Stage 2: Network Testing “knowledge”

S. Mandayam/ ANN/ECE Dept./Rowan University Recall: ANN Model ArtificialNeuralNetwork x Input Vector y Output Vector f Complex Nonlinear Function f(x) = y “knowledge”

S. Mandayam/ ANN/ECE Dept./Rowan University Recall: The Perceptron Model    (.) w k1 w k2 w km x1x1 x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k Output, y k ukuk Activation/ squashing function

S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Rules Error Correction Learning Delta Rule or Widrow-Hoff Rule Memory Based Learning Nearest Neighbor Rule Hebbian Learning Competitive Learning Boltzman Learning

S. Mandayam/ ANN/ECE Dept./Rowan University Error-Correction Learning   (.) w k1 (n) x 1 (n) x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k (n) Activation/ squashing function w k2 (n) w km (n)  Output, y k (n) Desired Output, d k (n) Error Signal e k (n) + -

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Paradigms Environment (Data) Teacher (Expert)  ANN error desired actual + - Supervised Unsupervised

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Paradigms Supervised Unsupervised Environment (Data) Delay ANN Delayed Reinforcement Learning Cost Function

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Tasks Pattern Association Pattern Recognition Function Approximation Filtering Classification x1x1 x2x2 1 2 DB x1x1 x2x2 1 2

S. Mandayam/ ANN/ECE Dept./Rowan University Perceptron Training Widrow-Hoff Rule (LMS Algorithm) w(0) = 0 n = 0 y(n) = sgn [w T (n) x(n)] w(n+1) = w(n) +  [d(n) – y(n)]x(n) n = n+1 Matlab Demo

S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 1 /fall10/ann/lab1.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab1.html

S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 1 Double-moon Classification Problem

S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 1 Double-moon Classification Problem d = 1; linearly separable d = -4; NOT linearly separable

S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 1 /fall10/ann/lab1.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab1.html UCI Machine Learning Repository: Face Recognition: Generate images

S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary