Perceptron Learning Rule

Learning Rules Learning Rules : A procedure for modifying the weights and biases of a net work. Learning Rules : Supervised Learning Reinforcement Learning Unsupervised Learning

Learning Rules Network is provided with a set of examples • Supervised Learning Network is provided with a set of examples of proper network behavior (inputs/targets) {p1, t1}, {p2, t2}, ……{pn, tn} • Reinforcement Learning Network is only provided with a grade, or score, which indicates network performance • Unsupervised Learning Only network inputs are available to the learning algorithm. Network learns to categorize (cluster) the inputs.

Perceptron Architecture W w T 1 2 S = w i 1 , 2 R =

Single-Neuron Perceptron

Decision Boundary

Example - OR

OR Solution Weight vector should be orthogonal to the decision boundary. Pick a point on the decision boundary to find the bias.

Multiple-Neuron Perceptron Each neuron will have its own decision boundary. iwTp+bi =0 A single neuron can classify input vectors into two categories. A S-neuron perceptron can classify input vectors into 2S categories.

Learning Rule Test Problem

Starting Point Random initial weight: Present p1 to the network: Incorrect Classification.

Tentative Learning Rule

Second Input Vector (Incorrect Classification) Modification to Rule:

Patterns are now correctly classified. Third Input Vector (Incorrect Classification) Patterns are now correctly classified.

Unified Learning Rule A bias is a weight with an input of 1.

Multiple-Neuron Perceptrons To update the i-th row of the weight matrix: Matrix form:

Apple/Orange Example Training Set Initial Weights

Apple/Orange Example First Iteration e t 1 a – -1 =

Second Iteration Second Iteration

Third Iteration Third Iteration e t 1 a – -1 =

Check a = hardlim(-3.5) = 0 = t1 a = hardlim(0.5) = 1 = t2

Perceptron Rule Capability The perceptron rule will always converge to weights which accomplish the desired classification, assuming that such weights exist.

Perceptron Limitations Linear Decision Boundary Linearly Inseparable Problems

Example

Example

Example

Example

Example

Example

Example

Example