Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ming-Feng Yeh1 CHAPTER 4 Perceptron Learning Rule.

Similar presentations

Presentation on theme: "Ming-Feng Yeh1 CHAPTER 4 Perceptron Learning Rule."— Presentation transcript:

1 Ming-Feng Yeh1 CHAPTER 4 Perceptron Learning Rule

2 Ming-Feng Yeh2 Objectives How do we determine the weight matrix and bias for perceptron networks with many inputs, where it is impossible to visualize the decision boundaries? The main object is to describe an algorithm for training perceptron networks, so that they can learn to solve classification problems.

3 Ming-Feng Yeh3 History : 1943 Warren McCulloch and Walter Pitts introduced one of the first artificial neurons in The main feature of their neuron model is that a weighted sum of input signals is compared to a threshold to determine the neuron output. in principle They went to show that networks of these neurons could, in principle, compute any arithmetic or logic function. Unlike biological networks, the parameters of their networks had to designed, as no training method was available.

4 Ming-Feng Yeh4 History : 1950s Frank Rosenblatt and several other researchers developed a class of neural networks called perceptrons in the late 1950s. Rosenblatt’s key contribution was the introduction of a learning rule for training perceptron networks to solve pattern recognition problems. The perceptron could even learn when initialized with random values for its weights and biases.

5 Ming-Feng Yeh5 History : ~1980s Marvin Minsky and Seymour Papert (1969) demonstrated that the perceptron networks were incapable of implementing certain elementary functions (e.g., XOR gate). It was not until the 1980s that these limitations were overcome with improved (multilayer) perceptron networks and associated learning rules. The perceptron network remains a fast and reliable network for the class of problems that it can solve.

6 Ming-Feng Yeh6 Learning Rule Learning rule: a procedure (training algorithm) for modifying the weights and the biases of a network. The purpose of the learning rule is to train the network to perform some task. Supervised learning, unsupervised learning and reinforcement (graded) learning.

7 Ming-Feng Yeh7 Supervised Learning The learning rule is provided with a set of examples (the training set) of proper network behavior: {p 1,t 1 }, {p 2,t 2 },…, {p Q,t Q } where p q is an input to the network and t q is the corresponding correct (target) output. As the inputs are applied to the network, the network outputs are compared to the targets. The learning rule is then used to adjust the weights and biases of the network in order to move the network outputs closer to the targets.

8 Ming-Feng Yeh8 Supervised Learning

9 Ming-Feng Yeh9 Reinforcement Learning The learning rule is similar to supervised learning, except that, instead of being provided with the correct output for each network input, the algorithm is only given a grade. The grade (score) is a measure of the network performance over some sequence of inputs. It appears to be most suited to control system applications.

10 Ming-Feng Yeh10 Unsupervised Learning The weights and biases are modified in response to network inputs only. There are no target outputs available. Most of these algorithms perform some kind of clustering operation. They learn to categorize the input patterns into a finite number of classes. This is especially in such applications as vector quantization.

11 Ming-Feng Yeh11 Two-Input / Single-Neuron Perceptron n = 0 The boundary is always orthogonal to W.  point toward

12 Ming-Feng Yeh12 Perceptron Network Design  The input/target pairs for the AND gate are AND Dark circle  : 1 Light circle  : 0 Step 1: Select a decision boundary Step 2: Choose a weight vector W that is orthogonal to the decision boundary Step 3: Find the bias b, e.g., picking a point on the decision boundary and satisfying n = Wp + b = 0

13 Ming-Feng Yeh13 Test Problem  The given input/target pairs are Two-input and one output network without a bias  the decision boundary must pass through the origin.  The length of the weight vector does not matter; only its direction is important.  Dark circle  : 1 Light circle  : 0 p1p1 p2p2 p3p3

14 Ming-Feng Yeh14 Constructing Learning Rule  Training begins by assigning some initial values for the network parameters.  p1p1 p2p2 p3p3  The network has not returned the correct value, a = 0 and t 1 = 1.  The initial weight vector results in a decision boundary that incorrectly classifies the vector p 1.

15 Ming-Feng Yeh15 Constructing Learning Rule p1p1 p2p2 p3p3  One approach would be set W equal to p 1 such that p 1 was classified properly in the future.  Unfortunately, it is easy to construct a problem for which this rule cannot find a solution.

16 Ming-Feng Yeh16 Constructing Learning Rule p1p1 p2p2 p3p3 Another way would be to add W equal to p 1. Adding p 1 to W would make W point more in the direction of p 1.

17 Ming-Feng Yeh17 Constructing Learning Rule  The next input vector is p 2. A class 0 vector was misclassified as a 1, a = 1 and t 2 = 0.  p1p1 p2p2 p3p3

18 Ming-Feng Yeh18 Constructing Learning Rule  Present the 3rd vector p 3 p1p1 p2p2 p3p3 A class 0 vector was misclassified as a 1, a = 1 and t 2 = 0. 

19 Ming-Feng Yeh19 One iteration Constructing Learning Rule  If we present any of the input vectors to the neuron, it will output the correct class for that input vector. The perceptron has finally learned to classify the three vectors properly.  The third and final rule:  Training sequence: p 1  p 2  p 3  p 1  p 2  p 3  

20 Ming-Feng Yeh20 Unified Learning Rule  Perceptron error: e = t – a

21 Ming-Feng Yeh21 Training Multiple-Neuron Perceptron Learning rate  :

22 Ming-Feng Yeh22 Apple/Orange Recognition Problem

23 Ming-Feng Yeh23 Limitations The perceptron can be used to classify input vectors that can be separated by a linear boundary, like AND gate example.  linearly separable (AND, OR and NOT gates) Not linearly separable, e.g., XOR gate

24 Ming-Feng Yeh24 Solved Problem P4.3 (1,2) (1,1) (2,0) (2,1) (  1,2) (  2,1) (  1,  1) (  2,  2) Class 1: t = (0,0) Class 2: t = (0,1) Class 4: t = (1,1) Class 3: t = (1,0) Design a perceptron network to solve the next problem A two-neuron perceptron creates two decision boundaries.

25 Ming-Feng Yeh25 Solution of P4.3 Class 1Class 2 Class 3 Class x in x out y in y out

26 Ming-Feng Yeh26 Solved Problem P4.5 Train a perceptron network to solve P4.3 problem using the perceptron learning rule.

27 Ming-Feng Yeh27 Solution of P4.5

28 Ming-Feng Yeh28 Solution of P4.5

29 Ming-Feng Yeh29 Solution of P x in x out y in y out

Download ppt "Ming-Feng Yeh1 CHAPTER 4 Perceptron Learning Rule."

Similar presentations

Ads by Google