Neural Networks Chapter 5 Joost N. Kok Universiteit Leiden

Simple Perceptrons Learning = find weights by successive improvement from an arbitrary starting point Supervised learning = learning with a teacher Training set = list of correct input-output pairs

Simple Perceptrons Layered feedforward networks Perceptrons (Rosenblatt 1962) Input units Hidden units Output units

Simple Perceptrons Activation function g Thresholds: fix

Simple Perceptron We want actual output pattern to be equal to target pattern:

Simple Perceptron Hetero-association vs. Auto-association Simple Perceptron = one layer perceptron First consider deterministic threshold units Weight vectors

Simple Perceptrons

Simple Perceptrons Desired:

Simple Perceptrons Problem is solvable by simple perceptrons if the problem is linearly separable No thresholds = separating plane goes through origin XOR problem = Boolean exclusive OR

Simple Perceptrons

Simple Perceptrons Learning rule for Simple Perceptron

Simple Perceptrons

Simple Perceptrons

Simple Perceptrons Linear Units

Simple Perceptrons Gradient Descent Learning

Simple Perceptrons Gradient descent algorithm

Simple Perceptrons Nonlinear Units

Simple Perceptrons