1. Neural Networks Chapter 5
Joost N. Kok
Universiteit Leiden

2. Simple Perceptrons
Learning = find weights by successive improvement from an arbitrary starting point
Supervised learning = learning with a teacher
Training set = list of correct input-output pairs

3. Simple Perceptrons Layered feedforward networks
Perceptrons (Rosenblatt 1962)
Input units
Hidden units
Output units

5. Simple Perceptrons
Activation function g
Thresholds: fix

6. Simple Perceptron
We want actual output pattern to be equal to target pattern:

7. Simple Perceptron Hetero-association vs. Auto-association
Simple Perceptron = one layer perceptron
First consider deterministic threshold units
Weight vectors

9. Simple Perceptrons

10. Simple Perceptrons
Desired:

11. Simple Perceptrons
Problem is solvable by simple perceptrons if the problem is linearly separable
No thresholds = separating plane goes through origin
XOR problem = Boolean exclusive OR

12. Simple Perceptrons

13. Simple Perceptrons
Learning rule for Simple Perceptron

14. Simple Perceptrons

15. Simple Perceptrons

17. Simple Perceptrons
Linear Units

18. Simple Perceptrons
Gradient Descent Learning

19. Simple Perceptrons
Gradient descent algorithm

20. Simple Perceptrons
Nonlinear Units

21. Simple Perceptrons