Back Propagation and Representation in PDP Networks Psychology 209 February 6, 2013

Homework 4 Part 1 due Feb 13 –Complete Exercises 5.1 and 5.2. –It may be helpful to carry out some explorations of parameters, as suggested in Exercise 5.3. This may help you achieve a solution in the last part of the homework, below. However, no write-up is required for this. Part 2 due Feb 20 –Consult Chapter 8 of the PDP Book by Rumelhart, Hinton, and Williams (In readings directory for Feb 6). Consider the problems described there that were solved using back propagation, and choose one; or create a problem of your own to investigate with back propagation. –Carry out Exercise 5.4, creating your own network, template, pattern, and startup file (similar to bpxor.m), and answer question

The Perceptron For input pattern p, teacher t p and output o p, change the threshold And weights as follows: Note: including bias = - in net and using threshold of 0, then treating bias as a weight from a unit that is always on is equivalent

AND, OR, XOR

Adding a unit to make XOR solvable

Gradient Descent Learning in the ‘LMS’ Associator Output is a linear function of inputs and weights: Find a learning rule to minimize the Summed squared Error: Taking derivatives, we find: Consider the policy: This breaks down into the sum over patterns of terms of the form:

Error Surface for OR function in LMS Associator

What if we want to learn how to solve xor? We need to figure out how to adjust the weights into the ‘hidden’ unit, following the principle of gradient descent:

We start with an even simpler problem Assume units are linear, both weights =.5 and, i = 1, t = 1. We use the chain rule to calculate for each weight w 10 w 21 Weight changes should follow the gradient: First we unpack the chain, then we calculate the elements of it.

Including a non-linear activation function Let Then So our chains from before become:

Including the activation function in the chain rule and including more than one output unit leads to the formulation below, in which we use ‘  i ’ to represent ∂E/∂net i We can continue this back indefinitely…  s = f’(net s )  r  r w rs The weight change rule at every layer is:  w rs =  r a s Calculating the  term for output unit i:  i = (t i -a i )f’(net i ) And the  term for hidden unit j:  j = f’(net j )  i  i w ij i j k

Back propagation algorithm Propagate activation forward –Activation can only flow from lower-numbered units to higher numbered units Propagate “error” backward –Error flows from higher numbered units back to lower numbered units Calculate ‘weight error derivative’ terms =  r a s One can change weights after processing a single pattern or accumulate weight error derivatives over a batch of patterns before changing the weights.

Variants/Embellishments to back propagation Full “batch mode” (epoch-wise) learning rule with weight decay and momentum:  w rs =   p  rp a sp –  w rs +  w rs (prev) Weights can alternatively be updated after each pattern or after every k patterns. An alternative error measure has both conceptual and practical advantages: CE p = -  i [t ip log(a ip ) + (1-t ip )log(1-a ip )] If targets are actually probabilistic, minimizing CE p maximizes the probability of the observed target values. This also eliminates the ‘pinned output unit’ problem.

Why is back propagation important? Provides a procedure that allows networks to learn weights that can solve any deterministic input-output problem. –Contrary to expectation, it does not get stuck in local minima except in cases where the network is exceptionally tightly constrained. Allows networks to learn how to represent information as well as how to use it. Raises questions about the nature of representations and of what must be specified in order to learn them.

Is Backprop biologically plausible? Neurons do not send error signals backward across their weights through a chain of neurons, as far as anyone can tell But we shouldn’t be too literal minded about the actual biological implementation of the learning rule. Some neurons appear to use error signals, and there are ways to use differences between activation signals to carry error information