Artificial Neural Networks (ANNs) and the Error Backpropagation Procedure Prof. Carolina Ruiz Department of Computer Science Worcester Polytechnic Institute

A 2-layer feedforward ANN Input hidden layer output layer -1 -1 -1

Error Backpropagation Out 1 1. Initialize the weights to small random values -1 0.5 0.1 A C -0.2 -0.1 E 0.05 0.3 D B 0.2 0.5

Error Backpropagation Out 1 2. For each of the examples: 2.1. Present example to input layer 2.2. Propagate the example forward -1 0.5 0.1 0.377 A C -0.2 -0.1 E 0.5094 0.05 0.3 D 0.377 B 0.2 0.5

Error Backpropagation Out 1 2. For each of the examples: 2.3. Compute node errors for output layer 2.4. Compute node errors for hidden layer -1 0.025 0.5 0.1 0.377 A C -0.2 -0.5094 -0.1 E 0.5094 0.05 0.3 D 0.377 B 0.2 -0.0382 0.5

Error Backpropagation Out 1 2. For each of the examples: 2.5. Compute and record weight change for each connection -1 0.025 0.5 0.1 0.377 A C -0.2 -0.5094  A->C 0.0000  A->D  B->C  B->D  C->E -0.0481  D->E -0.1 E 0.5094 0.05 0.3 D 0.377 B 0.2 -0.0382 0.5

Error Backpropagation Out 1 3. After processing all examples update weight 4. Repeat process until obtaining “good” weights -1 0.025 0.5 0.1 0.377 A C -0.2 -0.5094  A->C 0.0001  A->D -0.0795  B->C 0.0004  B->D -0.0863  C->E 0.3853  D->E -0.049 -0.1 E 0.5094 0.05 0.3 D 0.377 B 0.2 -0.0382 0.5