1 Pertemuan 13 BACK PROPAGATION Matakuliah: H0434/Jaringan Syaraf Tiruan Tahun: 2005 Versi: 1

2 Learning Outcomes Pada akhir pertemuan ini, diharapkan mahasiswa akan mampu : Menjelaskan konsep Back Propagation.

3 Outline Materi Algoritma Back Propagation

4 Multilayer Perceptron R – S 1 – S 2 – S 3 Network

5 Example

6 Elementary Decision Boundaries First Subnetwork First Boundary: Second Boundary:

7 Elementary Decision Boundaries Third Boundary: Fourth Boundary: Second Subnetwork

8 Total Network

9 Function Approximation Example Nominal Parameter Values

10 Nominal Response

11 Parameter Variations

12 Multilayer Network

13 Performance Index Training Set Mean Square Error Vector Case Approximate Mean Square Error (Single Sample) Approximate Steepest Descent

14 Chain Rule Example Application to Gradient Calculation

15 Gradient Calculation Sensitivity Gradient

16 Steepest Descent s m F ˆ  n m   F ˆ  n 1 m  F ˆ  n 2 m   F ˆ  n S m m  = Next Step: Compute the Sensitivities (Backpropagation)

17 Jacobian Matrix F Ý m n m  f Ý m n 1 m  0  0 0f Ý m n 2 m  0  00  f Ý m n S m m  =

18 Backpropagation (Sensitivities) The sensitivities are computed by starting at the last layer, and then propagating backwards through the network to the first layer.

19 Initialization (Last Layer) a i  n i M  a i M  n i M  f M n i M  n i M  f Ý M n i M  === s i M 2t i a i –  –f Ý M n i M  =

20 Summary Forward Propagation Backpropagation Weight Update