Download presentation
Presentation is loading. Please wait.
1
11 1 Backpropagation
2
11 2 Multilayer Perceptron R – S 1 – S 2 – S 3 Network
3
11 3 Example
4
11 4 Elementary Decision Boundaries First Subnetwork First Boundary: Second Boundary:
5
11 5 Elementary Decision Boundaries Third Boundary: Fourth Boundary: Second Subnetwork
6
11 6 Total Network
7
11 7 Function Approximation Example Nominal Parameter Values
8
11 8 Nominal Response
9
11 9 Parameter Variations
10
11 10 Multilayer Network
11
11 Performance Index Training Set Mean Square Error Vector Case Approximate Mean Square Error (Single Sample) Approximate Steepest Descent
12
11 12 Chain Rule Example Application to Gradient Calculation
13
11 13 Gradient Calculation Sensitivity Gradient
14
11 14 Steepest Descent s m F ˆ n m ---------- F ˆ n 1 m --------- F ˆ n 2 m --------- F ˆ n S m m ----------- = Next Step: Compute the Sensitivities (Backpropagation)
15
11 15 Jacobian Matrix F Ý m n m f Ý m n 1 m 0 0 0f Ý m n 2 m 0 00 f Ý m n S m m =
16
11 16 Backpropagation (Sensitivities) The sensitivities are computed by starting at the last layer, and then propagating backwards through the network to the first layer.
17
11 17 Initialization (Last Layer) a i n i M ---------- a i M n i M ---------- f M n i M n i M -----------------------f Ý M n i M === s i M 2t i a i – –f Ý M n i M =
18
11 18 Summary Forward Propagation Backpropagation Weight Update
19
11 19 Example: Function Approximation 1-2-1 Network + - t a e p
20
11 20 Network 1-2-1 Network a p
21
11 21 Initial Conditions
22
11 22 Forward Propagation
23
11 23 Transfer Function Derivatives
24
11 24 Backpropagation
25
11 25 Weight Update
26
11 26 Choice of Architecture 1-3-1 Network i = 1i = 2 i = 4i = 8
27
11 27 Choice of Network Architecture 1-5-1 1-2-11-3-1 1-4-1
28
11 28 Convergence 1 2 3 4 5 0 1 2 3 4 5 0
29
11 29 Generalization 1-2-11-9-1
Similar presentations
