1. 2. Matrix-Vector Formulation of Backpropagation Learning
(1) 2-Layer Case x i w
(1) w
(2) j = Φ [ W ( 2 ) Φ ( W ( 1 ) x )] f ( 1 ) i w j w k ij jk n f ( 1 ) f ( 2 ) n f ( 2 ) = f h
h+1 n m
desired
actual
n+1 e
(1) e
(2) e = -
Let

2. Then
Note :

3. l (2) L-layer Extension With Boundary Conditions For layer Virtual           
Virtual

4. Computational Complexity of (n, h, m) MLP
per Learning Cycle in # of Operations
for Logistic.
Mult.
Add.
Forward
Inner
Outer
h (n + 1)
m (h + 1) hn hm h m
Backward e
e(2) = * e
h ( n + 1)
h (m – 1)
Total
h(2n + 3m + 4) + 5m
h(2m + n +1) + 3m
h + m
For n=5, m=5, h=5,
170 M, 95A,
10 φ

5. What if φ is non-differentiable ?
How many hidden layers are suitable for real time use ?
Does the output error become more uncertain in the case of complex multilayer than simple layer ?
Should we use only up to 3 layers ?
How much does the bias weight affect the overall computation ?
Any other algorithm than BP to train the MLP ?