Download presentation

Presentation is loading. Please wait.

Published byLexie Ide Modified over 2 years ago

1
Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks Author : Bogdan M. Wilamowski, Fellow, IEEE, Nicholas J. Cotton, Okyay Kaynak, Fellow, IEEE, and Günhan Dündar Source : IEEE INDUSTRIAL ELECTRONICS MAGAZINE Date : 2012/3/28 Presenter : 林哲緯 1

2
Outline Numerical Analysis Method Neuron Network Architectures NBN Algorithm 2

3
Minimization problem 3 Newton's method

4
Minimization problem 4 http://www.nd.com/NSBook/NEURAL%20AND%20ADAPTIVE%20SYSTEMS14_Adaptive_Linear_Systems.html Steepest descent method

5
Least square problem 5 Gauss–Newton algorithm http://en.wikipedia.org/wiki/Gauss%E2%80%93Newton_algorithm

6
Levenberg–Marquardt algorithm – Combine the advantages of Gauss–Newton algorithm and Steepest descent method – far off the minimum like Steepest descent method – Close to the minimum like Newton algorithm – It’s find local minimum not global minimum 6

7
Levenberg–Marquardt algorithm Advantage – Linear – First-order differential Disadvantage – inverting is not used at all 7

8
Outline Numerical Analysis Method Neuron Network Architectures NBN Algorithm 8

9
Weight updating rule 9 Second-order algorithm First-order algorithm α : learning constant g : gradient vector J : Jacobian matrix μ : learning parameter I : identity matrix e : error vector MLPACNFCN

10
Forward & Backward Computation 10 Forward : 12345, 21345, 12435, or 21435 Backward : 54321, 54312, 53421, or 53412

11
Jacobian matrix 11 Row : pattern(input)*output Column : weight p = input number no = output number Row = 2*1 = 2 Column = 8 Jacobin size = 2*8

12
Jacobian matrix 12

13
Outline Numerical Analysis Method Neuron Network Architectures NBN Algorithm 13

14
Direct Computation of Quasi-Hessian Matrix and Gradient Vector 14

15
Conclusion memory requirement for quasi-Hessian matrix and gradient vector computation is decreased by(P × M) times can be used arbitrarily connected neural networks two procedures – Backpropagation process(single output) – Without backpropagation process(multiple outputs) 15

Similar presentations

OK

Dr. Hala Moushir Ebied Faculty of Computers & Information Sciences

Dr. Hala Moushir Ebied Faculty of Computers & Information Sciences

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Differential display ppt online Ppt on unity in diversity and organic farming A ppt on loch ness monster found Ppt on collection of primary data Ppt on brain and spinal cord Ppt on water scarcity definition Ppt on issue of shares and debentures Ppt on all types of motion Ppt on online shopping cart Ppt on 21st century skills teacher