Presentation on theme: "Presentation By Utkarsh Trivedi Y8544"— Presentation transcript:
1Presentation By Utkarsh Trivedi Y8544 Hopfield NetworksPresentation ByUtkarsh TrivediY8544
2Topics Covered What is Hopfield Network Some interesting facts Major ApplicationsMathematical model of HN’sLearning HNs through examples
3What is Hopfield Network ?? According to Wikipedia, Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. Hopfield nets serve as content-addressable memory systems with binary threshold units. They are guaranteed to converge to a local minimum, but convergence to one of the stored patterns is not guaranteed.
4What are HN (informally) Hopfield NetworkThese are single layered recurrent networksAll the neurons in the network are fedback from all other neurons in the networkThe states of neuron is either +1 or -1 instead of (1 and 0) in order to work correctly.No of the input nodes should always be equal to no of output nodesFollowing figure shows a Hopfield network with four nodes
5Some Interesting Facts …. Hopfield NetworkSome Interesting Facts ….The recall pattern of Hopfield network is similar to our recall mechanism.Both are based on content addressable memoryIf some of the neurons of network are destroyed the performance is degraded but some network capabilities may be retained even with major network damage. Just like our brainsDid you know that we are similar
6Major Applications Recalling or Reconstructing corrupted patterns Hopfield NetworkMajor ApplicationsRecalling or Reconstructing corrupted patternsLarge-scale computational intelligence systemsHandwriting Recognition SoftwarePractical applications of HNs are limited because number of training patterns can be at most about 14% the number of nodes in the network.If the network is overloaded -- trained with more than the maximum acceptable number of attractors -- then it won't converge to clearly defined attractors.
7Mathematical Modeling of HN’s Hopfield NetworkMathematical Modeling of HN’s
8Mathematical Modeling of HN’s Hopfield NetworkMathematical Modeling of HN’s
9Mathematical Modeling of HN’s Hopfield NetworkMathematical Modeling of HN’sConsider signum function to be neuron’s activation function.i.e.vi = +1 if hi>0vi = -1 if hi<0
10Mathematical Modeling of HN’s Hopfield NetworkMathematical Modeling of HN’sLiapunov Energy function :-
11Power Of Hopfield Networks We want to understand how to achieve this kind of performance form simple Hopfield networks
12Learning HNs through simple example Hopfield NetworkLearning HNs through simple exampleThere are various ways to train these kinds of networks like back propagation algorithm , recurrent learning algorithm, genetic algorithm but there is one very simple algorithm to train these simple networks called ‘One shot method’.I will be using this algorithm in order to train the network.OaObOcW3,2W1,2W2,1W3,1W1,3W3,3W2,2W2,3W1,1
14Learning HNs through example Hopfield NetworkLearning HNs through exampleMoving onto little more complex problem described in Haykin’s Neural Network BookThey book used N=120 neuron and trained network with 120 pixel images where each pixel was represented by one neuron.Following 8 patterns were used to train neural network.
15Learning HNs through example Hopfield NetworkLearning HNs through exampleIn order to recognizing power of HNsFor this they need corrupted image. They flipped the value of each pixel with p=0.25.Using these corrupted images trained HN was run. And after certain number of iteration the output images converged to one of the learned pattern.Next slides shows the results that they obtained
16Learning HNs through example Hopfield NetworkLearning HNs through example
17Learning HNs through example Hopfield NetworkLearning HNs through example
18Flow Chart summarizing overall process Hopfield NetworkFlow Chart summarizing overall processTrain HN using Standard patternsUpdate weight vectors of NetworkRun the trained network with corrupted patternNetwork returns the decrypted pattern
19Hopfield NetworkShortcomings of HNsTraining patterns can be at most about 14% the number of nodes in the network.If more patterns are used thenthe stored patterns become unstable;spurious stable states appear (i.e., stable states which do not correspond with stored patterns).Can sometimes misinterpret the corrupted pattern.
23Hopfield NetworkQuestions…What is the major difference between HN and fully recurrent networks?What is content addressable memory and how is it different from RAM?What will happen if we train HN for only one pattern?If we train a HN with a pattern will it be automatically trained for its inverse ?Why can’t we increase number of nodes in network in order to overcome its shortcomings?(ignore the increased computation complexity or time)