2IntroductionPattern association involves associating a new pattern with a stored pattern.Is a “simplified” model of human memory.Types of associative memory:Heteroassociative memoryAutoassociative memoryHopfield NetBidirectional Associative Memory (BAM)
3Introduction These are usually single-layer networks. The neural network is firstly trained to store a set of patterns in the form s : ts represents the input vector and t the corresponding output vector.The neural network is then tested on a set of data to test its “memory” by using it to identify patterns containing incorrect or missing information.
4Introduction Associative memory can be feedforward or recurrent. Autoassociative memory cannot hold an infinite number of patterns. Factors that affect this:Complexity of each patternSimilarity of input patterns
6Heteroassociative Memory The inputs and output vectors s and t are different.The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair.The heteroassociative application algorithm is used to test the algorithm.
7The Hebb AlgorithmInitialize weights to zero, wij =0, where i = 1, …, n and j = 1, …, m.For each training case s:t repeat:xi = si , where i=1,...,nyi = tj, where j = 1, .., mAdjust weights wij(new) = wij(old) + xiyj, where i = 1, .., n and j = 1, .., m
8ExerciseTrain a heteroassociative neural network to store the following input and output vectors:Test the neural network using all input data and the following input vector:
9Autoassociative Memory The inputs and output vectors s and t are the same.The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair.The autoassociative application algorithm is used to test the algorithm.
11ExerciseStore the pattern in an autoassociative neural network.Test the neural network on the following input:
12The Hopfield Neural Network Is a recurrent associative memory neural network.Application algorithmExercise: Store the pattern [ ] using a Hopfield neural network. Test the neural network to see whether it is able to correctly identify an input vector with two mistakes in it: [ ]. Note θi=0, for i=1,..,4
13Bidirectional Associative Memory (BAM) Consists of two layers, x and y.Signals are sent back and forth between both layers until an equilibrium is reached.An equilibrium is reached if the x and y vectors no longer change.Given an x vector the BAM is able to produce the y vector and vice versa.Application algorithm
14BAM ExerciseStore the vectors representing the following patterns using a BAM:[ ] with the output vector [1 -1][ ] with the output vector [-1 1]θi=0, θj=0 for i = 1,..3 and j=1..2