1. The Hopfield Model
- Jonathan Amazon

2. Neural Network

3. Neural Network Can be modeled as a spin glass. Each neuron is a spin.
Can be in an excited state (s = +1)
or a quiet state (s = -1)
The synapses between neurons are spin couplings.
Can be excitory (ferromagnetic, J > 0)
or inhibitory (anti-ferromagnetic, J < 0)
Experimentally observed: Neurons spend most time in quiet state due to activation threshold.
External field (H < 0) captures this behavior

4. Spin Glass Ising model with non-uniform coupling strength.
Couplings are usually quenched variables drawn from distribution.

5. Hopfield Network Spin glass neural network Completely connected
Couplings are not chosen from a distribution
Pre-defined memory states are encoded into the coupling strengths.
Hebbian rule fixes couplings.
Memory states become minimal energy configurations (mostly).
Gives network associative memory properties.
Memory states are randomly generated by uniform probability of up or down spin.

6. Dynamics Method of Decent. Calculate local field from all other spins
Compare to activation threshold.
H = 0 for my simulation, Implies inversion symmetry of hamiltonian.
Flip accordingly.
Total energy is monotonically decreasing and system tends to a local energy well.

7. Associative Memory
Relaxation from arbitrary starting state to nearest energy minimum.
Hebbian rule: Local minimum will be memory state most closely resembling starting state.
Or its inverse (two fold degeneracy).
Memory capacity
Extensively measured as p/N (memory density).
Critical Memory threshold above which your 'brain explodes'
How much is too much?

8. Thermodynamic Limit
Percent of misaligned spins
D. Amit
H. Gutfreund
H Sompolinsky
Memory Density
Critical memory density at p/N ~ All energy minima are null correlated with desired memory states.

9. Memory Reliability Testing reliability of memory storage.
Initialize in pure state.
Relax lattice to ground state.
All or nothing. Does relaxed state match initial state perfectly?
Measures the percentage of times the lattice successfully retained the memory state.

10. Percent chance of recovering pure state
Memory Density

11. Memory Degradation
Testing how memory degrades as memory density increases.
CASE1:
Start in pure state.
Relax network and record percent of spins that differ from initial state.
CASE2:
Start in random state.
Relax network and determine closest pure state (prone to bias when null correlated).
Record percent of spins that differ from closest memory state.

12. FINITE SIZE EFFECTS? SAMPLING BIAS? Percent of misaligned spins
Memory Density

13. Percent of misaligned spins
Memory Density

14. Applications Facial recognition (secutiry cameras, digital cameras...)
Hand writing recognition (scanners, LateX help...)
Numerical/graphical operations