1. Introduction to Neural Networks
Neural Nets slides mostly from: Andy Philippides,University of Sussex

2. Uses of NNs Neural Networks Are For
Applications Science
Character recognition Neuroscience
Optimization Physics, mathematics statistics
Financial prediction Computer science
Automatic driving Psychology

3. What are biological NNs?
UNITs: nerve cells called neurons, many different types and are extremely complex
around neurons in the brain (depending on counting technique) each with 103 connections
INTERACTIONs: signal is conveyed by action potentials, interactions could be chemical (release or receive neurotransmitters) or electrical at the synapse
STRUCTUREs: feedforward and feedback and self-activation recurrent

5. “The nerve fibre is clearly a signalling mechanism of limited scope.
It can only transmit a succession of brief explosive waves, and the
message can only be varied by changes in the frequency and in the
total number of these waves. … But this limitation is really a small matter, for in the body the nervous units do not act in isolation as
they do in our experiments. A sensory stimulus will usually affect a
number of receptor organs, and its result will depend on the
composite message in many nerve fibres.” Lord Adrian, Nobel Acceptance Speech, 1932.

6. We now know it’s not quite that simple
Single neurons are highly complex electrochemical devices
Synaptically connected networks are only part of the story
Many forms of interneuron communication now known – acting over many different spatial and temporal scales

7. The complexity of a neuronal system can be
partly seen from a picture in a book on computational neuroscience
edited by Jianfeng

8. How do we go from real neurons to artificial ones?
Hillock
input
output

9. Single neuron activity
Membrane potential is the voltage difference between a neuron and its surroundings (0 mV)
Membrane potential
Cell
0 Mv

10. Single neuron activity
If you measure the membrane potential of a neuron and print it out
on the screen, it looks like:
spike

11. Single neuron activity
A spike is generated when the membrane potential is greater than
its threshold

12. Abstraction
So we can forget all sub-threshold activity and concentrate on spikes (action potentials), which are the signals sent to other neurons
Spikes

13. Only spikes are important since other neurons receive them
(signals)
Neurons communicate with spikes
Information is coded by spikes
So if we can manage to measure the spiking time, we decipher how the brain works ….

14. Again its not quite that simple
spiking time in the cortex is random

15. With identical input
for the identical neuron
spike patterns are similar, but not identical

16. Recording from a real neuron: membrane potential

17. = Single spiking time is meaningless
To extract useful information, we have to average
to obtain the firing rate r
for a group of neurons in a local circuit where neuron
codes the same information
over a time window
Local circuit =
Time window = 1 sec
r =
= 6 Hz

18. Hence we have firing rate of a group of neurons
So we can have a network of these local groups r1
w1: synaptic strength wn rn

19. ri is the firing rate of input local circuit
The neurons at output local circuits receives signals in the form
The output firing rate of the output local circuit is then given by R
where f is the activation function, generally a Sigmoidal function of some sort
wi weight, (synaptic strength) measuring the strength of the interaction between neurons.

20. Artificial Neural networks
Local circuits (average to get firing rates)
Single neuron (send out spikes)

21. Artificial Neural Networks (ANNs)
A network with interactions, an attempt to mimic the brain
UNITs: artificial neuron (linear or nonlinear input-output unit), small numbers, typically less than a few hundred
INTERACTIONs: encoded by weights, how strong a neuron affects others
STRUCTUREs: can be feedforward, feedback or recurrent
It is still far too naïve as a brain model and an information processing

22. Four-layer networks x1 x2
Input
(visual
input)
Output
(Motor
output) xn
Hidden layers

23. The general artificial neuron model has five components, shown in the following list. (The subscript i indicates the i-th input or weight.)
A set of inputs, xi.
A set of weights, wi.
A bias, u.
An activation function, f.
Neuron output, y

24. Thus the key to understanding ANNs is to understand/generate the local input-output relationship