Presentation on theme: "Introduction to Neural Networks"— Presentation transcript:
1 Introduction to Neural Networks Neural Nets slides mostly from: Andy Philippides,University of Sussex
2 Uses of NNs Neural Networks Are For Applications ScienceCharacter recognition NeuroscienceOptimization Physics, mathematics statisticsFinancial prediction Computer scienceAutomatic driving Psychology
3 What are biological NNs? UNITs: nerve cells called neurons, many different types and are extremely complexaround neurons in the brain (depending on counting technique) each with 103 connectionsINTERACTIONs: signal is conveyed by action potentials, interactions could be chemical (release or receive neurotransmitters) or electrical at the synapseSTRUCTUREs: feedforward and feedback and self-activation recurrent
5 “The nerve fibre is clearly a signalling mechanism of limited scope. It can only transmit a succession of brief explosive waves, and themessage can only be varied by changes in the frequency and in thetotal number of these waves. … But this limitation is really a small matter, for in the body the nervous units do not act in isolation asthey do in our experiments. A sensory stimulus will usually affect anumber of receptor organs, and its result will depend on thecomposite message in many nerve fibres.” Lord Adrian, Nobel Acceptance Speech, 1932.
6 We now know it’s not quite that simple Single neurons are highly complex electrochemical devicesSynaptically connected networks are only part of the storyMany forms of interneuron communication now known – acting over many different spatial and temporal scales
7 The complexity of a neuronal system can be partly seen from a picture in a book on computational neuroscienceedited by Jianfeng
8 How do we go from real neurons to artificial ones? Hillockinputoutput
9 Single neuron activity Membrane potential is the voltage difference between a neuron and its surroundings (0 mV)Membrane potentialCell0 Mv
10 Single neuron activity If you measure the membrane potential of a neuron and print it outon the screen, it looks like:spike
11 Single neuron activity A spike is generated when the membrane potential is greater thanits threshold
12 AbstractionSo we can forget all sub-threshold activity and concentrate on spikes (action potentials), which are the signals sent to other neuronsSpikes
13 Only spikes are important since other neurons receive them (signals)Neurons communicate with spikesInformation is coded by spikesSo if we can manage to measure the spiking time, we decipher how the brain works ….
14 Again its not quite that simple spiking time in the cortex is random
15 With identical inputfor the identical neuronspike patterns are similar, but not identical
16 Recording from a real neuron: membrane potential
17 = Single spiking time is meaningless To extract useful information, we have to averageto obtain the firing rate rfor a group of neurons in a local circuit where neuroncodes the same informationover a time windowLocal circuit=Time window = 1 secr == 6 Hz
18 Hence we have firing rate of a group of neurons So we can have a network of these local groupsr1w1: synaptic strengthwnrn
19 ri is the firing rate of input local circuit The neurons at output local circuits receives signals in the formThe output firing rate of the output local circuit is then given byRwhere f is the activation function, generally a Sigmoidal function of some sortwi weight, (synaptic strength) measuring the strength of the interaction between neurons.
20 Artificial Neural networks Local circuits (average to get firing rates)Single neuron (send out spikes)
21 Artificial Neural Networks (ANNs) A network with interactions, an attempt to mimic the brainUNITs: artificial neuron (linear or nonlinear input-output unit), small numbers, typically less than a few hundredINTERACTIONs: encoded by weights, how strong a neuron affects othersSTRUCTUREs: can be feedforward, feedback or recurrentIt is still far too naïve as a brain model and an information processing
23 The general artificial neuron model has five components, shown in the following list. (The subscript i indicates the i-th input or weight.)A set of inputs, xi.A set of weights, wi.A bias, u.An activation function, f.Neuron output, y
24 Thus the key to understanding ANNs is to understand/generate the local input-output relationship