Presentation is loading. Please wait.

Presentation is loading. Please wait.

NEURAL NETWORKS Biological analogy

Similar presentations

Presentation on theme: "NEURAL NETWORKS Biological analogy"— Presentation transcript:

1 NEURAL NETWORKS Biological analogy
Introduction to Artificial Neural Networks Typical architectures

2 Biological neuron Soma: body of the neuron.
Dendrites: receptors (inputs) of the neuron. Axon: output of neuron; connected to dendrites of other neurons via synapses. Synapses: transfer of information between neurons (electrical-chemical-electrical). João Sousa, José Borges XI.2

3 Neural networks Biological neural networks Artificial neural networks
Neuron switching time: second Number of neurons: 1010 Connections per neuron (synapses): 104,5 Recognition time: 0.1 s parallel computation Artificial neural networks Weighted connections amongst units Highly parallel, distributed process Emphasis on tuning weights automatically João Sousa, José Borges XI.3

4 Artificial Neural Networks
Artificial Neuron Threshold function Piece-wise Linear Sigmoidal function João Sousa, José Borges XI.4

5 Use of Artificial Neural Networks
Input is high-dimensional Output is multidimensional Mathematical form of system is unknown Interpretability of identified model is unimportant Applications Pattern recognition Classification Prediction Modeling Biological neural network Artificial neural network Soma Neuron Dendrite Input Axon Output Synapse Weight João Sousa, José Borges XI.5

6 Architectures of typical ANN
Feedforward ANN João Sousa, José Borges XI.6

7 Architectures of typical ANN
Recurrent ANN João Sousa, José Borges XI.7

8 ADAPTIVE NETWORKS Adaptive ANN Network Classification Backpropagation

9 Adaptive (neural) networks
Massively connected computational units inspired by the working of the human brain Provide a mathematical model for biological neural networks (brains) Characteristics: learning from examples adaptive and fault tolerant robust for fulfilling complex tasks João Sousa, José Borges XI.9

10 Network classification
Learning methods: supervised, unsupervised Architectures: feedforward, recurrent Output types: binary, continuous Node types: uniform, hybrid Implementations: software, hardware Connection weights: adjustable, hard-wired Inspirations: biological, psychological João Sousa, José Borges XI.10

11 Adaptive network Nodes can be static or parametric
Network can consist of heterogeneous nodes Links do not have parameters associated Node functions are differentiable except at a finite number of points fixed nodes 3 x1 6 8 x8 4 7 9 x9 x2 5 adaptive nodes Input layer Layer 1 Layer 2 Output layer João Sousa, José Borges XI.11

12 Calculating with a network
x g u y h v a x f y x a f y a y h v x g u João Sousa, José Borges XI.12

13 Backpropagation learning rule
Simple gradient descent applied to layered networks An overall error measure is minimized for P data points and L layers Derivative information propagated by the use of chain rule, change in parameter a change in outputs of nodes containing a change in network's outputs change in error measure João Sousa, José Borges XI.13

14 Ordered vs. partial derivatives
y x f g z partial derivative ordered derivative João Sousa, José Borges XI.14

15 BP for feedforward networks
Define an error signal at each node output node hidden layer node João Sousa, José Borges XI.15

16 Error propagation network
x1 x2 4 5 3 6 7 9 8 x8 x9 e1 e2 4 5 3 6 7 9 8 e8 e9 w83 w97 w52 w75 w31 João Sousa, José Borges XI.16

Download ppt "NEURAL NETWORKS Biological analogy"

Similar presentations

Ads by Google