Presentation is loading. Please wait.

Presentation is loading. Please wait.

NEURAL NETWORKS Biological analogy Introduction to Artificial Neural Networks Typical architectures.

Similar presentations

Presentation on theme: "NEURAL NETWORKS Biological analogy Introduction to Artificial Neural Networks Typical architectures."— Presentation transcript:

1 NEURAL NETWORKS Biological analogy Introduction to Artificial Neural Networks Typical architectures

2 João Sousa, José Borges XI.2 Biological neuron Soma: body of the neuron. Dendrites: receptors (inputs) of the neuron. Axon: output of neuron; connected to dendrites of other neurons via synapses. Synapses: transfer of information between neurons (electrical-chemical-electrical).

3 João Sousa, José Borges XI.3 Neural networks Biological neural networks Neuron switching time: 0.001 second Number of neurons: 10 10 Connections per neuron (synapses): 10 4,5 Recognition time: 0.1 s parallel computation Artificial neural networks Weighted connections amongst units Highly parallel, distributed process Emphasis on tuning weights automatically

4 João Sousa, José Borges XI.4 Artificial Neural Networks Artificial Neuron Threshold functionPiece-wise LinearSigmoidal function

5 João Sousa, José Borges XI.5 Use of Artificial Neural Networks Input is high-dimensional Output is multidimensional Mathematical form of system is unknown Interpretability of identified model is unimportant Biological neural network Artificial neural network SomaNeuron DendriteInput AxonOutput SynapseWeight Applications Pattern recognition Classification Prediction Modeling

6 João Sousa, José Borges XI.6 Architectures of typical ANN Feedforward ANN

7 João Sousa, José Borges XI.7 Architectures of typical ANN Recurrent ANN

8 ADAPTIVE NETWORKS Adaptive ANN Network Classification Backpropagation

9 João Sousa, José Borges XI.9 Adaptive (neural) networks Massively connected computational units inspired by the working of the human brain Provide a mathematical model for biological neural networks (brains) Characteristics: learning from examples adaptive and fault tolerant robust for fulfilling complex tasks

10 João Sousa, José Borges XI.10 Network classification Learning methods: supervised, unsupervised Architectures: feedforward, recurrent Output types: binary, continuous Node types: uniform, hybrid Implementations: software, hardware Connection weights: adjustable, hard-wired Inspirations: biological, psychological

11 João Sousa, José Borges XI.11 Adaptive network Nodes can be static or parametric Network can consist of heterogeneous nodes Links do not have parameters associated Node functions are differentiable except at a finite number of points adaptive nodes x1x1 x2x2 4 5 3 6 7 9 8 Input layerLayer 1Layer 2Output layer x8x8 x9x9 fixed nodes

12 João Sousa, José Borges XI.12 Calculating with a network x f y x a f y x g u y h v a a y h v x g u

13 João Sousa, José Borges XI.13 Backpropagation learning rule Simple gradient descent applied to layered networks An overall error measure is minimized for P data points and L layers change in parameter  change in outputs of nodes containing  change in network's outputs change in error measure Derivative information propagated by the use of chain rule,

14 João Sousa, José Borges XI.14 Ordered vs. partial derivatives y x f g z partial derivative ordered derivative

15 João Sousa, José Borges XI.15 BP for feedforward networks Define an error signal at each node output node hidden layer node

16 João Sousa, José Borges XI.16 Error propagation network x1x1 x2x2 4 5 3 6 7 9 8 x8x8 x9x9 11 22 4 5 3 6 7 9 8 88 99 w 83 w 97 w 52 w 75 w 31

Download ppt "NEURAL NETWORKS Biological analogy Introduction to Artificial Neural Networks Typical architectures."

Similar presentations

Ads by Google