Presentation on theme: "Neural NetworksNN 11 Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22"— Presentation transcript:
Neural NetworksNN 11 Neural Networks Teacher: Elena Marchiori R4.47 firstname.lastname@example.org Assistant: Kees Jong S2.22 email@example.com
Neural NetworksNN 12 Course Outline Basics of neural network theory and practice for supervised and unsupervised learning. Most popular Neural Network models: architectures learning algorithms applications
Neural NetworksNN 13 Course Outline Rules:- 4 s.p - Final mark is based on two assignments, which will be available at the end of the course. - one assignment is on theory (to do alone). - one assignment is on practice (to do in couples). - Programming in Matlab 5.3. - Registration: send email to firstname.lastname@example.org
Neural NetworksNN 14 Course Organization There is no text book. Course schedule, slides and exercises will be available at http://www.cs.vu.nl/~elena/nn.html
Neural NetworksNN 15 Neural Networks A NN is a machine learning approach inspired by the way in which the brain performs a particular learning task: –Knowledge about the learning task is given in the form of examples. –Inter neuron connection strengths (weights) are used to store the acquired information (the training examples). –During the learning process the weights are modified in order to model the particular learning task correctly on the training examples.
Neural NetworksNN 16 Supervised Learning –Recognizing hand-written digits, pattern recognition, regression. –Labeled examples (input, desired output) –Neural Network models: perceptron, feed-forward, radial basis function, support vector machine. Unsupervised Learning –Find similar groups of documents in the web, content addressable memory, clustering. –Unlabeled examples (different realizations of the input alone) –Neural Network models: self organizing maps, Hopfield networks. Learning
Neural NetworksNN 17 Network architectures Three different classes of network architectures –single-layer feed-forward neurons are organized –multi-layer feed-forward in acyclic layers –recurrent The architecture of a neural network is linked with the learning algorithm used to train
Neural NetworksNN 18 Single Layer Feed-forward Input layer of source nodes Output layer of neurons
Neural NetworksNN 112 The Neuron The neuron is the basic information processing unit of a NN. It consists of: 1A set of synapses or connecting links, each link characterized by a weight: W 1, W 2, …, W m 2An adder function (linear combiner) which computes the weighted sum of the inputs: 3Activation function (squashing function) for limiting the amplitude of the output of the neuron.
Neural NetworksNN 113 The Neuron Input signal Synaptic weights Summing function Bias b Activation function Local Field v Output y x1x1 x2x2 xmxm w2w2 wmwm w1w1
Neural NetworksNN 114 Bias of a Neuron Bias b has the effect of applying an affine transformation to u v = u + b v is the induced field of the neuron v u
Neural NetworksNN 115 Bias as extra input Input signal Synaptic weights Summing function Activation function Local Field v Output y x1x1 x2x2 xmxm w2w2 wmwm w1w1 w0w0 x 0 = +1 Bias is an external parameter of the neuron. Can be modeled by adding an extra input.
Neural NetworksNN 116 Dimensions of a Neural Network Various types of neurons Various network architectures Various learning algorithms Various applications
Neural NetworksNN 117 Face Recognition 90% accurate learning head pose, and recognizing 1-of-20 faces