Presentation is loading. Please wait.

Presentation is loading. Please wait.

April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 1 Capabilities of Threshold Neurons By choosing appropriate.

Published byImogene West Modified over 7 years ago

Similar presentations

Presentation on theme: "April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 1 Capabilities of Threshold Neurons By choosing appropriate."— Presentation transcript:

1 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 1 Capabilities of Threshold Neurons By choosing appropriate weights w i and threshold  we can place the line dividing the input space into regions of output 0 and output 1in any position and orientation. Therefore, our threshold neuron can realize any linearly separable function R n  {0, 1}. Although we only looked at two-dimensional input, our findings apply to any dimensionality n. For example, for n = 3, our neuron can realize any function that divides the three-dimensional input space along a two-dimension plane.

2 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 2 Capabilities of Threshold Neurons What do we do if we need a more complex function? Just like Threshold Logic Units, we can also combine multiple artificial neurons to form networks with increased capabilities. For example, we can build a two-layer network with any number of neurons in the first layer giving input to a single neuron in the second layer. The neuron in the second layer could, for example, implement an AND function.

3 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 3 Capabilities of Threshold Neurons What kind of function can such a network realize? x1x1x1x1 x2x2x2x2 x1x1x1x1 x2x2x2x2 x1x1x1x1 x2x2x2x2... xixixixi

4 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 4 Capabilities of Threshold Neurons Assume that the dotted lines in the diagram represent the input-dividing lines implemented by the neurons in the first layer: 1 st comp. 2 nd comp. Then, for example, the second-layer neuron could output 1 if the input is within a polygon, and 0 otherwise.

5 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 5 Capabilities of Threshold Neurons However, we still may want to implement functions that are more complex than that. An obvious idea is to extend our network even further. Let us build a network that has three layers, with arbitrary numbers of neurons in the first and second layers and one neuron in the third layer. The first and second layers are completely connected, that is, each neuron in the first layer sends its output to every neuron in the second layer.

6 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 6 Capabilities of Threshold Neurons What type of function can a three-layer network realize? x1x1x1x1 x2x2x2x2 x1x1x1x1 x2x2x2x2 x1x1x1x1 x2x2x2x2... oioioioi...

7 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 7 Capabilities of Threshold Neurons Assume that the polygons in the diagram indicate the input regions for which each of the second-layer neurons yields output 1: 1 st comp. 2 nd comp. Then, for example, the third-layer neuron could output 1 if the input is within any of the polygons, and 0 otherwise.

8 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 8 Capabilities of Threshold Neurons The more neurons there are in the first layer, the more vertices can the polygons have. With a sufficient number of first-layer neurons, the polygons can approximate any given shape. The more neurons there are in the second layer, the more of these polygons can be combined to form the output function of the network. With a sufficient number of neurons and appropriate weight vectors w i, a three-layer network of threshold neurons can realize any (!) function R n  {0, 1}.

9 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 9Terminology Usually, we draw neural networks in such a way that the input enters at the bottom and the output is generated at the top. Arrows indicate the direction of data flow. The first layer, termed input layer, just contains the input vector and does not perform any computations. The second layer, termed hidden layer, receives input from the input layer and sends its output to the output layer. After applying their activation function, the neurons in the output layer contain the output vector.

10 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 10Terminology Example: Network function f: R 3  {0, 1} 2 output layer hidden layer input layer input vector output vector

11 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 11 Linear Neurons Obviously, the fact that threshold units can only output the values 0 and 1 restricts their applicability to certain problems. We can overcome this limitation by eliminating the threshold and simply turning f i into the identity function so that we get: With this kind of neuron, we can build networks with m input neurons and n output neurons that compute a function f: R m  R n.

12 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 12 Linear Neurons Linear neurons are quite popular and useful for applications such as interpolation. However, they have a serious limitation: Each neuron computes a linear function, and therefore the overall network function f: R m  R n is also linear. This means that if an input vector x results in an output vector y, then for any factor  the input  x will result in the output  y. Obviously, many interesting functions cannot be realized by networks of linear neurons.

13 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 13 Gaussian Neurons Another type of neurons overcomes this problem by using a Gaussian activation function: 1 0 1 f i (net i (t)) net i (t)

14 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 14 Gaussian Neurons Gaussian neurons are able to realize non-linear functions. Therefore, networks of Gaussian units are in principle unrestricted with regard to the functions that they can realize. The drawback of Gaussian neurons is that we have to make sure that their net input does not exceed 1. This adds some difficulty to the learning in Gaussian networks.

15 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 15 Sigmoidal Neurons Sigmoidal neurons accept any vectors of real numbers as input, and they output a real number between 0 and 1. Sigmoidal neurons are the most common type of artificial neuron, especially in learning networks. A network of sigmoidal units with m input neurons and n output neurons realizes a network function f: R m  (0,1) n

16 April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 16 Sigmoidal Neurons The parameter  controls the slope of the sigmoid function, while the parameter  controls the horizontal offset of the function in a way similar to the threshold neurons. 1 0 1 f i (net i (t)) net i (t)  = 1  = 0.1

Download ppt "April 5, 2016Introduction to Artificial Intelligence Lecture 17: Neural Network Paradigms II 1 Capabilities of Threshold Neurons By choosing appropriate."

Similar presentations

NEURAL NETWORKS Perceptron

NEURAL NETWORKS Perceptron
Multilayer Perceptrons 1. Overview  Recap of neural network theory  The multi-layered perceptron  Back-propagation  Introduction to training  Uses.

Multilayer Perceptrons 1. Overview  Recap of neural network theory  The multi-layered perceptron  Back-propagation  Introduction to training  Uses.
Neural Networks  A neural network is a network of simulated neurons that can be used to recognize instances of patterns. NNs learn by searching through.

Neural Networks  A neural network is a network of simulated neurons that can be used to recognize instances of patterns. NNs learn by searching through.
Computer Vision Lecture 18: Object Recognition II

Computer Vision Lecture 18: Object Recognition II
CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.
Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)

Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)
Radial Basis Functions

Radial Basis Functions
November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.

November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.
Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Prof. Carla P. Gomes Module: Neural Networks: Concepts (Reading:

Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Prof. Carla P. Gomes Module: Neural Networks: Concepts (Reading:
Artificial Neural Networks

Artificial Neural Networks
September 30, 2010Neural Networks Lecture 8: Backpropagation Learning 1 Sigmoidal Neurons In backpropagation networks, we typically choose  = 1 and 

September 30, 2010Neural Networks Lecture 8: Backpropagation Learning 1 Sigmoidal Neurons In backpropagation networks, we typically choose  = 1 and 
Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 15: Introduction to Artificial Neural Networks Martin Russell.

Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 15: Introduction to Artificial Neural Networks Martin Russell.
November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.

November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.
September 14, 2010Neural Networks Lecture 3: Models of Neurons and Neural Networks 1 Visual Illusions demonstrate how we perceive an “interpreted version”

September 14, 2010Neural Networks Lecture 3: Models of Neurons and Neural Networks 1 Visual Illusions demonstrate how we perceive an “interpreted version”
September 16, 2010Neural Networks Lecture 4: Models of Neurons and Neural Networks 1 Capabilities of Threshold Neurons By choosing appropriate weights.

September 16, 2010Neural Networks Lecture 4: Models of Neurons and Neural Networks 1 Capabilities of Threshold Neurons By choosing appropriate weights.
November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.
November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.

November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.
September 23, 2010Neural Networks Lecture 6: Perceptron Learning 1 Refresher: Perceptron Training Algorithm Algorithm Perceptron; Start with a randomly.

September 23, 2010Neural Networks Lecture 6: Perceptron Learning 1 Refresher: Perceptron Training Algorithm Algorithm Perceptron; Start with a randomly.
October 7, 2010Neural Networks Lecture 10: Setting Backpropagation Parameters 1 Creating Data Representations On the other hand, sets of orthogonal vectors.

October 7, 2010Neural Networks Lecture 10: Setting Backpropagation Parameters 1 Creating Data Representations On the other hand, sets of orthogonal vectors.
November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised.

November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised.

Similar presentations

Ads by Google