Presentation is loading. Please wait.

Presentation is loading. Please wait.

CHAPTER 6 SPECIAL NETWORKS “Principles of Soft Computing, 2nd Edition”

Published byPhilip McKenzie Modified over 8 years ago

Similar presentations

Presentation on theme: "CHAPTER 6 SPECIAL NETWORKS “Principles of Soft Computing, 2nd Edition”"— Presentation transcript:

1 CHAPTER 6 SPECIAL NETWORKS “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

2 SPECIAL NETWORKS There exists several other special networks, apart from those discussed in Chapters 4 and 5. A few special networks are: Simulated Annealing, Boltzmann Machine, Probablistic Net, Gaussian Machine, Cauchy Machine, Cascade Correlation Net, Cognitron Net, NeoCognitron Net, Cellular Neural Network, Spatio-Temporal Network and so on. This chapter discusses a few of these networks. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

3 SIMULATED ANNEALING “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

4 “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

5 “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

6 “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

7 BOLTZMANN MACHINE The primary goal of Boltzmann learning is to produce a neural network that correctly models input patterns according to a Boltzmann distribution. The Boltzmann machine consists of stochastic neurons. A stochastic neuron resides in one of two possible states (±1) in a probabilistic manner. The use of symmetric synaptic connections between neurons. The stochastic neurons partition into two functional groups: visible and hidden. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

8 During the training phase of the network, the visible neurons are all clamped onto specific states determined by the environment. The hidden neurons always operate freely; they are used to explain underlying constraints contained in the environmental input vectors. This is accomplished by capturing higher-order statistical correlations in the clamping vectors. The network can perform pattern completition provided that it has learned the training distribution properly. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

9 Units fire probabilistically based on a sigmoid activation function.
Learning adjusts weights to give states of visible units a particular desired probability distribution. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

10 The log-likelihood function L(w) = log∏ x α T P(Xα= xα)
The goal of Boltzmann learning is to maximize the likelihood or log-likelihood function in accordance with the maximum-likelihood principle. Positve phase: In this phase the network operates in its clamped condition. Negative phase: In this phase, the network is allowed to run freely, and therefore with no environmental input. The log-likelihood function L(w) = log∏ x α T P(Xα= xα) L(w) = ∑xα T [log∑x exp(-E(x)/T) - log∑x exp(-E(x)/T)] “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

11 ∆wji = L(w)/wji = (+ji - -ji)
Differentiating L(w) with respect to wji and introducing +ji and -ji , we get ∆wji = L(w)/wji = (+ji - -ji) where  is a learning-rate parameter  = /T. From a learning point of view, the two terms that constitute the Boltzmann learning rule have opposite meaning: +ji corresponding to the clamped condition of the network is a Hebbian learning rule; -ji corresponding to the free-running condition of the network is unlearning (forgetting) term. We have also a primitive form of an attention mechanism. The two phase approach and ,specifically, the negative phase means also increased computational time and sensitivity to statistical errors. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

12 PROBABILISTIC NEURAL NETWORK
The probabilistic neural net is based on the idea of conventional probability theory, such as Bayesian classification and other estimators for probability density functions, to construct a neural net for classification. The architecture for the net is as given below: “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

13 ALGORITHM FOR PROBABILISTIC NEURAL NETWORK
“Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

14 COGNITRON NETWORK Cognitron network was proposed by Fukushima in 1975.
The synaptic strength from cell X to cell Y is reinforced if and only if the following two conditions are true: Cell X: presynaptic cell fires. None of the postsynaptic cells present near cell Y fire stronger than Y. The connection between presynaptic and postsynaptic cells is as follows: “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

15 NEOCOGNITRON NETWORK Neocognitron is a multilayer feedforward network model for visual pattern recognition. It is an extension of cognitron network. Neocognitron net can be used for recognizing handwritten characters. The algorithm used in cognitron and neocognitron is same, except that neocognitron model can recognize patterns that are position-shifted or shape-distorted. The cells used in neocognitron are of two types: S-cell: Cells that are trained suitably to respond to only certain features in the previous layer. C-cell: A C-cell displaces the result of an S-cell in space, i.e., sort of “spreads” the features recognized by the S-cell. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

16 The model of neocognitron network is as given below:
Training is found to progress layer by layer. The weights from the input units to the first layer are first trained and then frozen. Then the next trainable weights are adjusted and so on. When the net is designed, the weights between some layers are fixed as they are connection patterns. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

17 OPTICAL NEURAL NETWORKS
Optical neural networks interconnect neurons with light beams. There are two classes of optical neural networks. They are: Electro-optical multipliers, Holographic correlators. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

18 ELECTRO-OPTICAL MULTIPLIERS
Electro-optical multipliers, also called electro-optical matrix multipliers, perform matrix multiplication in parallel. The network speed is limited only by the available electro-optical components; here the computation time is potentially in the nanosecond range. A model of electro-optical matrix multiplier is shown on the right. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

19 HOLOGRAPHIC CORRELATORS
In holographic correlators, the reference images are stored in a thin hologram and are retrieved in a coherently illuminated feedback loop. The input signal, either noisy or incomplete, may be applied to the system and can simultaneously be correlated optically with all the stored reference images. These correlations can be threshold and are fed back to the input, where the strongest correlation reinforces the input image. The enhanced image passes around the loop repeatedly, which approaches the stored image more closely on each pass, until the system gets stabilized on the desired image. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

20 “Principles of Soft Computing, 2nd Edition”
by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

21 NEURO PROCESSOR CHIPS Neural networks implemented in hardware can take advantage of their inherent parallelism and run orders of magnitude faster than software simulations. There exists a wide variety of commercial neural network chips and neuro computers. A few are listed below: Probabilistic RAM, pRAM-256 neural net processor. Neuro Accelerator Chip (NAC). Neural Network Processor (NNP), developed by Accurate Automation Corporation. CNAPS digital parallel processor chip. IBM ZISC036. INTEL 80170NX Electrically Trainable Analog Neural Network and so on. “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

22 SUMMARY This chapter discussed a few special networks like:
Boltzmann Machine Simulated Annealing Probabilistic Net Optical Neural Networks Cognitron Net Neocognitron Net Neuro Processor Chips in Practical Use “Principles of Soft Computing, 2nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.

Download ppt "CHAPTER 6 SPECIAL NETWORKS “Principles of Soft Computing, 2nd Edition”"

Similar presentations

1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)

1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)
Artificial Neural Networks (1)

Artificial Neural Networks (1)
Stochastic Neural Networks Deep Learning and Neural Nets Spring 2015.

Stochastic Neural Networks Deep Learning and Neural Nets Spring 2015.
1 Neural networks. Neural networks are made up of many artificial neurons. Each input into the neuron has its own weight associated with it illustrated.

1 Neural networks. Neural networks are made up of many artificial neurons. Each input into the neuron has its own weight associated with it illustrated.
Multilayer Perceptrons 1. Overview  Recap of neural network theory  The multi-layered perceptron  Back-propagation  Introduction to training  Uses.

Multilayer Perceptrons 1. Overview  Recap of neural network theory  The multi-layered perceptron  Back-propagation  Introduction to training  Uses.
Neural Network I Week 7 1. Team Homework Assignment #9 Read pp. 327 – 334 and the Week 7 slide. Design a neural network for XOR (Exclusive OR) Explore.

Neural Network I Week 7 1. Team Homework Assignment #9 Read pp. 327 – 334 and the Week 7 slide. Design a neural network for XOR (Exclusive OR) Explore.
2806 Neural Computation Stochastic Machines Lecture 10

2806 Neural Computation Stochastic Machines Lecture 10
Tuomas Sandholm Carnegie Mellon University Computer Science Department

Tuomas Sandholm Carnegie Mellon University Computer Science Department
5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Machine Learning Neural Networks

Machine Learning Neural Networks
Simple Neural Nets For Pattern Classification

Simple Neural Nets For Pattern Classification
Neural Networks Basic concepts ArchitectureOperation.

Neural Networks Basic concepts ArchitectureOperation.
Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Prof. Carla P. Gomes Module: Neural Networks: Concepts (Reading:

Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Prof. Carla P. Gomes Module: Neural Networks: Concepts (Reading:
1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.
Rutgers CS440, Fall 2003 Neural networks Reading: Ch. 20, Sec. 5, AIMA 2 nd Ed.

Rutgers CS440, Fall 2003 Neural networks Reading: Ch. 20, Sec. 5, AIMA 2 nd Ed.
Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs)
Artificial Neurons, Neural Networks and Architectures

Artificial Neurons, Neural Networks and Architectures
Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.

Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.

Similar presentations

Ads by Google