6Traditional Computers are inefficient at these tasks although their computation speed is faster. ApplicationsPattern MatchingPattern RecognitionAssociate Memory (Content Addressable Memory)Function ApproximationLearningOptimizationVector QuantizationData Clustering. . .
7The Configuration of ANNs An ANN consists of a large number of interconnected processing elements called neurons.A human brain consists of ~1011 neurons of many different types.How ANN works?Collective behavior.
13 The Artificial Neuron wij positive excitatory negative inhibitoryzero no connectionThe Artificial Neuronx1x2xmwi1wi2wimf (.)a (.)iyi
14 The Artificial Neuron Proposed by McCulloch and Pitts  M-P neuronsx1x2xmwi1wi2wimf (.)a (.)iyi
15What can be done by M-P neurons? A hard limiter.A binary threshold unit.Hyperspace separation.x1x2yw1w2x1x2w1 x1 + w2 x2 =1
16What ANNs will be? ANN A neurally inspired mathematical model. Consists a large number of highly interconnected PEs.Its connections (weights) holds knowledge.The response of PE depends only on local information.Its collective behavior demonstrates the computation power.With learning, recalling and, generalization capability.
17Three Basic Entities of ANN Models Models of Neurons or PEs.Models of synaptic interconnections and structures.Training or learning rules.
18Introduction to Artificial Neural Networks Basic Models and Learning RulesNeuron ModelsANN structuresLearning
19 Processing Elements f (.) a (.) Extensions of M-P neurons What integration functions we may have?What activation functions we may have?
20Integration Functions M-P neuronf (.)a (.)iQuadratic FunctionSpherical FunctionPolynomial Function
21 Activation Functions f (.) a (.) M-P neuron: (Step function) a f i 1af
22 Activation Functions f (.) a (.) Hard Limiter (Threshold function) a 1a1f
51The General Weight Learning Rule .wi1wi2wijwi,m-1x1x2xjxm-1yiiInput:Output:
52The General Weight Learning Rule We want to learn the weights & bias.The General Weight Learning Rulei.wi1wi2wijwi,m-1x1x2xjxm-1yiiInput:Output:
53The General Weight Learning Rule We want to learn the weights & bias.The General Weight Learning Rulex1wi1Input:x2wi2.wijixjLet xm = 1 and wim = i..xm-1wi,m-1i
54The General Weight Learning Rule x1wi1Input:x2wi2.wijixjLet xm = 1 and wim = i..xm-1wi,m-1wim=ixm= 1
55The General Weight Learning Rule We wantto learnwi=(wi1, wi2 ,…,wim)TThe General Weight Learning Rulex1wi1Input:x2wi2.wijixjyi.xm-1wi,m-1wi(t) = ?wim=ixm= 1
56The General Weight Learning Rule wixyirdiLearningSignalGenerator
57The General Weight Learning Rule wixyirdiLearningSignalGenerator
58The General Weight Learning Rule wixyirdiLearningSignalGenerator
59The General Weight Learning Rule wixyirdiLearningSignalGeneratorLearning Rate
60The General Weight Learning Rule We wantto learnwi=(wi1, wi2 ,…,wim)TThe General Weight Learning RuleDiscrete-Time Weight Modification Rule:Continuous-Time Weight Modification Rule:
61Hebb’s Learning LawHebb  hypothesis that when an axonal input from A to B causes neuron B to immediately emit a pulse (fire) and this situation happens repeatedly or persistently.Then, the efficacy of that axonal input, in terms of ability to help neuron B to fire in future, is somehow increased.Hebb’s learning rule is a unsupervised learning rule.
63Introduction to Artificial Neural Networks Distributed Representations
64Distributed Representations An entity is represented by a pattern of activity distributed over many PEs.Each Processing element is involved in representing many different entities.Local Representation:Each entity is represented by one PE.
66Advantages What is this? + _ Dog Cat Bread + _ + _ + P0 P1 P2 P3 P4 P5 Act as a content addressable memory.AdvantagesP0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+_DogCatBread+_+_P0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+What is this?
67Advantages Dog has 4 legs? How many for Fido? + _ Dog Cat Bread + _ + Act as a content addressable memory.Make induction easy.AdvantagesP0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+_DogCatBread+_+_P0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+_FidoDog has 4 legs? How many for Fido?
68Advantages Add doughnut by changing weights. + _ Dog Cat Bread + _ + _ Act as a content addressable memory.Make induction easy.Make the creation of new entities or concept easy (without allocation of new hardware).AdvantagesP0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+_DogCatBread+_+_+_DoughnutAdd doughnut by changing weights.
69Advantages Some PEs break down don’t cause problem. + _ Dog Cat Bread Act as a content addressable memory.Make induction easy.Make the creation of new entities or concept easy (without allocation of new hardware).Fault Tolerance.AdvantagesP0P1P2P3P4P5P6P7P8P9P10P11P12P13P14P15+_DogCatBread+_+_Some PEs break down don’t cause problem.
70Disadvantages Learning procedures are required. How to understand? How to modify?Learning procedures are required.