Neural Networks 2nd Edition Simon Haykin 柯博昌 Chap 2. Learning Processes

Learning vs. Neural Network The neural network is stimulated by an environment. The neural network undergoes changes in its free parameters as a result of this stimulation. The neural network responds in a new way to the environment because of the changes that have occurred in its internal structure.

Error-Correction Learning Desired Response (Target Output) Input vector One or more Layers of hidden neurons Output Neuron k x(n) yk(n)  dk(n) - + ek(n)=dk(n)-yk(n) Multilayer feedforward network Error Signal Objective: Minimizing a cost function or index of performance, The step-by-step adjustment are continued until the system reaches a steady state.

Delta Rule (Widrow-Hoff Rule) Let wkj(n) denote the value of synaptic weight wkj of neuron k excited by element xj(n) of the signal vector x(n) at time step n. : a positive constant determining the rate of learning as we proceed from one step in the learning process to another. (learning-rate parameter) In effect, wkj(n) and wkj(n+1) may be viewed as the old and new values of synaptic weight wkj, respectively.

Memory-based Learning Def: All (or most) of the past experiences are explicitly stored in a large memory of correctly classified input-output examples. Without loss of generality, the desired response is restricted to be a scalar. Ex: A binary pattern classification problem Assume there are two classes/hypotheses denoted C1 and C2 di= 0 (or -1) 1 for C1 for C2 Retrieving the training data in a “local neighborhood” of xtest Classification of a test vector xtest Nearest neighbor rule is a simple yet effective type of learning K-nearest neighbor classifier is to identify k patterns lying nearest to xtest and use a majority vote to make classification.

Hebbian Learning Hebb’s postulate of learning is the oldest and most famous of all learning rule. When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic changes take place in one or both cells (A, B). Hebb’s rule was expanded: If two neurons on either side of a synapse are activated simultaneously, then the strength of that synapse is selectively increased. Otherwise, if such two neurons are activated asynchronously, then that synapse is selectively weakened or eliminated. Such a synapse is called a Hebbian synapse. Hebbian Synapse Characteristics Time-dependent mechanism Local mechanism Interactive mechanism Conjunctional or correlational mechanism

Mathematical Models of Hebbian Modifications Let wkj denote a synaptic weight of neuron k with pre-synaptic and post-synaptic signals denoted by xj and yk, respectively. Hebby’s hypothesis (the Simplest form): (Activity product rule) slope=xj Hebb’s hypothesis Covariance hypothesis Postsynaptic activity yk Balance point yk Maximum depression point wkj Covariance hypothesis: : the rate of learning (a positive constant ) Limitation of Hebby’s hypothesis: The repeated application of xj leads to an increase in yk and exponential growth that finally drives the synaptic connection into saturation. No information will be stored in the synapse and selectivity is lost.

Competitive Learning Characteristics: Architectural Graph The output neurons compete among themselves to become active. Only a single output neuron is active at any one time. The neuron that wins the competition is called a winner-takes-all neuron. Architectural Graph If k is the winning neuron, it induced that vk must be the largest among all the neurons in the network for a specified input pattern x.

Boltzmann Learning A stochastic learning algorithm derived from ideas rooted in statistical mechanism. A neural network designed based on Boltzmann learning rule is called a Bolzmann machine. The neurons constitute a recurrent structure, and operate in a binary manner (ex: +1, -1). Energy Function: xj is the state of neuron j. jk means that none of the neurons has self-feedback. The machine operates by choosing a neuron at random. (A brief review of statistical mechanics is presented in Chapter 11)

Credit-Assignment Problem The problem of assigning credit or blame for overall outcomes to each of the internal decisions. For example: the error-correction learning is applied to a multilayer feedforward neural network. (Presented in Chapter 4)

Learning with a Teacher (Supervised Learning)

Learning without a Teacher Reinforcement Learning Unsupervised Learning I/O mapping is performed through continued interaction with the environment to minimize a scaler index of performance. No teacher or critic to oversee the learning process. Environment Learning System Ex: Competitive learning

Learning Tasks Pattern Association Autoassociation: A neural network stores a set of patterns by repeatedly presenting them to network. Then, the network is presented a partial description or distorted version of an original pattern. Heteroassociation: An arbitrary set of input patterns is paired with another arbitrary set of output patterns. xk: key pattern yk: memorized pattern Pattern Association: xkyk, k=1, 2, …, q where q is the number of patterns stored in the network In Autoassociation, yk=xk In Heteroassociation, yk=xk

Learning Tasks Pattern Recognition Def: A received pattern/signal is assigned to one of a prescribed number of classes. Feature vector y Unsupervised network for feature extraction 1 Input pattern x Supervised network for classification (A) 2 … r (B)

Learning Tasks Function Approximation Nonlinear input-output mapping x: input vector d: output vector f() is assumed to be unknown d=f(x) Given a set of labeled examples: Requirement: Design a neural network to approximate this unknown function f() such that F(). ||F(x)-f(x)||< for all x, where  is a small positive number System Identification Inverse System

Learning Tasks Control Goal: Supply appropriate inputs to the plant to make its output y track reference d. Error-correction algorithm needs the Jacobian matrix:

Learning Tasks Filtering and Beamforming Extract information from a set of noisy data. Ex: Cocktail party problem. Beamforming A spatial form of filtering, which is used to distinguish between the spatial properties of a target signal and background noise. Ex: Echo-locating bats

Memory Signal-flow graph model of a linear neuron labeled i W(k) W(k) is a weight matrix determined by input-output pair (xk,yk). Memory matrix M defines the overall connectivity between input and output layers.

Correlation Matrix Memory (Recursion Form)

Recall xk: a randomly selected key pattern y: the yielded response Let each of key patterns x1, x2, …, xq be normalized to have unit energy Desired Response Error Vector Because If These key vectors are orthogonal. It means vj=0 But, the key patterns presented to an associative memory are neither orthogonal nor highly separated from each other.

Adaptation If the operating environment is stationary (the statistical characteristics do not change with the time), the essential statistics of the environment can, in theory, be learned under the supervision of a teacher. In general, the environment of interest is non-stationary. So, the neural network must adapt its free parameters to variations continuously in a real-time fashion. (Continuous Learning or Learning-on-the-fly) Pseudo-stationary: The statistical characteristics of a non-stationary process change slowly enough over a window of short enough duration. Dynamic Approach to Learning Select a window short enough. When a new data sample is received, update the window by dropping the oldest data and shifting the remaining data by one time unit. Use the updated data window to retrain the network. Repeat the procedure on a continuing basis.