Download presentation

Presentation is loading. Please wait.

Published byGerard Sanders Modified over 4 years ago

1
Ch. 4: Radial Basis Functions Stephen Marsland, Machine Learning: An Algorithmic Perspective. CRC 2009 based on slides from many Internet sources Longin Jan Latecki Temple University latecki@temple.edu

2
Perceptorn

3
In RBFN

4
architecture Input layerHidden layerOutput layer x1x1 x2x2 x3x3 xnxn h1h1 h2h2 h3h3 hmhm f(x) W1W1 W2W2 W3W3 WmWm

5
Three layers Input layer – Source nodes that connect to the network to its environment Hidden layer – Hidden units provide a set of basis function – High dimensionality Output layer – Linear combination of hidden functions architecture

6
Radial basis function h j (x) = exp( -(x-c j ) 2 / r j 2 ) f(x) = w j h j (x) j=1 m Wherec j is center of a region, r j is width of the receptive field architecture

7
7 Function Approximation with Radial Basis Functions RBF Networks approximate functions using (radial) basis functions as the building blocks.

8
8 Exact Interpolation RBFs have their origins in techniques for performing exact function interpolation [Bishop, 1995]: – Find a function h(x) such that h(x n ) = t n n=1,... N Radial Basis Function approach (Powel 1987): – Use a set of N basis functions of the form (||x-x n ||), one for each point,where (.) is some non-linear function. – Output: h(x) = n w n (||x-x n ||)

9
9 Exact Interpolation Goal (exact interpolation): – Find a function h(x) such that h(x n ) = t n n=1,... N Radial Basis Function approach (Powel 1987): – Use a set of N basis functions of the form (||x-x n ||), one for each point,where (.) is some non-linear function. – Output: h(x) = n w n (||x-x n ||) w 1 (||x 1 -x 1 ||) + w 2 (||x 1 -x 2 ||) +... + w N (||x 1 -x N ||) = t 1 w 1 (||x 2 -x 1 ||) + w 2 (||x 2 -x 2 ||) +... + w N (||x 2 -x N ||) = t 2 W = T... w 1 (||x N -x 1 ||) + w 2 (||x N -x 2 ||) +... + w N (||x N -x N ||)= t N

10
10 Exact Interpolation

11
11 Exact Interpolation

12
12 Due to noise that may be present in the data exact interpolation is rarely useful. By introducing a number of modifications, we arrive at RBF networks: Complexity rather than the size of the data is what is important – Number of the basis functions need not be equal to N Centers need not be constrained by the input Each basis function can have its own adjustable width parameter Bias parameter may be included in the linear sum.

13
13 Illustrative Example - XOR Problem 22 11

14
14 Function Approximation via Basis Functions and RBF Networks Using nonlinear functions, we can convert a nonlinearly separable problem into a linearly separable one. From a function approximation perspective, this is equivalent to implementing a complex function (corresponding to the nonlinearly separable decision boundary) using simple functions (corresponding to the linearly separable decision boundary) Implementing this procedure using a network architecture, yields the RBF networks, if the nonlinear mapping functions are radial basis functions. Radial Basis Functions: – Radial: Symmetric around its center – Basis Functions: Also called kernels, a set of functions whose linear combination can generate an arbitrary function in a given function space.

15
15 RBF Networks

16
16 RBF Networks

17
17 Network Parameters What do these parameters represent? : The radial basis function for the hidden layer. This is a simple nonlinear mapping function (typically Gaussian) that transforms the d- dimensional input patterns to a (typically higher) H- dimensional space. The complex decision boundary will be constructed from linear combinations (weighted sums) of these simple building blocks. u Ji : The weights joining the first to hidden layer. These weights constitute the center points of the radial basis functions. Also called prototypes of data. : The spread constant(s). These values determine the spread (extend) of each radial basis function. W jk : The weights joining hidden and output layers. These are the weights which are used in obtaining the linear combination of the radial basis functions. They determine the relative amplitudes of the RBFs when they are combined to form the complex function. ||x-u J ||: the Euclidean distance between the input x and the prototype vector u J. Activation of the hidden unit is determined according to this distance through

18
18

19
19 Training RBF Networks Approach 1: Exact RBF Approach 2: Fixed centers selected at random Approach 3: Centers are obtained from clustering Approach 4: Fully supervised training

20
20 Training RBF Networks Approach 1: Exact RBF Guarantees correct classification of all training data instances. Requires N hidden layer nodes, one for each training instance. No iterative training is involved: w are obtained by solving a set of linear equations Non-smooth, bad generalization

21
21 Exact Interpolation

22
22 Exact Interpolation

23
23 Too Many Receptive Fields? In order to reduce the artificial complexity of the RBF, we need to use fewer number of receptive fields. Approach 2: Fixed centers selected at random. Use M < N data points as the receptive field centers. Fast but may require excessive centers Approach 3: Centers are obtained from unsupervised learning (clustering). Centers no longer has to coincide with data points This is the most commonly used procedure, providing good results.

24
24 Approach 2 Approach 3 Approach 3.b

25
25 Determining the Output Weights through learning (LMS)

26
26 RBFs for Classification

27
Homework Problem 4.1, p. 117 Problem 4.2, p. 117

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google