1. Lecture 24 Radial Basis Network (I)

2. Outline Interpolation Problem Formulation Radial Basis Network Type 1
(C) 2001 by Yu Hen Hu

3. What is Radial Basis Function?
RBF is a kernel function that is symmetric w. r. t. origin. Hence its variable is r that is the norm-distance from origin.
Examples of RBF
(C) 2001 by Yu Hen Hu

4. Interpolation Problem Formulation
Radial Basis function for interpolation:
Given {xi; 1  i  K} and {di; 1  i  K }, find a function F(x) that satisfies the interpolation condition:
F(xi) = di 1  i  K (1)
One possible choice of F(x) is a radial basis function of the following form:
(2)
where {xi; 1  i  K } are the centers of the radial basis functions.
(C) 2001 by Yu Hen Hu

5. Solving Radial Basis Coefficients
Substitute (1) into (2), we obtain a set of linear system of equations
M w = d (3)
where M = [M(i,j), 1  i, j,  K] is the interpolation matrix, M(i,j) = (||xi – xj||), w = [w1, w2, • • •, wK]t, and d = [d1, d2, • • •, dK]t.
Given M and d, assuming the N centers are distinct, w can be solved as: w = M1d if M is non-singular. If the (r) = (r2 + c2)–1/2, or (r) = exp(–r2/(2s2)), it can further be shown that M is also positive definite.
(C) 2001 by Yu Hen Hu

6. An Example Let F(–1) = 0.2, F(–0.5) = 0.5, and F(1) = –0.5.
Use a triangular radial basis function j(r) = (1–r)[u(r) –u(r –1)]
u(r) = 1 if r  0 and = 0 if r < 0.
(C) 2001 by Yu Hen Hu
rbfexample1.m

7. Example continued Use Gaussian rbfs:
Parzen window: No weighting, and no target values of F(x) needed. ,
(C) 2001 by Yu Hen Hu

8. Example (Comparison)
(C) 2001 by Yu Hen Hu

9. Regularization Problem Formulation
When there are too many data points, the M matrix may become singular.
This is because by impose a rbf to each data point, we have an over-determined system.
Regularization is the mathematical tool that addresses this problem.
By regularization, we add an additional term to the cost function that represents additional constraints on the solution:
Regularization term (e.g.):
(C) 2001 by Yu Hen Hu

10. Solution to Regularization Problem
The solution to this regularization problem is
G(x; xi) is the Green's function corresponding to the self-adjoin differential operator P*P such that
P*P G(x; xi) = d(x – xi)
A solution to the Green function that is of special interests to us is a multi-variate Gaussian function
Hence
With individual training data substituted into G(x, xi), a matrix equation
(G + lI) w = d
can be solved for w.
(C) 2001 by Yu Hen Hu

11. Implementation Consideration
However, other radial basis function other than the multi-variate Gaussian rbf can also be used.
The regularized F(x) may no longer match data points exactly, but it will be more smooth.
The value of l is usually determined empirically although generalized cross-validation (GCV) may be applied here.
(C) 2001 by Yu Hen Hu