1. Chapter 8 Geocomputation Part B:
Artificial Neural Networks (ANNs) & Genetic Algorithms (GAs)

2. Geocomputation: ANNs In this presentation on geocomputation:
ANNs discussed include
Multi-level perceptrons (MLPs)
Radial basis function neural networks (RBFNNs)
Self organising feature maps (SOFMs)
ANNs are particularly concerned with
Function approximation and interpolation
Image analysis and classification
Spatial interaction modelling
3rd edition

3. Geocomputation: Evolutionary computing
In this presentation on geocomputation:
EC elements discussed include
Genetic algorithms (GAs)
Genetic programming (GP)
EC is particularly concerned with
Complex problem solving using GAs
Model design using GP methods
3rd edition

4. Geocomputation Artificial Neural Networks (ANNs)
A computational model based on emulating biological neural networks
A form of non-linear modelling tool
Often a 3-layer network structure is used:
input, hidden, output
The output layer of such structures are typically modified weighted sums of intermediate layers, which are modified weighted sums of the input layer
3rd edition

5. Artificial Neural Networks
Hence at each output node (hidden or final) a two-step process takes place:
3rd edition

6. Artificial Neural Networks
Simple 3-layer feedforward ANN
Fully inter-connected; each connection is given a weight, w
Known as a Multi-level perceptron (MLP)
In this case: 3 input nodes, 5 hidden nodes, 2 output nodes and 2 bias nodes (bias, B, is similar to the constant term in regression models)
At hidden node 1 we have:
where the wij are weights to be determined, b1=1, and the xi are the observed input values
3rd edition

7. Artificial Neural Networks
Sample activation functions
is simply a linear weighted sum of the inputs. To generate a non-linear output it must be modified by some (well behaved) non-linear function, g(), e.g. the logistic function:
i.e.
3rd edition

8. Artificial Neural Networks
We can now compute the output layer values as the weighted sum
Suppose we have known input values x1=1, x2=-3, x3=5, and known outputs of 0 and 1. Can we select the weights to ensure the inputs generate the known outputs?
Suggestion: <build your own worked example & program here>
3rd edition

9. Artificial Neural Networks
Learning
Supervised learning
Split training/test data sets (control data)
Known inputs and output (target) values for training data
(Network output-Target output) = Error signal, e
Systematically adjust weights to minimise sum of e2
Adjustment typically based on backpropagation and gradient descent
Used in many classification/pattern recognition applications and in function approximation
Unsupervised learning
No training data
Must create clusters by analysing dataset for patterns/clusters
3rd edition

10. Artificial Neural Networks
Some basic issues:
local vs global minimisation
Initialisation and selection
Data normalisation and coding
Momentum
Model design and over-fitting
Overtraining
Interpolation vs Extrapolation/Forecasting
3rd edition

11. Artificial Neural Networks
MLP: Example 1 function approximation
source data
fitted solution curve
RMSE vs epochs
3rd edition

12. Artificial Neural Networks
MLP Example 2: LCM
3rd edition

13. Artificial Neural Networks
MLP Example 2: LCM
3rd edition

14. Artificial Neural Networks
MLP Example 2: LCM
3rd edition

15. Artificial Neural Networks
MLP Example 2: LCM
weights matrix
3rd edition

16. Artificial Neural Networks
MLP Example 3: Spatial interaction model
Generalised model: Tij=f(Oi,Dj,dij)
Sample data format (log transformed):
3rd edition

17. Artificial Neural Networks
MLP Example 3: Spatial interaction model
3rd edition

18. Artificial Neural Networks
Radial Basis Function Networks
Basic functional form:
Gaussian RBF:
3rd edition

19. Artificial Neural Networks
Self organising function maps
SOM as an output space
Neighbourhood relations
Grid size, form and topology
3rd edition

20. Artificial Neural Networks
Self organising function maps
Dimensional reductions
Mapped output – similar vectors (units) are close to each other
Typically an unsupervised procedure
Spatial mapping of SOM can follow using simple assignment to best matching unit (BMU)
3rd edition

21. Artificial Neural Networks
Self organising function maps
Choose a grid size, form and topology
Train the network
Identify the best matching units
Modify the BMU and its neighbours (spatially biased learning rule)
Map the trained network
3rd edition

22. Artificial Neural Networks
Self organising function maps – some issues
Initialisation
Pre-processing
Normalisation
Missing data
Masking and weighting
Learning and tuning
Distance metrics
Neighbourhood functions (kernels)
Learning rate functions
3rd edition

23. Artificial Neural Networks
Self organising function maps – Idrisi
3rd edition

24. Artificial Neural Networks
Self organising function maps – Idrisi
3rd edition

25. Genetic Algorithms Solutions are represented as individuals
Individuals are modelled as chromosomes
Chromosomes are comprised of genes
Genes have values known as alleles
Chromosomes have a measurable fitness
New chromosomes (children) are created by reproduction and mutation processes
The fittest individuals survive
The creation process is then iterated
3rd edition

26. Genetic Algorithms GAs: Example 1 - TSP
allele=12 (ID of town in TSP problem set)
chromosome
genes
Each chromosome contains complete list of towns
create a set of m randomly permuted strings and compute lengths, d
evaluate the fitness of each string (e.g. 1/d)
select random pairs of tours (biased by fitness)
combine pairs by crossover operation
evaluate fitness of offspring
apply replacement rule (fittest retained) and iterate till stable
3rd edition

27. Genetic Algorithms GA components
Encoding or representation – binary, list, tree etc
Fitness function selection – use of rank transforms
Population initialisation
Selection: roulette, tournament, uniform random
Reproduction
Crossover e.g. A = [a b c d e f g h] B = [ ]
and the crossover point is 3, the following children are generated:
child 1 = [a b c ] child 2= [1 2 3 d e f g h]
Mutation
Local search
Termination
3rd edition

28. Genetic Algorithms GAs: application areas TSP (as above) Clustering
Map labelling
Optimum location with capacity constraints
Concept can be extended to alleles that are expressions or program elements rather than numerical values  Genetic programming
3rd edition