1. Unsupervised Learning
Business School Institute of Business Informatics
Unsupervised Learning
Uwe Lämmel

2. Neural Networks Idea Artificial Neuron & Network Supervised Learning
Unsupervised Learning
Data Mining – other Techniques

3. Unsupervised Learning
Self-Organizing Map (SOM)
Learning
Clustering – Example
Visualisation
Application: TSP

4. Self Organizing Maps (SOM)
A natural brain can organize itself
Now we look at the position of a neuron and its neighbourhood
Kohonen Feature Map
 two layer pattern associator
Input layer is fully connected with map-layer
Neurons of the map layer are fully connected to each other (virtually)

5. Clustering f ai output B Input set A
objective: All inputs of a class are mapped onto one and the same neuron f
Input set A
output B ai
Problem: classification in the input space is unknown
Network performs a clustering

6. Winner Neuron
Kohonen- Layer
Input-Layer
Winner Neuron

7. Learning in an SOM Choose an input k randomly
Detect the neuron z which has the maximal activity -> winner neuron
Adapt the weights in the neighbourhood of z: neuron i within a radius r of z.
Stop if a certain number of learning steps is finished otherwise decrease learning rate and radius, go on with step 1.

8. Centre of Activation
Idea: highly activated neurons push down the activation of neurons in the neighbourhood
Problem: Finding the centre of activation:
Neuron j with a maximal net-input
Neuron j, having a weight vector wj which is similar to the input vector (Euklidian Distance):
z:  x - wz  = minj  x - wj 

9. SOM Training
Kohonen layer
input pattern mp Wj
find the winner neuron z for an input pattern p (minimal Euclidian distance)
adapt weights of connections
input neurons  winner neuron
input neurons  neighbours

10. Example Credit Scoring
A1: Credit History
A2: Debts
A3: Collateral
A4: Income
We do not look at the Classification
SOM performs a Clustering

11. Credit Scoring good = {5,6,9,10,12} average = {3, 8, 13}
bad = {1,2,4,7,11,14}

12. Credit Scoring Pascal tool box (1991) 10x10 neurons
32,000 training steps

13. Visualisation of a SOM Colour reflects Euclidian distance to input
Weights used as coordinates of a neuron
Colour reflects cluster
NetDemo
ColorDemo
TSPDemo

14. Experiment: Pascal Program, 1998
Example TSP
Travelling Salesman Problem
A salesman has to visit certain cities and will return to his home. Find an optimal route!
problem has exponential complexity: (n-1)! routes
31/32 states in Mexico?
Experiment: Pascal Program, 1998

15. Nearest Neighbour: Example
Some cities in Northern Germany:
Initial city is Hamburg
Kiel
Rostock
Berlin
Hamburg
Hannover
Frankfurt
Essen
Schwerin
Exercise:
Put in the coordinates of 20 important places
Find a solution for the TSP using a SOM!

16. Draw a neuron at position:
SOM solves TSP
Kohonen layer
input
Draw a neuron at position:
(x,y)=(w1i,w2i)
w1i= six X
w2i= siy Y

17. SOM solves TSP Initialisation of weights:
weights to input (x,y) are calculated so that all neurons form a circle
The initial circle will be expanded to a round trip
Solutions for problems of several hundreds of towns are possible
Solution may be not optimal!

18. Applications Data Mining - Clustering Customer Data Weblog ...
You have a lot of data, but no teaching data available – unsupervised learning
you have at least an idea about the result
Can be applied as a first approach to get some training data for supervised learning