1. Self-organizing map Speech and Image Processing Unit Department of Computer Science University of Joensuu, FINLAND Pasi Fränti Clustering Methods: Part 9

2. SOM main principles Self-organizing map (SOM) is a clustering method suitable especially for visualization. Clustering represented by centroids organized in a 1-d or 2-d network. Dimensionality reduction and visualization possibility achieved as side product. Clustering performed by competitive learning principle.

3. Self-organizing map Initial configuration M nodes, one for each cluster Initial locations not important Nodes connected by network topology (1-d or 2-d)

4. Self-organizing map Final configuration Node locations adapt during learning stage Network keeps neighbor vectors close to each other Network limits the movement of vectors during learning

5. SOM pseudo code (1/2) Learning stage

6. SOM pseudo code Update centroids (2/2)

7. Competitive learning Each data vector is processed once. Find nearest centroid: The centroid is updated by moving it towards the data vector by: Learning stage similar to k-means but centroid update has different principle.

8. Decreases with time  movement is large in the beginning but eventually stabilizes. Linear decrease of weighting: Learning rate (  ) Exponential decrease of weighting:

9. Neighborhood (d) Neighboring centroids are also updated: Effect is stronger for nearby centroids:

10. Weighting of the neighborhood Weighting decreases exponentially

11. Number of iterations T –Convergence of SOM is rather slow  Should be set as high as possible –Roughly 100-1000 iterations at minimum. Size of the initial neighborhood D max –Small enough to allow local adaption. –Value D=0 indicates no neighbor structure Maximum learning rate A –Higher values have mostly random effect. –Most critical are the final stages (D  2) –Optimal choices of A and D max highly correlated. Parameter setup

12. Difficulty of parameter setup Fixing total number of iterations (T  D max ) to 20, 40 and 80. Optimal parameter combination non-trivial.

13. To reduce the effect of parameter set- up, should be as high as possible. Enough time to adapt at the cost of high time complexity. Adaptive number of iterations: Adaptive number of iterations For D max =10 and T max =100: Ti = {1, 1, 1, 1, 2, 3, 6, 13, 25, 50, 100}

14. Example of SOM (1-d) One cluster too many One cluster missing

15. Example of SOM (2-d) (to appear sometime in future)

16. 1.T. Kohonen, Self-Organization and Associative Memory. Springer- Verlag, New York, 1988. 2.N.M. Nasrabadi and Y. Feng, "Vector quantization of images based upon the Kohonen self-organization feature maps", Neural Networks, 1 (1), 518, 1988. 3.P. Fränti, "On the usefulness of self-organizing maps for the clustering problem in vector quantization", 11th Scandinavian Conf. on Image Analysis (SCIA’99), Kangerlussuaq, Greenland, vol. 1, 415-422, 1999. Literature