1. Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology On multidimensional scaling and the embedding of self-organizing maps Presenter : Shu-Ya Li Authors : Hujun Yin NN, 2008

2. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 2 Outline Motivation Objective Methodology Review  SOM, ViSOM & MDS  On multidimensional scaling of SOM  PCA & Principal curve  Nonlinear principal manifold and SOM  On the embedding of SOM Experiments and Results Conclusion Personal Comments

3. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 3 Motivation SOM and ViSOM, have been known to yield similar results to multidimensional scaling (MDS).  However, the exact connection has not been established. The SOM-based methods not only produce topological or metric scaling but also provide a principal manifold.

4. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 4 Objectives This paper reveals the connection between the SOM (or its variant ViSOM) and multidimensional scaling(MDS) through analyzing their cost functions. Their relationship with the principal manifold is also discussed.

5. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 5 1.Initialize SOM 2.For each input data 2.1 Identify its Best Matching Unit (BMU) 2.2 Update BMU and its neighborhood 3.Repeat Step 2 till centroids don’t change much or a threshold is exceeded 4.Assign each data to its BMU and return the BMS and clusters Basic SOM algorithm Data x1=[8, 5, 9] x2=[7, 4, 2] … x1=[8, 5, 9] training Final projection BMU m i =[7, 5, 8] × +○

6. Intelligent Database Systems Lab N.Y.U.S.T. I. M. ViSOM  To preserve distance/metric (locally) on the map  To extrapolate smoothly Visualization induced SOM (ViSOM) preserve distance on the map 存在比例關係

7. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Multidimensional Scaling (MDS) 1. Assign points to arbitrary coordinates in p-dimensional space. 2. Compute euclidean distances among all pairs of points, to form the Dhat matrix. 3. Compare the Dhat matrix with the input D matrix by evaluating the stress function. 4. Adjust coordinates of each point MDS 1. Classical MDS 2. Metric MDS 投入原始距離 ( 或相似 ) 矩陣 3. Nonmetric MDS 投入距離 ( 或相似 ) 資料之順序等級 input Data matrix Dhat matrix 座標上點跟點 之間的距離 Compare

8. Intelligent Database Systems Lab N.Y.U.S.T. I. M. On multidimensional scaling of SOM SOM is a nonmetric MDS or ordinal scaling.  nonmetric MDS condition ViSOM is a distance-preserving, metric MDS.  The cost function of metric MDS  The cost function of the SOM can be expressed as 真實資料距離 投射到 MDS Map 的距離 Data x1=[8, 5, 9] x2=[7, 4, 2] … 投射到 SOM Map m 2 =[7, 5, 2] m 1 =[8, 5, 8] x1=[8, 5, 9] x2=[7, 4, 2] SOMViSOM

9. Intelligent Database Systems Lab N.Y.U.S.T. I. M. PCA Principal Curve/Surface Reduction by x-axis projection: [X 1 X 2 ]  [X 1 ] [1, 1]  [1] [1, 0.5]  [1] X1 X2 X1 X2 Y1 Y2 Reduction by PCA Y 1 = a 11 x 1 + a 12 x 2 Y 2 = a 21 x 1 + a 22 x 2 [X1 X2]  [Y1] [X 1 X 2 ]  [Y 1 Y 2 ] [1, 1]  [1.414, 0] [1, 0.5]  [1.2, -0.3] X1 X2 Y1

10. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Principal Curve/Surface 10 Principal Curve/Surface  Projection index  Self-consistent principal curve  Kernel smoother X1 X2 Y1

11. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Nonlinear principal manifold and SOM 11 ViSOM is a discrete principal manifold, and it is also a MDS. In the SOM, data are projected onto the nodes rather than onto the curve/surface. The smoothing process in the SOM and ViSOM, as a convergence criterion The smoothing process in the ViSOM resembles that of the principal curve as shown below The MDS and principal manifold perform the same underlying task at least in the context of data visualization and dimension reduction.

12. Intelligent Database Systems Lab N.Y.U.S.T. I. M. On the embedding of SOM growing ViSOM  Start with a small initial map, say M0 × M0. (5*5)  Update the weights of the neurons of the neighborhood using the ViSOM principle  Grow the map by adding a column or row to the side with the highest activities  Refresh the map (neurons) probabilistically  Check if the map has converged  Project the data samples onto the map, either to the neurons or by the LLP resolution enhancement method

13. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 13 Experiments

14. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 14 Conclusion This paper reveals the connection between the SOM, ViSOM and MDS through analyzing their cost functions.  SOMs and MDS are similar mappings for the principle of data visualization.  The ViSOM is closer to MDS than SOM. SOM is a useful tool for data clustering, relational visualisation (nonmetric scaling) and management. ViSOM is particularly suited for direct visualisation and is a metric preserving nonlinear manifold. gViSOM is an effective algorithm.

15. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 15 Personal Comments Advantage  … Drawback  如果圖多一點就好了  Application  …