Presenter : Lin, Shu-Han Authors : Jeen-Shing Wang, Jen-Chieh Chiang A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm 為SVC演算法開發的一個以混合參數搜尋方法為基礎的分群有效性評估模式 Presenter : Lin, Shu-Han Authors : Jeen-Shing Wang, Jen-Chieh Chiang PR (2008)

Outline Introduction of SVC Motivation Objective Methodology Experiments Conclusion Comments

SVC SVC is from SVMs SVMs is supervised clustering technique Fast convergence Good generalization performance Robustness for noise SVC is unsupervised approach Data points map to HD feature space using a Gaussian kernel. Look for smallest sphere enclose data. Map sphere back to data space to form set of contours. Contours are treated as the cluster boundaries. 3

SVC - Sphere Analysis To find the minimal enclose sphere with soft margin: To solve this problem, the Lagrangian function: a where ||・|| is the Euclidean norm, a is the center of the sphere, ξj are slack variables that loosen the constraints to allow some data points lying outside the sphere, C is a constant, and CΣξj is a penalty term. To solve the optimization problem as in (1), it is convenient to introduce the Lagrangian function The first and last terms are the objective function to be minimized. The second term represents the inequality constraints associated with the slack variables. The third term is the result of the non-negativity requirements on the values of ξj’s. 第一項與最後一項表示要最小化的目標函數，第二項表示加入差額變數的不等式限制式，第三項是在ξj為非負數的情形下所得到的結果。 4

SVC - Sphere Analysis 為了要使lagragian最小化，我們必須將R、a、ξj分別微分，並且設定為零。 Lagragian multiplier unknown 5

SVC - Sphere Analysis Karush-Kuhn-Tucker complementarity: Based on (3)–(7), we can classify each data point into the following: 1) an internal point; 2) an external point, and 3) a boundary point in the feature space. Point xj is classified as an internal point if βj = 0. When 0 < βj < C, the data point xj is denoted as an SV. SVs lying on the surface of the featurespace sphere are the so-called boundary points. These SVs can be used to describe the cluster contour in the input space. When βj = C, the data points located outside the feature space are defined as the external points or BSVs. Note that if C ≥ 1, no external points will exist. Hence, the value of C can be used to control the existence of external points, namely, outliers, during the clustering process. 所谓径向基函数 (Radial Basis Function 简称 RBF), 就是某种沿径向对称的标量函数。 通常定义为空间中任一点x到某一中心xc之间欧氏距离的单调函数 , 可记作 k(||x-xc||), 其作用往往是局部的 , 即当x远离xc时函数取值很小。 6

Wolfe dual optimization problem SVC -Sphere Analysis To find the minimal enclose sphere with soft margin: C : existence of outliers allowed Wolfe dual optimization problem C越大，outlier越少，C等於（或大於）0的時候，就不容許有任何outlier產生了。 a Bound SV; Outlier 7

SVC -Sphere Analysis Mercer kernel Kernel: Gaussian a The distance (similarity) between x and a: q : |clusters| & the smoothness/tightness of the cluster boundaries. Mercer kernel Kernel: Gaussian By using the above conditions, (1) can be turned into a Wolfe dual optimization problem with only variablesβj 所謂徑向基函數(Radial Basis Function簡稱RBF),就是某種沿徑向對稱的標量函數。 通常定義為空間中任一點x到某一中心xc之間歐氏距離的單調函數,可記作k(||x-xc||),其作用往往是局部的,即當x遠離xc時函數取值很小。 q : 如果不是使用高斯來當作kernel函數的話，就不會有q這個值了；但是無論你選什麼當kernel，都會面臨到選擇C這個數值。 Q是變異數的倒數；因為變異數越大的話，代表這個R越大，也代表這個分出來的群會比較寬鬆；相反的那麼也就是說，q越大的話，群就會越緊實。 a Gaussian function: 8

Motivation Drawbacks of Cluster validation Compactness Different densities or size As the # of clusters increases, it will monotonic decrease Separation Irregular cluster structures 9 9

Motivation Their previous study Can handle Different sizes Different densities Arbitrary shape But… 10 10

Objectives – A cluster validity method and a parameter search algorithm for SVC Auto determine the two parameter: Increasing q lead to increasing # of clusters C regulates the existence of outliers and overlapping clusters To Identify the optimal structure C這個參數代表著對outlier 跟overlap的容忍度 Q是變異數的倒數；因為變異數越大的話，代表這個R越大，也代表這個分出來的群會比較寬鬆；相反的那麼也就是說，q越大的話，群就會越緊實。 11

Methodology - Idea N=64, max # of cluster = , 8 q is related to the densities of the clusters Each cluster structure corresponds to an interval of q Identify the optimal structure is equivalent to finding the largest interval N=64, max # of cluster = , 8 12

Methodology - Problem How to locate overall search range of q How to detect outliers/noises How to identify the largest interval 13

Methodology – Locate range of q Lower bound Upper bound: Employ K-Means to get clusters, and get variance of each clusters vi Ascending order: cluster size 意思是如果資料集的range很大的話， q min就會比較小，意思是就要從更小的地方開始搜尋 然後意思是如果群裡面的變異數很大的話，q max就會比較小，意思是如果群都很大的話，那就不用搜尋到這麼多q了 n =3, the biggest 3 clusters’ variance 14

Methodology – Outlier Detection singleton Set q = qmax , the tightest of q And we get Copt, remove these outlier 15

Methodology – the largest interval qopt 16

Methodology – the largest interval Fibonacci search: locate the interval where the cluster structure is the same Bisection search n: iteration 17

Methodology – Overview Locate range of q the largest interval Outlier Detection 18

Experiments - Benchmark and Artificial Examples 19

Experiments - Outlier Copt 20

Experiments ？ 21

Conclusions q C A new measure: Inspired from the observations of q Determine the optimal cluster structure with its corresponding range of q and C q C

Comments Advantage Drawback Application Inspired from observation of parameter Drawback … Application SVC DBSCAN: MinPts / Eps