Presentation is loading. Please wait.

Presentation is loading. Please wait.

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Support Vector Machines l Find a linear hyperplane (decision boundary) that will separate.

Similar presentations


Presentation on theme: "© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Support Vector Machines l Find a linear hyperplane (decision boundary) that will separate."— Presentation transcript:

1 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l Find a linear hyperplane (decision boundary) that will separate the data

2 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l One Possible Solution

3 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l Another possible solution

4 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l Other possible solutions

5 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l Which one is better? B1 or B2? l How do you define better?

6 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l Find hyperplane maximizes the margin => B1 is better than B2

7 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ X1 X2 此條線為由原點沿 方向走 與 垂直的線

8 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines

9 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ w d x1 x2 x1-x2 wx+b = -1 wx+b=1 WX +b =0 wx+b=0 0 此線在沿法向量走 -b/|w| 的距離,與法向量垂直

10 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ 開始推導 l wx+b=0 這是一條怎樣的線呢? x a, x b 為線上兩點  wx a +b=0, wx b +b=0 兩式相減 w (x a -x b )=0 線上兩點所形成的 向量與 w 垂直 所以 w 是法向量 l w x s +b=k , x s 在上面的線 k > 0 l w x c +b=k’ , x c 在下面的線 k’ < 0 l 方塊分類為 y=1 , 圓圈分類為 y=-1 l 分類公式 z 是 y=1 類或 y=-1 類依據:

11 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ l 調整 w 與 b 得到 l w(x1-x2)=2 l |w|*d=2 l ∴

12 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Support Vector Machines l We want to maximize: –Which is equivalent to minimizing: –But subjected to the following constraints:  This is a constrained optimization problem –Numerical approaches to solve it (e.g., quadratic programming)

13 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ l Min f(w)= l Subject to l Change to l Min f(w)= l Subject to

14 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/

15 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Nonlinear Support Vector Machines l What if decision boundary is not linear?

16 © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Nonlinear Support Vector Machines l Transform data into higher dimensional space


Download ppt "© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Support Vector Machines l Find a linear hyperplane (decision boundary) that will separate."

Similar presentations


Ads by Google