Presentation is loading. Please wait.

Presentation is loading. Please wait.

An Automatic Method for Selecting the Parameter of the RBF Kernel Function to Support Vector Machines Cheng-Hsuan Li 1,2 Chin-Teng.

Published byGriffin Scott Modified over 7 years ago

Similar presentations

Presentation on theme: "An Automatic Method for Selecting the Parameter of the RBF Kernel Function to Support Vector Machines Cheng-Hsuan Li 1,2 Chin-Teng."— Presentation transcript:

1 An Automatic Method for Selecting the Parameter of the RBF Kernel Function to Support Vector Machines Cheng-Hsuan Li 1,2 (ChengHsuanLi@gmail.com) Chin-Teng Lin 1 (ctlin@mail.nctu.edu.tw) Bor-Chen Kuo 2 (kbc@mail.ntcu.edu.tw) Hui-Shan Chu 2 (Roxanne90@gmail.com) 1 Institute of Electrical Control Engineering, National Chiao Tung University, Hsinchu, Taiwan, R.O.C. 2 Graduate Institute of Educational Measurement and Statistics, National Taichung University, Taichung, Taiwan, R.O.C.

2 Motivation There are a lot of papers on TGARS which use the k-fold cross- validation to find the proper parameter of the RBF kernel function. B. Waske, S. van der Linden,J. A. Benediktsson, A. Rabe, and P. Hostert, “Sensitivity of Support Vector machines to random feature selection in classification of hyperspectral data,” IEEE Trans. Geosci. Remote Sens., vol. 48, no. 7, pp.2880-2889, July 2010. J. Chen, C. Wang, and R. Wang, “Using stacked generalization to combine SVMs in magnitude and shape feature spaces for classification of hyperspectral data,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 7, pp.2193-2205, July 2009. L. Bruzzone, and M. Marconcini, “Toward the automatic updating of land-cover maps by a domain-adaptation SVM classifier and a circular validation strategy,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 4, pp.1108-1112, April 2009. T.V. Bandos, L. Bruzzone, and G. Camps-Valls, “Classification of hyperspectral images with regularized linear discriminant analysis,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 3, pp. 862-873, March 2009. G. Camps-Valls, L. Gomez-Chova, J. Munoz-Mari, J.L. Rojo-Alvarez, and M. Martinez-Ramon, “Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 6, pp. 1822-1835, June 2008. etc. Nevertheless, it is time consuming.

3 Kernel Method The samples in the same class should be mapped into the same area. The samples in the different classes should be mapped into the different areas.

4 The Properties of the RBF Kernel In the feature space determined by the RBF kernel, the norm of every sample is one and positive. Hence, the samples will be mapped onto the surface of a hypersphere.

5 The Properties of the RBF Kernel Hence, the cosine values, i.e., the values of the RBF kernel function, indicate the similarities between samples.

6 The Properties of the RBF Kernel If the cosine values are close to 1, then these samples are more similar in the feature space. If the cosine values are close to 0, then these samples are more dissimilar in the feature space. They are more similar and the corresponding cosine value is close to 1. They are more dissimilar and the corresponding cosine value is close to 0.

7 The Ideal Distribution For example, there are three classes and are the -th samples in class,. We want to find a feature space such that the distribution of samples in this feature space is like the following figures.

8 The Ideal Distribution The values of the RBF kernel function should be close to 1 if the samples are in the same class. That is The values of the RBF kernel function would be close to 0 if the samples are in the different classes. That is

9 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

10 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

11 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

12 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

13 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

14 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

15 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

16 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

17 An Example of Varying the Parameter small large Vectors in the original space Vectors in the corresponding feature space

18 Proposed Criteria The mean of values applied by the RBF kernel function on the samples in the same class are close to 1, i.e.,

19 Proposed Criteria The mean of values applied by the RBF kernel function on the samples in different classes are close to 0, i.e.,

20 Proposed Criteria We want to find a parameter such that This means that The proposed criterion is defined as

21 VERSUS -RBF KERNEL Obtain from the Indian Pine Site dataset

22 VERSUS ACCURACIES-RBF KERNEL

23 Observations has a global minimum on. From the figure “ versus,” one can find that the values near the minimum value occur at in the rage (3500,4000). From the figure “ versus accuracies,” one can find that the values near the highest accuracy or kappa accuracy also occur at in the rage (3500,4500). The obtained by our proposed method is approximate to the which can achieve the highest accuracy. Therefore, the minimizer of is a good estimator of.

24 How to Find the Best and are differentiable with respect to. is also differentiable with respect to. The gradient descent method is used to find the optimizer.

25 Experiment Data sets IR Image Image (No. of bands) Indian Pine Site (dims=220) Washington, DC Mall (dims=191) # of class97 Category (No. of labeled data) Soybeans-min, Soybeans-notill Soybeans-clean, Corn-min Grass/Pasture, Grass/Tree Corn-notill, Hay-windrowed Woods Roof Road Path Grass Tree Water Shadow

26 Experiment Results Table 1. Overall and Kappa Accuracies in Indian Pine Site Dataset method CPU Time (sec) Overall Accuracy Overall Kappa Accuracy 20 CV197.5081920.7490.712 OP21.223622.800.7680.733 40 CV531.2581920.8110.781 OP58.783615.360.8310.804 300 CV22859.9540960.9280.915 OP2416.613795.660.9280.916 : the number of samples in class i OP : our proposed method CV : 5-fold cross-validation on the set

27 Experiment Results Table 2. Overall and Kappa Accuracies in Washington, DC Mall method CPU Time (sec) Accuracy Kappa Accuracy 20 CV92.341310720.8290.80 OP10.02179265.610.8300.80 40 CV251.955242880.8830.86 OP28.27180187.810.8810.86 300 CV14256.955242880.9490.94 OP1460.19181939.490.9490.94 : the number of samples in class i OP : our proposed method CV : 5-fold cross-validation on the set

28 Conclusion In this paper, an automatic method for selecting the parameter of the RBF kernel was proposed. We have experimentally compared it to k-fold cross- validation. The experiment results of two hyperspectral images show that the cost of the proposed method is 9 times less. Furthermore, we will try to apply our proposed method to other kernel-based algorithms. In addition, we will develop similar procedures for other kernel functions.

29 Thanks for your attention Cheng-Hsuan Li (ChengHsuanLi@gmail.com) Chin-Teng Lin (ctlin@mail.nctu.edu.tw) Bor-Chen Kuo (kbc@mail.ntcu.edu.tw) Hui-Shan Chu (Roxanne90@gmail.com)

30 Proposed Criteria Note that Hence,

31 Experimental Results Table 3. Overall and Kappa Accuracies in UCI datasets DatasetsMethod CPU Time (sec) Overall Accuracy Overall Kappa Accuracy Ionosphere CV32.22280.9800.93 OP2.531.9440.9800.93 Monks CV14.031.3322.670.8670.72 OP1.711.36240.8740.74 Pima CV318.88128640.7550.49 OP25.0273.83320.7860.56 Liver CV34.8696241.040.6760.32 OP2.8013.241.340.6780.32 WDBC CV247.1912832.000.9300.85 OP14.22404.592560.9440.88 Iris CV34.800.520.9730.96 OP2.481.4540.9870.98

32 How to Find the Best

Download ppt "An Automatic Method for Selecting the Parameter of the RBF Kernel Function to Support Vector Machines Cheng-Hsuan Li 1,2 Chin-Teng."

Similar presentations

ECG Signal processing (2)

ECG Signal processing (2)
1 Manifold Alignment for Multitemporal Hyperspectral Image Classification H. Lexie Yang 1, Melba M. Crawford 2 School of Civil Engineering, Purdue University.

1 Manifold Alignment for Multitemporal Hyperspectral Image Classification H. Lexie Yang 1, Melba M. Crawford 2 School of Civil Engineering, Purdue University.
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Evaluating Color Descriptors for Object and Scene Recognition Koen E.A. van de Sande, Student Member, IEEE, Theo Gevers, Member, IEEE, and Cees G.M. Snoek,

Evaluating Color Descriptors for Object and Scene Recognition Koen E.A. van de Sande, Student Member, IEEE, Theo Gevers, Member, IEEE, and Cees G.M. Snoek,
Machine learning continued Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning continued Image source:
Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,

Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,
LPP-HOG: A New Local Image Descriptor for Fast Human Detection Andy Qing Jun Wang and Ru Bo Zhang IEEE International Symposium.

LPP-HOG: A New Local Image Descriptor for Fast Human Detection Andy Qing Jun Wang and Ru Bo Zhang IEEE International Symposium.
Feature/Model Selection by Linear Programming SVM, Combined with State-of-Art Classifiers: What Can We Learn About the Data Erinija Pranckeviciene, Ray.

Feature/Model Selection by Linear Programming SVM, Combined with State-of-Art Classifiers: What Can We Learn About the Data Erinija Pranckeviciene, Ray.
Local or Global Minima: Flexible Dual-Front Active Contours Hua Li Anthony Yezzi.

Local or Global Minima: Flexible Dual-Front Active Contours Hua Li Anthony Yezzi.
Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.

Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Locally Constraint Support Vector Clustering

Locally Constraint Support Vector Clustering
The value of kernel function represents the inner product of two training points in feature space Kernel functions merge two steps 1. map input data from.

The value of kernel function represents the inner product of two training points in feature space Kernel functions merge two steps 1. map input data from.
Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.

Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.
Speaker Adaptation for Vowel Classification

Speaker Adaptation for Vowel Classification
Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Deep Belief Networks for Spam Filtering

Deep Belief Networks for Spam Filtering
Sample-Separation-Margin Based Minimum Classification Error Training of Pattern Classifiers with Quadratic Discriminant Functions Yongqiang Wang 1,2, Qiang.

Sample-Separation-Margin Based Minimum Classification Error Training of Pattern Classifiers with Quadratic Discriminant Functions Yongqiang Wang 1,2, Qiang.
Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy Remote.

Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I Povo, Trento, Italy Remote.
 C. C. Hung, H. Ijaz, E. Jung, and B.-C. Kuo # School of Computing and Software Engineering Southern Polytechnic State University, Marietta, Georgia USA.

 C. C. Hung, H. Ijaz, E. Jung, and B.-C. Kuo # School of Computing and Software Engineering Southern Polytechnic State University, Marietta, Georgia USA.

Similar presentations

Ads by Google