Support Vector Machines Exercise solutions Ata Kaban The University of Birmingham.

Slides:

Advertisements

Similar presentations

ECG Signal processing (2)

Advertisements

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Input Space versus Feature Space in Kernel- Based Methods Scholkopf, Mika, Burges, Knirsch, Muller, Ratsch, Smola presented by: Joe Drish Department of.

An Introduction of Support Vector Machine

Support Vector Machines

Support vector machine

Machine learning continued Image source:

Discriminative and generative methods for bags of features

Support Vector Machine

Support Vector Machines (and Kernel Methods in general)

Support Vector Machines and Kernel Methods

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Support Vector Machines (SVMs) Chapter 5 (Duda et al.)

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

Rutgers CS440, Fall 2003 Support vector machines Reading: Ch. 20, Sec. 6, AIMA 2 nd Ed.

Support Vector Classification (Linearly Separable Case, Primal) The hyperplanethat solves the minimization problem: realizes the maximal margin hyperplane.

1 Introduction to Kernels Max Welling October (chapters 1,2,3,4)

Support Vector Machines Kernel Machines

Classification Problem 2-Category Linearly Separable Case A- A+ Malignant Benign.

Support Vector Machines and Kernel Methods

Support Vector Machines

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of non- linear features.

2806 Neural Computation Support Vector Machines Lecture Ari Visa.

What is Learning All about ?  Get knowledge of by study, experience, or being taught  Become aware by information or from observation  Commit to memory.

Lecture 10: Support Vector Machines

Support Vector Machines Exercise solutions Ata Kaban The University of Birmingham.

Ch. Eick: Support Vector Machines: The Main Ideas Reading Material Support Vector Machines: 1.Textbook 2. First 3 columns of Smola/Schönkopf article on.

Linear hyperplanes as classifiers Usman Roshan. Hyperplane separators.

Support Vector Machine & Image Classification Applications

CS 8751 ML & KDDSupport Vector Machines1 Support Vector Machines (SVMs) Learning mechanism based on linear programming Chooses a separating plane based.

Support Vector Machines Mei-Chen Yeh 04/20/2010. The Classification Problem Label instances, usually represented by feature vectors, into one of the predefined.

Support Vector Machine (SVM) Based on Nello Cristianini presentation

1 SUPPORT VECTOR MACHINES İsmail GÜNEŞ. 2 What is SVM? A new generation learning system. A new generation learning system. Based on recent advances in.

INTRODUCTION TO ARTIFICIAL INTELLIGENCE Massimo Poesio LECTURE: Support Vector Machines.

计算机学院计算感知 Support Vector Machines. 2 University of Texas at Austin Machine Learning Group 计算感知计算机学院 Perceptron Revisited: Linear Separators Binary classification.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Classifiers Given a feature representation for images, how do we learn a model for distinguishing features from different classes? Zebra Non-zebra Decision.

CISC667, F05, Lec22, Liao1 CISC 667 Intro to Bioinformatics (Fall 2005) Support Vector Machines I.

Kernel Methods: Support Vector Machines Maximum Margin Classifiers and Support Vector Machines.

Linear hyperplanes as classifiers Usman Roshan. Hyperplane separators.

CS 1699: Intro to Computer Vision Support Vector Machines Prof. Adriana Kovashka University of Pittsburgh October 29, 2015.

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

CS558 Project Local SVM Classification based on triangulation (on the plane) Glenn Fung.

Support vector machine LING 572 Fei Xia Week 8: 2/23/2010 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A 1.

Support Vector Machine Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata November 3, 2014.

Support Vector Machines Tao Department of computer science University of Illinois.

Support Vector Machines. Notation Assume a binary classification problem. –Instances are represented by vector x   n. –Training examples: x = (x 1,

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

CSSE463: Image Recognition Day 15 Announcements: Announcements: Lab 5 posted, due Weds, Jan 13. Lab 5 posted, due Weds, Jan 13. Sunset detector posted,

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Linear hyperplanes as classifiers Usman Roshan. Hyperplane separators.

Greg GrudicIntro AI1 Support Vector Machine (SVM) Classification Greg Grudic.

Chapter 10 The Support Vector Method For Estimating Indicator Functions Intelligent Information Processing Laboratory, Fudan University.

Kernel Methods: Support Vector Machines Maximum Margin Classifiers and Support Vector Machines.

SVMs in a Nutshell.

An Introduction of Support Vector Machine In part from of Jinwei Gu.

1 Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable.

Support Vector Machines Part 2. Recap of SVM algorithm Given training set S = {(x 1, y 1 ), (x 2, y 2 ),..., (x m, y m ) | (x i, y i )   n  {+1, -1}

Day 17: Duality and Nonlinear SVM Kristin P. Bennett Mathematical Sciences Department Rensselaer Polytechnic Institute.

Support Vector Machines Reading: Textbook, Chapter 5 Ben-Hur and Weston, A User’s Guide to Support Vector Machines (linked from class web page)

Non-separable SVM's, and non-linear classification using kernels Jakob Verbeek December 16, 2011 Course website:

Support Vector Machines (SVMs) Chapter 5 (Duda et al.) CS479/679 Pattern Recognition Dr. George Bebis.

Geometrical intuition behind the dual problem

Nonparametric Methods: Support Vector Machines

Support Vector Machines Introduction to Data Mining, 2nd Edition by

Support Vector Machines

Support Vector Machines Most of the slides were taken from:

CSSE463: Image Recognition Day 14

Presentation transcript:

Support Vector Machines Exercise solutions Ata Kaban The University of Birmingham

a)What is the main idea behind linear Support Vector Machines (SVM)? Illustrate your explanation by drawing a figure. ANSWER The figure should show e.g. two linearly separable clusters of points, each cluster corresponding to a different class. Even though there are many possible separating lines, we pick the one that has maximal minimum distance from the closest points of each class. This choice is supported by a theorem in learning theory that bounds generalization error in terms of separation margins.

b) How can the linear SVM be made non-linear? ANSWER The non-linearity comes from using the `Kernel Trick': Instead of the dot product in the input space, use a kernel function K(x 1,x 2 ). For a proper kernel K (cf. Mercer Theorem) there is another (usually) high-dimensional feature space F and a feature map, such that K(x 1,x 2 ) can be interpreted as a dot product between images F(x i ) in F.

c) Decide which of the following formulae define proper kernels and explain why. ANSWER K 3 and K 4 are proper kernels, since given a proper kernel K, then aK, a > 0, is a proper kernel. Also given proper kernels K' and K'', then K' K'' is a proper kernel. K 5 is not a proper kernel, since the Gram matrix will be negative definite for all training sets.

d) Consider the 2-dimensional inputs. Is the following a proper kernel? Explain why. ANSWER It is a proper kernel, since for any real valued function over the input space, is a proper kernel.

Remember to master the worked questions / exercises How do we know if a kernel is proper? - - given a proper kernel K, then aK, a > 0, is a proper kernel - - given proper kernels K' and K'', then K' K'' is a proper kernel - - if K is a proper kernel, for any real valued function over the input space, is a proper kernel.