1. Support Vector Machines Exercise solutions Ata Kaban The University of Birmingham

2. Exercise 1. a)What is the main idea behind linear Support Vector Machines (SVM)? Illustrate your explanation by drawing a figure. ANSWER The figure should show e.g. two linearly separable clusters of points, each cluster corresponding to a different class. Even though there are many possible separating lines, we pick the one that has maximal minimum distance from the closest points of each class. This choice is supported by a theorem in learning theory that bounds generalization error in terms of separation margins.

3. b) Given K1 and K2 two proper kernels. Determine which of the following formulae define proper kernels and explain why. ANSWER K 4 and K 5 are proper kernels, since given a proper kernel K, then aK, a > 0, is a proper kernel. Also given proper kernels K' and K'', then K' K'' is a proper kernel. K 3 is not a proper kernel because it is negative definite for any data sets.

4. c) Consider the 2-dimensional inputs. Is the following a proper kernel? Explain why. ANSWER It is a proper kernel, since for any real valued function over the input space, is a proper kernel.

5. Remember to master the worked questions / exercises How do we know if a kernel is proper? - - given a proper kernel K, then aK, a > 0, is a proper kernel - - given proper kernels K' and K'', then K' K'' is a proper kernel - - if K is a proper kernel, for any real valued function over the input space, is a proper kernel.