1. Support Vector Machines Formulation  Solve the quadratic program for some : min s. t.,, denotes where or membership.  Different error functions and measures of margin will lead to different SVM formulations.  Margin is maximized by minimizing reciprocal of margin.

2. Linear Program and Quadratic Program  An optimization problem in which the objective function and all constraints are linear functions is called a linear programming problem  If the objective function is convex quadratic while the constraints are all linear then the problem is called convex quadratic programming problem  Standard SVM formulation is in this category formulation is in this category 

3. 1-norm Support Vector Machines Good for Feature Selection  Solve the quadratic program for some : min s. t.,, denotes where or membership. Equivalent to solve a Linear Program as follows:

4. Decent Direction of  Move alone the decent direction for a certain stepsize will decrease the objective function value i.e.,

5. Feasible Direction of  Move alone the feasible direction from for a certain stepsize will not leave the feasible region i.e., where is the feasible region.

6. The Most Important Concept in Optimization (minimization)  A point is said to be an optimal solution of a unconstrained minimization if there exists no decent direction  A point is said to be an optimal solution of a constrained minimization if there exists no feasible decent direction  There might exist decent direction but move along this direction will leave out the feasible region