Presentation is loading. Please wait.

Presentation is loading. Please wait.

Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel Department of Management Science and Engineering Stanford University.

Similar presentations


Presentation on theme: "Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel Department of Management Science and Engineering Stanford University."— Presentation transcript:

1 Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. 1 Lecture #3; Based on slides by Yinyu Ye

2 LP Feasible Region in the Inequality Form 2 x simultaneously satisfy This is the intersection of the m Half-spaces, and it is a convex (polyhedron) set Lecture #3; Based on slides by Yinyu Ye

3 3

4 4

5 Corner or Extreme Points 5 Convex Hull: Extreme Points: A point in a set that does not belong to the hull (convex combination) of the other points For LP in inequality form, an extreme point is the intersection of n hyperplanes associated with the inequality constraints. Lecture #3; Based on slides by Yinyu Ye

6 Feasible Direction 6 Direction: A direction is notated by a vector d It is always associated with a “location” or point x Together a point and a direction define a ray: x + ϵd, for all ϵ > 0 where d and kd are considered the same direction for all k > 0 Feasible Direction: A direction, d, is said to be “feasible” (relative to a given feasible point x) if x + ϵd is feasible for some ϵ> 0 For LP, all feasible directions at a feasible point form a convex (cone) set Lecture #3; Based on slides by Yinyu Ye

7 Extreme Feasible Direction 7 A feasible direction d is extreme if d cannot be written as an convex combination of other feasible directions Interior Point is a point x where every direction is feasible Lecture #3; Based on slides by Yinyu Ye

8 LP Problem in Inequality Form 8 Lecture #3; Based on slides by Yinyu Ye

9 9 Recall the Production Problem Objective contour c Lecture #3; Based on slides by Yinyu Ye

10 Basic Theorems of Linear Programming 10 All LP problems fall into one of three cases: Problem is infeasible: Feasible region is empty. Problem is unbounded: Feasible region is unbounded towards the optimizing direction. Problem is feasible and bounded; and in this case: – there exists an optimal solution or optimizer. – There may be a unique optimizer or multiple optimizers. – All optimizers form a convex set and they are on a face of the feasible region. – There is always at least one corner (extreme) optimizer if the feasible region has a corner point. Moreover, Local optimality implies global optimality Lecture #3; Based on slides by Yinyu Ye

11 Sketch Proof of Local Optimality Implies Global 11 (P)minimize f (x) subject to x ∈ Ω ⊂ R n, is a Convex Optimization problem if f (x) is a convex function over a convex feasible region Ω. Proof by contradiction. Suppose x’ is a local minimizer but not a global minimizer x ∗, that is, x’ ∈ Ω and x ∗ ∈ Ω but f (x ∗ ) < f (x’). Now the convex combination point αx’ + (1 − α)x ∗ must be feasible (why?), and f (αx’+ (1 − α)x ∗ ) ≤ αf (x’)+ (1 − α)f (x ∗ ) < f (x’) for any 0 ≤ α < 1. This contradicts the local optimality as α can be arbitrarily close to 1 so that αx’+ (1 − α)x ∗ can be arbitrarily close to x’. Lecture #3; Based on slides by Yinyu Ye

12 12 Optimality Certification of the Production Problem a4 a5 a1 a2 a3 a2 a3 a4 Objective contour c Lecture #3; Based on slides by Yinyu Ye

13 13 Feasible Directions at a Corner a4 a5 a1 a2 a3 a2 a3 a4 Objective contour c Lecture #3; Based on slides by Yinyu Ye

14 How to Certify a Corner being an Optimizer 14 Every feasible direction at the corner is a worsening (descent in this case) direction, that is, c T d ≤0 in this case. Or, equivalently, c is a conic combination of the normal directions at the corner point, that is, there are multipliers α 1 ≥0 and α 2 ≥0, such as c= α 1 a i1 + α 2 a i2, in the 2-dimensional case. In the n-dimensional case: c= α 1 a i1 +…+ α n a in where a i1,…, a in are the normal directions associated with the corner point. Lecture #3; Based on slides by Yinyu Ye

15 LP in Standard (Equality) Form 15 Lecture #3; Based on slides by Yinyu Ye

16 Reduction to Standard Form 16 max c T x to min – c T x Eliminating ”free” variables: substitute with the difference of two nonnegative variables x := x l − x ll, (x l, x ll ) ≥ 0. Eliminating inequalities: add a slack variable a T x ≤ b = ⇒ a T x + s = b, s ≥ 0 a T x ≥ b = ⇒ a T x − s = b, s ≥ 0 Lecture #3; Based on slides by Yinyu Ye

17 Reduction of the Production Problem 17 min -x 1 -2x 2 s.t.x1x1 +x3+x3 = 1 x2x2 +x4+x4 x1x1 +x2+x2 +x5+x5 = 1.5 (x1,(x1,x2,x2,x 3, x 4, x5)x5)≥ 0 x 3, x 4,and x 5 are called slack variables Lecture #3; Based on slides by Yinyu Ye

18 Basic and Basic Feasible Solution 18 In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve A B x B = b for the dimension-m vector x B. By setting the variables, x N, of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b. Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in x B are called basic variables, those in x N are nonbasic variables, and A B is called a basis. If a basic solution x B ≥ 0, then x is called a basic feasible solution, or BFS. Note that A B and x B follow the same index order in B. Two BFS are adjacent if they differ by exactly one basic variable. Lecture #3; Based on slides by Yinyu Ye

19 BS of the Production Problem in Equality Form 19 x1x1 +x3+x3 = 1 x2x2 +x4+x4 x1x1 +x2+x2 +x5+x5 = 1.5 (x1,(x1,x2,x2,x 3, x 4, x5)x5)≥ 0 Basis3,4,51,4,53,4,13,2,53,4,21,2,31,2,41,2,5 Feasible?√√√√√ x 1, x 2 0, 01, 01.5, 00, 10, 1.5.5, 11,.51, 1 x1x1 x2x2 Lecture #3; Based on slides by Yinyu Ye

20 BFS and Corner Point Equivalence Theorem 20 Theorem Consider the feasible region in the standard LP form. Then, a basic feasible solution and a corner (extreme) point are equivalent; the formal is algebraic and the latter is geometric. Algorithmically, we need only to look at BFSs for finding an optimizer. But still too many to look… Lecture #3; Based on slides by Yinyu Ye


Download ppt "Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel Department of Management Science and Engineering Stanford University."

Similar presentations


Ads by Google