The Simplex Algorithm 虞台文 大同大學資工所 智慧型多媒體研究室

Content Basic Feasible Solutions The Geometry of Linear Programs Moving From Bfs to Bfs Organization of a Tableau

Linear Programming Basic Feasible Solutions 大同大學資工所 智慧型多媒體研究室

The Goal of the Simplex Algorithm Any LP can be converted into the standard form

Convex Polytope defines a convex set F F may be 1.Bounded 2.Unbounded 3.Empty

The Basic Idea of Simplex Algorithm basic feasible solution at a corner Finding optimum by moving around the corner of the convex polytope in cost descent sense

The Basic Idea of Simplex Algorithm basic feasible solution at a corner Finding optimum by moving around the corner of the convex polytope in cost descent sense How to find an initial feasible solution? How to move from corner to corner?

Assumption 1 Assume that A is of rank m. There is a basis Independent has an inverse.

Basic Solution The basis solution corresponding to B is: The basis solution may be infeasible.

Example x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7

x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 feasible

Example x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 infeasible

Basic Feasible Solution defines an nonempty convex set F If a basic solution is in F, it is called a basic feasible solution (bfs).

The Existence of BFS’s F may be 1.Bounded 2.Unbounded 3.Empty If F is nonempty, at least one bfs. defines an nonempty convex set F

The Existence of BFS’s F may be 1.Bounded 2.Unbounded 3.Empty If F is nonempty, at least one bfs. How to find it?

Assumptions 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. defines an nonempty convex set F

Assumptions 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. There exist bfs’s. defines an nonempty convex set F Ensure there is a bounded solution. Are all bfs’s the vertices of the convex polytope defined by F ?

The Simplex Alogrithm The Geometry of Linear Programs 大同大學資工所 智慧型多媒體研究室

Linear Subspaces of R d S  R d is a subspace of R d if it is closed under vector addition and scalar multiplication. S defined below is a subspace of R d. a set of homogenous linear equations

Dimensions

Affine subspaces A translated linear subspace. E.g., an affine subspace a set of nonhomogenous linear equations an affine subspace

Dimensions an affine subspace

Subsets of R d The following subsets are not subspace or affine subspace. A line segment The first quadrant A halfspace

Dimensions The following subsets are not subspace or affine subspace. A line segment The first quadrant A halfspace The dimension of any subset of R d is the smallest dimension of any affine subspace which contains it. Dim = 1 Dim = 2

The Feasible Spaces of LP 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. 1. A is of rank m. 2. F is nonempty. 3. c ’ x is bounded below for x  F. defines an nonempty convex set F

Hyperplane/Halfspace An affine subspace of R d of dimension d  1 is called a hyperplane. A hyperplane defines two closed halfspaces: Fact: Halfspaces are convex.

Convex Polytopes The intersection of a finite number of halfspaces, when it is bounded and nonempty is called a convex polytope, or simply a polytope. Fact: Halfspaces are convex.

x1x1 x2x2 x3x3 Example     (0, 0, 3) (1, 0, 3) (2, 0, 2) (2, 0, 0) (2, 2, 0)(0, 2, 0) (0, 1, 3) (0, 0, 0)    

More on Polytopes Geometric views of a polytope – The convex hull of a finite set of points. – The intersection of a finite number of halfspaces, when it is bounded and nonempty, i.e., defined by The algebraic view of a polytope The feasible space defined by LP (in standard form). Any relation?

LP  Polytopes Assume m columns n  m columns x1x1 x2x2 xnmxnm x n  m+1 xnxn A can always be in this form if rank(A)=m. LP

LP  Polytopes Assume m columns n  m columns x1x1 x2x2 xnmxnm x n  m+1 xnxn LP

LP  Polytopes 00 LP

Polytopes  LP m inequalities m equalities

Polytopes  LP m inequalities Slack variables

Polytopes  LP Slack variables

Polytopes and LP Slack variables H I H

Polytopes and LP The answer

Polytopes and LP The answer

Polytopes & F of LP defines a polytope defines a feasible set Are there any relations?

Polytopes & F of LP defines a polytope defines a feasible set Some points in P are vertices. Some points in F are bfs’s. Are there any relations?

Theorem 1 a convex polytope a feasible set of the corresponding LP is a vertex of P is a bfs.

Theorem 1 Pf) “”“” See textbook.

Theorem 1 Pf) “”“” Fact:is a vertex, then it cannot be the strict combination of points of P, i.e.,

Theorem 1 Pf) “”“” Fact:is a vertex, then it cannot be the strict combination of points of P, i.e.,

Theorem 1 Pf) “”“” Fact: Define B = {A j : x j > 0, 1  j  n} Let We want to show at most m nonzero x i ’s. We want to show that vectors in B are linearly independent. Suppose not. Then, some d j  0. >0 if  is sufficiently small.

Theorem 1 Pf) “”“” Fact: Define B = {A j : x j > 0, 1  j  n} Let We want to show at most m nonzero x i ’s. We want to show that vectors in B are linearly independent. Suppose not. Then, some d j  0. >0 if  is sufficiently small. Define  is sufficiently small.

Theorem 1 Pf) “”“” Fact: Define B = {A j : x j > 0, 1  j  n} Let We want to show at most m nonzero x i ’s. A j are linearly independent. | B |  m.  Since rank(A) = m, we can always augment B to include m linearly independent vectors. Using B to form basic columns renders x a bfs.

Discussion Pf) “”“” Fact: Define B = {A j : x j > 0, 1  j  n} Let We want to show at most m nonzero x i ’s. A j are linearly independent. | B |  m.  Since rank(A) = m, we can always augment B to include m linearly independent vectors. Using B to form basic columns renders x a bfs. If | B | < m, there may be many ways to augment to m linearly independent vectors. Two different B and B ’ may corresponds to the same bfs. If | B | < m, there may be many ways to augment to m linearly independent vectors. Two different B and B ’ may corresponds to the same bfs. Two different, their corresponding B and B ’ must be different.

Example x1x1 x2x2 x3x3     (0, 0, 3) (1, 0, 3) (2, 0, 2) (2, 0, 0) (2, 2, 0)(0, 2, 0) (0, 1, 3) (0, 0, 0)    

Example Corresponding F of LP

Example Corresponding F of LP

Example Corresponding F of LP B and B ’ determine the same bfs, i.e, x = (2, 2, 0, 0, 0, 3, 0) ’. B and B ’ determine the same bfs, i.e, x = (2, 2, 0, 0, 0, 3, 0) ’.

Degeneration A bfs is called degenerate if it contains more than n  m zeros.

x x Theorem 2 A bfs is called degenerate if it contains more than n  m zeros. If two distinct bases corresponding to the same bfs x, then x is degenerate. Pf) Let B and B ’ be two distinct bases which determine the same bfs x. B B’B’ n m m

x x Theorem 2 A bfs is called degenerate if it contains more than n  m zeros. Pf) Let B and B ’ be two distinct bases which determine the same bfs x. B B’B’ n m m 00 x has more than n  m zeros and, hence, is degenerate. If two distinct bases corresponding to the same bfs x, then x is degenerate.

Discussion A bfs is called degenerate if it contains more than n  m zeros. If two distinct bases corresponding to the same bfs x, then x is degenerate. Changing basis may keep bfs unchanged.

More on Theorem 1 and 2 Vertices of polytopeBfs’s of LP Change vertexChange bfs ? A bfs is determined by a chosen basis. Change verticesChange basis ?

Costs ? ? ?

Costs constant

Theorem 3 There is an optimal bfs in any instance of LP. Furthermore, if q bfs’s are optimal, their convex combinations are also optimal.

Theorem 3 There is an optimal bfs in any instance of LP. Furthermore, if q bfs’s are optimal, their convex combinations are also optimal. Let x 0  P be the optimal solution, and let x 1, …, x N be the vertices of P. with Let j be the index corresponding to the vertex with lowest cost. x j is the optimum. Pf) and

Theorem 3 There is an optimal bfs in any instance of LP. Furthermore, if q bfs’s are optimal, their convex combinations are also optimal. Assume y 1, …, y q be the optimal vertices. Pf) Let withand

Linear Programming Moving from Bfs to Bfs 大同大學資工所 智慧型多媒體研究室

Facts The optimal solution can be found from vertices of the corresponding polytope. The bfs’s of LP and the vertices of polytope are close correlated. The algorithm to solve LP: – Move from vertex to vertex; or – Move from bfs to bfs

The BFS’s Basis columns Nonbasis columns The bfs is determined from the set of basis columns, i.e., B. Move from bfs to bfs:

Move from Bfs to Bfs Let x 0 be the bfs determined by B. Denote the basic components of x 0 as x i0, i = 1, …, m. For any, we have    

Move from Bfs to Bfs m + 1 columns are involved. Choose one A j from to enter B. Choose one A B(i) to leave B. Who enters? Who leaves?

Move from Bfs to Bfs m + 1 columns are involved. Choose one A j from to enter B. Choose one A B(i) to leave B. Who enters? Who leaves?  must be positive. Make one of them zero, and keep others positive. The one that we want to make zero must have x ij > 0. Why?

Move from Bfs to Bfs Suppose that A j wants to enter B. Then, we choose Make one of them zero, and keep others positive.

Move from Bfs to Bfs Suppose that A j wants to enter B. Then, we choose Make one of them zero, and keep others positive. 1.How about if x i0 = 0 for some i ? 2.How about if all x ij  0 ? 1.How about if x i0 = 0 for some i ? 2.How about if all x ij  0 ?

Example x1x1 x2x2 x3x3     (0, 0, 3) (1, 0, 3) (2, 0, 2) (2, 0, 0) (2, 2, 0)(0, 2, 0) (0, 1, 3) (0, 0, 0)    

Example Corresponding F of LP

Example Corresponding F of LP 2214   

Example Corresponding F of LP 2214    Choosing  =1 makes A 6 leaves B. 

Example Corresponding F of LP 224  Choosing  =1 makes A 6 leaves B. 01 11 33 11 00 33   = 0  = 1

Linear Programming Organization of a Tableau 大同大學資工所 智慧型多媒體研究室

Example Ansx1x1 x2x2 x3x3 x4x4 x5x x1x1 x2x2 x3x3 x4x4 x5x 11 2 1 R1 R2  R1 R3  R1 Elementary Row operations

Example Ansx1x1 x2x2 x3x3 x4x4 x5x 11 2 1 R1 R2  R1 R3  R1 B Suppose that we want to choose A 1 to enter the basis, i.e., choosing A 1 as the pivot column.    1/3 2/2 min

Example Ansx1x1 x2x2 x3x3 x4x4 x5x 11 2 1 R1 R2  R1 R3  R1 B    1/3 2/2 min

Example Ansx1x1 x2x2 x3x3 x4x4 x5x 11 2 1 /3 2/2 min Ansx1x1 x2x2 x3x3 x4x4 x5x5 1/3 4/3 10/ /3  7/3 11/3 1/3  2/3 1/