# Introduction to Algorithms Linear Programming My T. UF.

## Presentation on theme: "Introduction to Algorithms Linear Programming My T. UF."— Presentation transcript:

1 Introduction to Algorithms Linear Programming My T. Thai @ UF

2 New crop problem  A farmer:  Has 10 acres to plant in wheat and rye  Has Only $1200 to spend  Has to plant at least 7 acres  Has to get the planting done in 12 hours  Each acre of wheat costs$200 and takes an hour to plant and gives $500 profit  Each acre of rye costs$100 and takes 2 hours to plant and gives $300 profit  Goal: Compute the numbers of acres of each should be planted to maximize profits My T. Thai mythai@cise.ufl.edu 2 x: # of wheat acres y: # of rye acres 3 Linear program  A linear programming problem is the problem of maximizing or minimizing a linear function subject to linear constraints (equalities and inequalities)  E.g. My T. Thai mythai@cise.ufl.edu 3 Constraints Objective function 4 Formulate shortest path as an LP  Triangle inequalities:  At the source vertex: d s = 0  d t is less than weights of all paths from s to t => d t is the maximum value that less than weights of all paths from s to t  We have the following: My T. Thai mythai@cise.ufl.edu 4 5 My T. Thai mythai@cise.ufl.edu 5 Max Flow  Capacity constraints:  Conservation constraints:  We have the following: 6 Minimum-cost flow  Pay cost to transmit flow f uv through edge (u, v)  Wish to send d units with the minimized cost My T. Thai mythai@cise.ufl.edu 6 7 Multicommodity flow  Has k commodities; i th commodity is defined by a triple  Total flow of all commodities through an edge does not exceed its capacity  Find a feasible flow My T. Thai mythai@cise.ufl.edu 7 8 Some Notations  Minimization (maximization) linear program: the value of objective function is minimized (maximized)  Feasible (infeasible) solution: a setting of variables satisfies all the constraints (conflicts at least one constraint)  Feasible region: set of feasible solutions  Objective value: value of the objective function at a particular point  Optimal solution: objective value is maximum over all feasible solutions  Optimal objective value: objective value of optimal solution  The linear program is infeasible if it has no feasible solutions; otherwise it is feasible  The linear program is unbounded if the optimal objective value is infinite My T. Thai mythai@cise.ufl.edu 8 9 Standard form My T. Thai mythai@cise.ufl.edu 9 Concise representation: 10 Convert into standard form My T. Thai mythai@cise.ufl.edu 10 11 Example My T. Thai mythai@cise.ufl.edu 11 (negate objective function) (replace x 2 ) (replace equality) (negate constrain and change variable name) 12 Geometry of Linear Programming Theorem 1 Feasible region of an LP is convex Proof: Feasible region is the intersection of half spaces defined by the constraints. Each half space is convex. My T. Thai mythai@cise.ufl.edu 12 13 Example My T. Thai mythai@cise.ufl.edu 13 14 Feasibility and Infeasibility  Simple solution: try all vertices of polyhedron  running time:  How to refine this approach? My T. Thai mythai@cise.ufl.edu 14 15 Simplex method  Start off from a vertex, which is called a basic feasible solution  Iteratively move along an edge of the polyhedron to another vertex toward the direction of optimization  For each move, need to make sure that the objective function is not decreased  Observation: when moving from a vertex to another vertex, an inequality achieves equality My T. Thai mythai@cise.ufl.edu 15 16 Questions Arise My T. Thai mythai@cise.ufl.edu 16 17 Slack form  All inequality constraints are non-negativity constraints  Convert by introducing slack variables My T. Thai mythai@cise.ufl.edu 17 Basic variables nonbasic variables 18 Concise representation of slack form  z: the value of the objective function  N: the set of indices of the nonbasic variables (|N| = n)  B: the set of indices of the basic variables (|B| = m)  v: an optional constant term in the objective function  A tuple (N, B, A, b, c, v) represents the slack form My T. Thai mythai@cise.ufl.edu 18 maximize 19 Simplex algorithm  Basic solution: all nonbasic variables equals 0  Each iteration converts one slack form into an equivalent slack form s.t. the objective value is not decreased  Choose a nonbasic variable x e (entering variable) such that its increase makes the objective value increase  Keeping all constraint satisfied, raise the variable raise it until some basic variable x l (leaving variable) becomes 0  Exchange the roles of that basic variable and the chosen nonbasic variable My T. Thai mythai@cise.ufl.edu 19 20 Example My T. Thai mythai@cise.ufl.edu 20 21 My T. Thai mythai@cise.ufl.edu 21 Substitute x_2 = x_3 = 0 to the slack variables, we have: 22 My T. Thai mythai@cise.ufl.edu 22 23 Cycling  SIMPLEX may run forever if the slack forms at two different iterations of SIMPLEX are identical (cycling phenomenon)  We need specific rule of picking the entering and leaving variables  There are quite a few methods to prevent cycling. The one we just used is called Bland’s pivoting rule My T. Thai mythai@cise.ufl.edu 23 24 Other Pivoting Rules My T. Thai mythai@cise.ufl.edu 24 25 Time Complexity My T. Thai mythai@cise.ufl.edu 25 26 My T. Thai mythai@cise.ufl.edu 26 Duality  Given a primal problem: P: min c T x subject to Ax ≥ b, x ≥ 0  The dual is: D: max b T y subject to A T y ≤ c, y ≥ 0 27 My T. Thai mythai@cise.ufl.edu 27 An Example 28 My T. Thai mythai@cise.ufl.edu 28 Weak Duality Theorem  Weak duality Theorem: Let x and y be the feasible solutions for P and D respectively, then:  Proof: Follows immediately from the constraints 29 My T. Thai mythai@cise.ufl.edu 29 Weak Duality Theorem  This theorem is very useful  Suppose there is a feasible solution y to D. Then any feasible solution of P has value lower bounded by b T y. This means that if P has a feasible solution, then it has an optimal solution  Reversing argument is also true  Therefore, if both P and D have feasible solutions, then both must have an optimal solution. 30 My T. Thai mythai@cise.ufl.edu 30 Hidden Message ≥ Strong Duality Theorem: If the primal P has an optimal solution x* then the dual D has an optimal solution y* such that: c T x* = b T y* 31 My T. Thai mythai@cise.ufl.edu 31 Complementary Slackness  Theorem: Let x and y be primal and dual feasible solutions respectively. Then x and y are both optimal iff two of the following conditions are satisfied: (A T y – c) j x j = 0 for all j = 1…n (Ax – b) i y i = 0 for all i = 1…m 32 My T. Thai mythai@cise.ufl.edu 32 Proof of Complementary Slackness Proof: As in the proof of the weak duality theorem, we have: c T x ≥(A T y) T x = y T Ax ≥ y T b (1) From the strong duality theorem, we have: (2) (3) 33 My T. Thai mythai@cise.ufl.edu 33 Proof (cont) Note that and We have: x and y optimal  (2) and (3) hold  both sums (4) and (5) are zero  all terms in both sums are zero (?)  Complementary slackness holds (4) (5) 34 My T. Thai mythai@cise.ufl.edu 34 Why do we care?  It’s an easy way to check whether a pair of primal/dual feasible solutions are optimal  Given one optimal solution, complementary slackness makes it easy to find the optimal solution of the dual problem  May provide a simpler way to solve the primal 35 My T. Thai mythai@cise.ufl.edu 35 Some examples  Solve this system: 36 My T. Thai mythai@cise.ufl.edu 36 Min-Max Relations  What is a role of LP-duality  Max-flow and Min-Cut 37 My T. Thai mythai@cise.ufl.edu 37 Max Flow in a Network  Definition: Given a directed graph G=(V,E) with two distinguished nodes, source s and sink t, a positive capacity function c: E → R+, find the maximum amount of flow that can be sent from s to t, subject to: 1.Capacity constraint: for each arc (i,j), the flow sent through (i,j), f ij bounded by its capacity c ij 2.Flow conservation: at each node i, other than s and t, the total flow into i should equal to the total flow out of i 38 My T. Thai mythai@cise.ufl.edu 38 An Example s t 4 3 4 3 2 3 2 3 2 3 1 5 2 4 2 3 4 1 1 3 2 0 0 1 4 4 3 1 2 0 0 0 39 My T. Thai mythai@cise.ufl.edu 39 Formulate Max Flow as an LP  Capacity constraints: 0 ≤ f ij ≤ c ij for all (i,j)  Conservation constraints:  We have the following: 40 My T. Thai mythai@cise.ufl.edu 40 LP Formulation (cont) s t 4 3 4 3 2 3 2 3 2 3 1 5 2 4 2 3 4 1 1 3 2 0 0 1 4 4 3 1 2 0 0 0 ∞ 41 My T. Thai mythai@cise.ufl.edu 41 LP Formulation (cont) 42 My T. Thai mythai@cise.ufl.edu 42 Min Cut Capacity of any s-t cut is an upper bound on any feasible flow If the capacity of an s-t cut is equal to the value of a maximum flow, then that cut is a minimum cut 43 My T. Thai mythai@cise.ufl.edu 43 Max Flow and Min Cut 44 My T. Thai mythai@cise.ufl.edu 44 Solutions of IP Consider: Let (d*,p*) be the optimal solution to this IP. Then:  p s * = 1 and p t * = 0. So define X = {p i | p i = 1} and$\bar X$= {p i | p i = 0}. Then we can find the s-t cut  d ij * =1. So for i in X and j in$\bar X\$, define d ij = 1, otherwise d ij = 0.  Then the object function is equal to the minimum s-t cut

Similar presentations