1. 1

2. 2 3.4 Pareto Linear Programming The Problem: P-opt Cx s.t Ax ≤ b x ≥ 0 where C is a kxn matrix so that Cx = (c (1) x, c (2) x,..., c (k) x) where c (j) = jth row of C.

3. 3 Example P- max {3x 1 + 2x 2, 2x 1 – x 2 } S.t. x 1 ≤ 2 3x 1 + x 2 ≤ 9 x 1, x 2 ≥ 0 Here z = (z 1, z 2 ). z1z1 z2z2

4. 4 We refer to the Pareto solutions also as efficient points, or non-dominated points. We shall focus on opt = max. For simplicity we shall assume that the set of feasible solution, namely X := {x in R n : Ax ≤ b, x ≥ 0} is bounded. Let Z:= {Cx: x in X} observing that Z is a subset of R k, and usually k <<<<<n.

5. 5

6. 6 We can solve for the efficient points graphically if we only have 2 variables and 2 objective functions. This also gives us a good picture of what is going on, and ideas for generalizing. P- max {3x 1 + 2x 2, 2x 1 – x 2 } S.t. x 1 ≤ 2 3x 1 + x 2 ≤ 9 x 1, x 2 ≥ 0

7. 7 So more generally: Geometry Ax ≤ b x ≥ 0 X  R n Z  R k Cx x  X

8. 8 P- max {3x 1 + 2x 2, 2x 1 – x 2 } S.t. x 1 ≤ 2 3x 1 + x 2 ≤ 9 x 1, x 2 ≥ 0 Find X and Z for the problem below.

9. 9 How do we generate all the Pareto Solutions? Idea: Generate the non-dominated extreme points of Z Use them to generate the other efficient points of Z. Rationale: Other efficient points of Z are convex combinations of the efficient extreme points of Z

10. 10 k=2 (i.e 2-objectives) z2z2 z1z1

11. 11 k=2 z2z2 z1z1 Z

12. 12 k=2 z2z2 z1z1 Z Efficient points (Pareto solutions)

13. 13 k=2 z2z2 z1z1 Z Efficient points Efficient extreme points Every efficient point can be expressed as a convex combination of the efficient extreme points. So we first aim to generate the efficient extreme points.

14. 14 An extremely important theorem Idea: Theorem 4.6.1 An extreme point of Z must be produced by an extreme point of X, ie if z* is an extreme point of Z then z*=Cx* where x* is an extreme point of X Proof: By contradiction. Assume that z* is an extreme point of Z but x* in X for which z*=Cx*, is not an extreme point of X.

15. 15 Geometry Ax ≤ b x ≥ 0 X  R n Z  R k Cx x  X x* z*

16. 16 Since x* is not an extreme point of X we have x* =  x’ + (1-  )x”, 0 <  < 1 for some points x’ and x” in X. Thus, z* = Cx* = C(  x’ + (1-  )x”) =  Cx’ + (1-  )Cx” =  z’ + (1-  )z”, (z’=Cx’, z”=Cx”) This, however, contradicts the assertion that z* is an extreme point of Z. (This is true only when z’ and z” are different and neither of z’ and z” is an extreme point)

17. 17 Comment Ax ≤ b x ≥ 0 X  R n Z  R k Cx x  X x* z* If z* extreme then x* extreme. BUT not necessarily vice versa.

18. 18

19. 19 One way to generate the efficient extreme points of Z is by deploying the following well known results: Theorem 4.6.2: If z* is an efficient point of Z then there must be a in R k such that z* is an optimal solution to the problem: max { z: z in Z} (****) i.e. max {  z 1 +  z 2 +…+  z k : z in Z} Furthermore, > 0 (all components are strictly positive)

20. 20 Theorem 4.6.3: If  > 0 then any optimal solution to max {  z 1 +  z 2 +…+  z k : z in Z} (****) is an efficient point of Z.

21. 21 Proof of Theorem 4.6.3: Assume that > 0 and let z* be any optimal solution to (****). Contrary to the Theorem, assume that z* is not an efficient point of Z. This means that there exists some z’ in Z such that z’ j ≥ z* j, for all j=1,2,...,k and z’ p > z* p, for some 1 ≤ p ≤ k. (Remember that we mean Pareto-max here) Since >0, this implies that z’ > z*, contradicting the assertion that z* is an optimal solution to (****).

22. 22 Understanding Theorem 4.6.2 requires some fundamental results!!! We will omit slides 17 - 40 (not examinable). From slide 41 on is examinable.

23. 23 Review (618-261) Convex set: If y’ and y” are elements of Y then the entire line segment connecting these points is also in Y. Line segment: Line segment connecting y’ and y” is the set of all the convex combinations of y’ and y”. Convex combinations: y =  y’ + (1-  )y”, 0 ≤  ≤ 1.

24. 24

25. 25 X1 X2 X1 X2 X1 X2 Convex comb. of X1 and X2 Not convex comb. of X1 and X2 Extreme Points A point y in Y is an extreme point of Y if it cannot be expressed as a convex combination of two other points in Y.

26. 26 2 extreme points 4 extreme points (the corners) Entire boundary is extreme

27. 27 Properties of Convex Sets If S is a convex set, then for any  in R +,  S := {  s: s in S} is a convex set. If S and T are convex sets, then so is S + T : = {s+t: s in S, t in T} The intersection of any collection of convex sets is convex.

28. 28 origin S SS S T S+T A B

29. 29 facts about hyperplanes Fact 1: Let be a non-zero element of R n and let b be a real number. Then the set H := {x in R n : x = b} is a hyperplane in R n. Example: In R 3, the set of all points satisfying 3x 1 + 3x 2 + x 3 =5 is a hyperplane (set b=5 and = (3,3,1) )

30. 30 Fact 2: Let H be a hyperplane in R n. Then there is a non-zero vector in R n and a constant b such that H = S:= {x in R n : x=b}.

31. 31 bottom line A hyperplane in R n is the set of solutions to a single linear equation.

32. 32 Half Spaces Given a hyperplane, say H := {x in R n : x = b} we shall consider the two closed half spaces it generates: H + := {x in R n : x ≥ b} H – := {x in R n : x ≤ b}

33. 33 H+H+ H–H– H := {x in R n : x=b}

34. 34 Main Results (for our purposes) Theorem 4.6.4 Separating Hyperplanes Given a convex set S and a point x exterior to its closure*, there is a hyperplane containing x that contains S in one of its half spaces. Closure of S: the smallest closed set containing S. Closed set: A set with the property that any point that is arbitrarily close to it is a member of the set.

35. 35 S H x

36. 36 S H x x S

37. 37 Theorem 4.6.5 Supporting Hyperplane Let S be a convex set and let x be a boundary point of S. Then there is a hyperplane containing x and containing S in one of its closed half spaces. S Supporting hyperplane at x x

38. 38 Theorem 4.6.6 Let S be a convex set, H a supporting hyperplane of S and I the intersection of H and S. Then every extreme point of I is an extreme point of S.

39. 39 One common extreme point S H I S H I two extreme points in common Face Facet

40. 40 And so...... For every extreme point in Z there is a supporting hyperplane Each extreme point of Z is an optimal solution to max { z : z in Z} for some non zero in R k.

41. 41 And so...... For each efficient extreme point of Z there is a strictly positive in R k such that the point is an optimal solution to max { z : z in Z}. Each extreme point of Z is an optimal solution to max { z : z in Z} for some non zero in R k. And also 1 +  + k = 1 (See supplementary notes for details).

42. 42

43. 43 Bottom line We can generate the efficient extreme points associated with P-opt Cx Ax ≤ b x ≥ 0 by solving max Cx Ax ≤ b x≥ 0 for all .

44. 44 Example

45. 45 k=2 Thus we need two multipliers 1 and 2. The objective function of the parametric linear programming problem will therefore be of the form: z( ) := 1 c (1) x + 2 c (2) x But since the parameters are positive, we can divide say by 1, so obtain an equivalent objective function of the form z(  ) := c (1) x +  c (2) x,  = 2 / 1 The parametric problem is thus:

46. 46 Note that  varies from 0 to infinity but cannot equal 0.

47. 47 z2z2 z1z1 Z Efficient points Efficient extreme points

48. 48 z2z2 z1z1 Z

49. 49 z2z2 z1z1 Z z 1 +  z 2 = Constant

50. 50 z2z2 z1z1 Z z 1 +  z 2 = Constant

51. 51 z2z2 z1z1 Z Slope = –1/ 

52. 52 z2z2 z1z1 Z Slope = 0

53. 53 z2z2 z1z1 Z

54. 54 z2z2 z1z1 Z

55. 55 z2z2 z1z1 Z

56. 56 z2z2 z1z1 Z

57. 57 z2z2 z1z1 Z

58. 58 z2z2 z1z1 Z

59. 59 z2z2 z1z1 Z Slope = - M If M big  is close to 0.

60. 60 Comments Read 618-261 Lecture Notes (Chapter 8) regarding changes in the objective function. In particular, changes to the cost coefficient of a basic variable (Why?) Check your result: We know something about the optimal values of the objective function as  changes.

61. 61 Parametric Linear Programming (Objective function) Set up: z*(  ) := opt z(  ):=c(  )x, 0 <  <  s.t. Ax ≤ b x ≥ 0 We want to generate the optimal solution x* as a function of the parameter . Symbolically we write it x*(  ).

62. 62 Procedure Step 1: Initialization Set  =0 and solve the resulting linear programming problem. This yields x*(0) and an “optimal” simplex tableau. Step 2: Range analysis Determine the largest value of  for which the current optimal solution remains optimal, say  *. Step 3: Stopping rule If  =  stop. Step 4: Iteration Construct the new optimal solution for  * and go to Step 2.

63. 63 Details Step 2: Range analysis This is done in the usual manner (618- 261 chapter 8) using the optimality test for the reduced costs. Step 4: Iteration This involves the usual pivot operation which produces an adjacent extreme point (exchange of one basic and one non-basic variable).

64. 64 Example Find all the Pareto extreme points of the following problem:

65. 65 To do this we solve z*(  ):  max{(3  2  )x 1  (5  )x 2 s.t. x 1  4 2x 2  12 3x 1  2x 2  18 x 1,x 2  0 }, for all  >0. i.e. (Hillier and Lieberman p. 308)

66. 66 Step 1: initialization For  =0 the objective function is z(0)=3x 1 +5x 2. We solve the problem for this objective function in the usual (simplex) manner.

67. 67 Final Tableau  =0 x 1 x 2 x 3 x 4 x 5 RHS x 3 0 0 1 1/3 -1/3 2 x 2 0 1 0 0.5 0 6 x 1 1 0 0 -1/3 1/3 2 Z 0 0 0 1.5 1 36 X 1 = 2, X 2 = 6 x*(0) = (2,6) z*(0) = 36

68. 68 Step 2: Range analysis We now increase  from zero until a new (adjacent) optimal solution is generated. We determine the critical value of , call it  * by introducing  in the z-row of the final tableau. This typically involves solving a system of simple (range) inequalities (requiring the reduced costs to be non-negative).

69. 69 z(  ) = (3+2  )x 1 + (5-  )x 2. Thus, we have to add -2  to the reduced costs of x 1 and +  to the reduced cost of x 2. This will destroy the canonical form of the tableau, so we shall fix it by pivoting. x 1 x 2 x 3 x 4 x 5 RHS x 3 0 0 1 1/3 -1/3 2 x 2 0 1 0 0.5 0 6 x 1 1 0 0 -1/3 1/3 2 Z 0 0 0 1.5 1 36  = 0

70. 70 To restore the canonical form we have to fix the z-row (2 pivot operations) x 1 x 2 x 3 x 4 x 5 RHS x 3 0 0 1 1/3 -1/3 2 x 2 0 1 0 0.5 0 6 x 1 1 0 0 -1/3 1/3 2 Z -2   0 1.5 1 36

71. 71 This tableau is optimal if 1.5 - 7  /6 ≥ 0 (and 1+ 2  /3≥0) I.e if 0 ≤  ≤ 9/7 yielding  *=9/7. X*(  (2,6), z*(  36-2  0 ≤  ≤ 9/7. x 1 x 2 x 3 x 4 x 5 RHS x 3 0 0 1 1/3 -1/3 2 x 2 0 1 0 0.5 0 6 x 1 1 0 0 -1/3 1/3 2 Z 0  0 (1.5-7  /6) (1+ 2  /3) 36-2 

72. 72 Step 4: Iteration The critical value of  is generated by x 4 so we select x 4 as the new basic variable. The ratio test indicates that we have to take x 3 out of the basis. We thus pivot on (i=1,j=4). x 1 x 2 x 3 x 4 x 5 RHS x 3 0 0 1 1/3 -1/3 2 x 2 0 1 0 0.5 0 6 x 1 1 0 0 -1/3 1/3 2 Z 0  0 (1.5-7  /6) (1+ 2  /3) 36-2 

73. 73 So now we repeat the iteration in the context of the new basic solution. x 1 x 2 x 3 x 4 x 5 RHS x 4 0 0 3 1 -1 6 x 2 0 1 -3/2 0 0.5 3 x 1 1 0 0 0 0 4 Z 0 0 (-9/2+7  /2) 0 (2.5-  /2) (27+5  This is optimal if -9/2+7  /2 ≥ 0 and 2.5-  /2≥ 0 i.e. 9/7≤  ≤5, x*(  = (4,3), z* *(  = (27 + 5   * = 5.

74. 74 Step 4: Iteration the variable yielding the critical value of  is x 5 thus we enter x 5 into the basis. the ratio test indicates that we take x 2 out of the basis. Thus, we pivot on (i=2,j=5). x 1 x 2 x 3 x 4 x 5 RHS x 4 0 0 3 1 -1 6 x 2 0 1 -3/2 0 0.5 3 x 1 1 0 0 0 0 4 Z 0 0 (-9/2+7  /2) 0 (2.5-  /2) (27+5 

75. 75 This is optimal if  ≥ 5. X* (  =(4,0), z* (  =12 +8  *=  Stop. The extreme points are x = (2,6), (4,3), (4,0) and corresponding efficient extreme points are (36, -2), (27, 5), (12, 8). x 1 x 2 x 3 x 4 x 5 RHS x 4 0 2 0 1 0 2 x 5 0 2 -3 0 1 6 x 1 1 0 1 0 0 4 Z 0 -5+  3+2  0 0 12+8 

76. 76 Summary  0 z*(  ) 36 9/7 33 3/7 5 52 36-2  x*=(2,6) 27+5  x*=(4,3) 12+8  x*=(4,0)

77. 77 Geometry

78. 78 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 Feasible set

79. 79 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 z(  ) = (3+2  )x 1 + (5  )x 2 x 2 = [z(  )/(5-  ] - [(3+2  )/(5-  x 1

80. 80 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 z(  ) = (3+2  )x 1 + (5  )x 2 x 2 = [z(  )/(5-  ] - [(3+2  )/(5-  x 1

81. 81 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 z(  ) = 3x 1 + 5x 2 x 2 = (z(  )/5) - (3/5)x 1

82. 82 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 z(  ) = 7x 1 + 3x 2 x 2 = [z(  )/3] - (7/3)x 1

83. 83 x2x2 x1x1 1 2 3 4 5 12345 678 9 6 7 8 9 z(  ) = 13x 1

84. 84 Some Results Consider the parametric problem: L(  ) := max {f(x) +  g(x): x in X},  in R Theorem: The set of  ’s for which the same x in X is optimal is convex. Theorem: Then, L(  ) is convex with . Furthermore, if there are finitely many optimal solutions, then, L(  ) is piecewise linear.

85. 85 for  in R, X is discrete L(  ) := max {f(x) +  g(x): x in X},  in R L(  )  The same x is optimal for all  in this range