1. Production systems LP and MILP models

2. Introduction  Linear programing (LP) is a base tool in optimization  George B. Dantzig developed the simplex method which can solve linear programing problems Tibor Holczinger2

3. Example (pizzeria)  Two types of pizzas: margareta, hawaii  All prepared pizza can be sold  The price of margareta is 600, the price of hawaii is 800  Maximize the income  There is no cost  Question: How many pizza must be prepare for each type?  Not whole pizzas can be prepared (and sold) Tibor Holczinger3

4. Example (pizzeria)  This problem definition yields to prepare only hawaii pizzas in infinite number  We need additional limitations  Limited ingredients  550 cheese, 150 ham and 120 pineapples per day  “Recipe” Tibor Holczinger4 margaretahawaii cheese105 ham24 pineapple03

5. Solution  Linear programing can solve this type of problems  Develop a model  Main topic of the lecture  Solve the model  No detailed algorithm  Using a free solver Tibor Holczinger5

6. Variables  Question: How many pizza must be prepare for each type?  Answer: Two values  Two variables for solution  Let x 1 the number of prepared margareta pizzas  Let x 2 the number of prepared hawaii pizzas  Only nonnegative number of pizzas can be prepared  The variables must be nonnegative  x i ≥ 0, i = 1, 2 Tibor Holczinger6

7. Objective function  A linear expression is needed for maximizing income max 600 x 1 + 800 x 2  This expression is called objective function  In case of minimization “max” must be replaced by “min”  The aim is to determine the value of variables to reach the maximal value of the objective function Tibor Holczinger7

8. Constraints  We have limitations on ingredients which can be expressed by linear inequalities 10 x 1 + 5 x 2 ≤ 550(cheese) 2 x 1 + 4 x 2 ≤ 150(ham) 3 x 2 ≤ 120(pineapple)  These inequalities are called constraints Tibor Holczinger8

9. Linear model  The model is linear because the objective function and the equalities are linear  Weighted sum of the variables  Between the left and the right side of a constraints there can be “≤”, “=“ or “≥”  “ ” is not allowed because not solvable  For example: Which is the smallest number which is bigger than 1? Tibor Holczinger9

10. Solving the model  Let x = (x 1, x 2 ) a vector  A solution of the model is a vector which satisfies the constraints  A feasible solution of the model is a solution where the types of the variables are valid  An optimal solution is a feasible solution where there is no feasible solution which value is bigger than the value of x in case of maximization  In case of minimization there is no feasible solution which has smaller value Tibor Holczinger10

11. Optimal  Optimal solution means the best solution  There is no “more optimal” or “most optimal” solution Tibor Holczinger11

12. LP  The form of problem which consist of linear constraints and linear objective function is called linear programing model or LP model  The development and optimization of the LP model is called linear programing or LP Tibor Holczinger12

13. Alternative solution  The definition of optimal solution allows to have multiple optimal solutions  They are called alternative solutions  The optimal solution get first is often not good in practice  The customer does not define every constraint  Alternative solutions can be useful in this case Tibor Holczinger13

14. Solution methods  Two main approaches  Simplex method  Easy to find alternative solutions  Interior point method  More suitable for bigger models Tibor Holczinger14

15. Simplex (pizzeria)  Represent the three inequalities in a coordinate system  The horizontal line represent x 1  The vertical line represent x 2  Denote the feasible region with grey  This region is called simplex  The optimal solution must be at a vertex or on a side of the simplex  The simplex method checks the vertices of the simplex till it finds an optimal solution Tibor Holczinger15

16. Simplex example Tibor Holczinger16

17. Interior point method  The interior point methods starts from inside of the simplex  If more optimal solution exist, the method gives back an interior point of the surface of optimal solutions Tibor Holczinger17

18. Software  There are libraries and complete software to solve LP models  The most known and effective solver is the IBM ILOG CLPEX software which is a non- free solution  The two most spread free solver are the Coin-OR and the glpk  They are less effective than the CPLEX but usually enough for most problems Tibor Holczinger18

19. glpk  The glpk (GNU Linear Programming Kit) is a open source, free software  http://www.gnu.org/software/glpk/  It can install on Linux or Windows too  The examples will be solved by glpk Tibor Holczinger19

20. Modeling language  To define an LP problem for a software a modeling language is needed  The first developed and most common language is the GAMS (General Algebraic Modeling System)  The glpk knows multiple modeling languages  We will use the GNU MathProg Tibor Holczinger20

21. Using glpk  Define the model with a word processor and save it in a file  Solve the model using glpk in command line Tibor Holczinger21

22. GNU MathProg  Every expression has to be closed by a semicolon  Multiple expressions are allowed in a line  The lower and upper case characters are distinguished  From a # character till the end of the line everything means comment Tibor Holczinger22

23. Example (pizzeria) var x1 >= 0; var x2 >= 0; s.t. cheese: 10*x1 + 5*x2 <= 550; s.t. ham: 2*x1 + 4*x2 <= 150; s.t. pineapple: 3*x2 <= 120; maximize income: 600*x1 + 800*x2; end; Tibor Holczinger23

24. Variables  Two variables ( x1 and x2 ) are defined with the var keyword  Optionally bounds can be defined with >=, = and <=  The >= 0 in the model means the lower bound of the variable is 0  The variables are continuous in default Tibor Holczinger24

25. Constraints  Syntax  s.t.  The name of the constraint  A colon  The constraint  It can contains brackets  It can be ordered to any constant or variable Tibor Holczinger25

26. Objective function  Syntax  maximize or minimize  The name of the objective function  A colon  The objective function Tibor Holczinger26

27. Closing the model  The model can be close by an end keyword  Optional  After the end everything is comment Tibor Holczinger27

28. Objects  In GNU MathProg we use objects  Variables  Constraints  Objective function  …  Every object has a unique name Tibor Holczinger28

29. Solving  If the model above is save into a pizza.txt file, it could be solved by the following command glpsol -m pizza.txt -o solution.txt  -m : model file  -o : output file  In case of mistyping the reading of the input stops an give an error message  If there is no error, the following message appears Tibor Holczinger29

30. Message GLPSOL: GLPK LP/MIP Solver 4.38 Reading model section from pizza.txt... 9 lines were read Generating cheese... Generating ham... Generating pineapple... Generating income... Model has been successfully generated glp_simplex: original LP has 4 rows, 2 columns, 7 non-zeros glp_simplex: presolved LP has 2 rows, 2 columns, 4 non-zeros Scaling... A: min|aij| = 2.000e+00 max|aij| = 1.000e+01 ratio = 5.000e+00 Problem data seem to be well scaled Crashing... Size of triangular part = 2 *0: obj = 0.000000000e+00 infeas = 0.000e+00 (0) *2: obj = 3.966666667e+04 infeas = 0.000e+00 (0) OPTIMAL SOLUTION FOUND Time used: 0.0 secs Memory used: 0.1 Mb (114490 bytes) Writing basic solution to ‘solution.txt’... Tibor Holczinger30

31. Solution  The most important line indicates the program found the optimal solution  OPTIMAL SOLUTION FOUND  If the model has no solution other message appears  PROBLEM HAS NO FEASIBLE SOLUTION  In this case there is contradiction among constraints Tibor Holczinger31

32. Solution.txt (first part) Problem: pizza Rows: 4 Columns: 2 Non-zeros: 7 Status: OPTIMAL Objective: income = 39666.66667 (MAXimum) No. Row name St Activity Lower bound Upper bound Marginal ------ ------------ -- ------------- ------------- ------------- ------------- 1 cheese NU 550 550 26.6667 2 ham NU 150 150 166.667 3 pineapple B 40 120 4 income B 39666.7 No. Column name St Activity Lower bound Upper bound Marginal ------ ------------ -- ------------- ------------- ------------- ------------- 1 x1 B 48.3333 0 2 x2 B 13.3333 0 Tibor Holczinger32

33. Interpretation of output  After Objective there is the value of the optimal solution  In the last two lines there are the values of the variables x1 and x2  Number of pizzas to prepare  In optimal solution 48⅓ pieces margareta and 13⅓ pieces hawaii yields 39666.67 income per day  Later we discuss problem when only whole pizzas can be prepared and sold Tibor Holczinger33

34. Model and data  In previous example all variables and constraints are defined one by one  Not usable for bigger problems  For example much more pizzas and ingredients  Increasing size of problem does not effects the structure of the model  Number of variables, constraints and coefficients change Tibor Holczinger34

35. Model and data  Model and data can be separated  Very useful when data change but model does not  Using two files  Structure of the model  Data  Additional objects are needed Tibor Holczinger35

36. Example  Identification of the objects  Pizzas  Ingredients  Two sets to store Tibor Holczinger36

37. Sets  The data and the model file contains the same sets  Definition in the model file  Elements in the data file  Enumerating elements  Does not need braces and comas  The elements of a set can be strings, numbers Tibor Holczinger37

38. Example  Definition (model file) set Pizzas; set Ingredients;  Elements (data file) set Pizzas := margareta hawaii; set Ingredients := cheese ham pineapple; Tibor Holczinger38

39. Parameters  Parameters describe objects  Pizzas has price  Limit for ingredients  Amount on ingredients for (pizza, raw material) pairs  Each parameters belongs to an element of a set  Defining a new parameter contains to which set is belonged Tibor Holczinger39

40. Example param Price{p in Pizzas}; param Amount{p in Pizzas, i in Ingredients}; param IngMax{i in Ingredients}; Tibor Holczinger40

41. Parameters  The parameter object Price assigns a parameter to each element of Pizzas  The parameter object Amount assigns a parameter to each ( p, i ) pair where p  Pizzas and i  Ingredients  The values of parameters must be defined in data file Tibor Holczinger41

42. Example param Price := margareta 600 hawaii 800; param IngMax := cheese 550 ham 150 pineapple 120; param Amount : cheese ham pineapple := margareta 10 2 0 hawaii 5 4 3; Tibor Holczinger42

43. Parameters for sets  Price and IngMax in the example  Syntax  param  Name of the parameter object  :=  Enumeration of parameters  List of the elements of set together with their values  It worth to write each element in a new row  A semicolon Tibor Holczinger43

44. Parameters for pairs  Amount in the example  Syntax  param  Name of the parameter object  A colon  List of the elements of the second set  To define the order of parameters  :=  Enumeration of parameters  List of the elements of the first set followed by values for each elements of the second set  It worth to write each element in a new row  A semicolon Tibor Holczinger44

45. Variables  Definition of variables  A variable belongs to each pizza in the model  A variable belongs to each element of the Pizzas var x{Pizzas} >= 0;  Usage of variables  The name of a pizza in brackets after the name of the variable  For example: x[margareta] Tibor Holczinger45

46. Objective function  The income of the problem is the number of prepared pizzas multiplied by the price of the pizzas maximize income: sum{p in Pizzas} x[p] * Price[p];  Interpretation of sum  Substituting all elements of set in braces into temporary variable i then evaluating the other part  In this case the value of i is margareta first, then hawaii Tibor Holczinger46

47. Constraints  Can be defined similarly s.t. ingredient{i in Ingredients}: sum{p in Pizzas} x[p] * Amount[p, i] <= IngMax[i];  With the help of expression ingredient{i in Ingredients} a constraint is defined for all elements of Ingredients Tibor Holczinger47

48. Closing  The model and the data file can be closed by the end keyword Tibor Holczinger48

49. Model file set Pizzas; set Ingredients; param Price{p in Pizzas}; param Amount{p in Pizzas, i in Ingredients}; param IngMax{i in Ingredients}; var x{Pizzas} >= 0; s.t. ingredient{i in Ingredients}: sum{p in Pizzas} x[p] * Amount[p, i] <= IngMax[hi]; maximize income: sum{p in Pizzas} x[p] * Price[p]; end; Tibor Holczinger49

50. Data file set Pizzas := margareta hawaii; set Ingredients := cheese ham pineapple; param Price := margareta 600 hawaii 800; param Amount : cheese ham pineapple:= margareta 10 2 0 hawaii 5 4 3; param IngMax := cheese 550 ham 150 pineapple 120; end; Tibor Holczinger50

51. Solving  The modification of data yields to modification of data file only  For example modification of price of a pizza or introducing a new pizza  Command line glpsol -m pizza_model.txt -d pizza_data.txt -o solution.txt  -m : model file  -d : data file  -o : output file Tibor Holczinger51

52. solution.txt Problem: pizza_model Rows: 4 Columns: 2 Non-zeros: 7 Status: OPTIMAL Objective: income = 39666.66667 (MAXimum) No. Row name St Activity Lower bound Upper bound Marginal ------ ------------------ -- ----------- ------------ ------------ ------------ 1 ingredient[cheese] NU 550 550 26.6667 2 ingredient[ham] NU 150 150 166.667 3 ingredient[pineapple] B 40 120 4 income B 39666.7 No. Column name St Activity Lower bound Upper bound Marginal ------ ------------ -- ------------- ------------- ------------- ------------- 1 x[margareta] B 48.3333 0 2 x[hawaii] B 13.3333 0 Tibor Holczinger52

53. Integer variables  Not whole pizza was allowed  It is not a real life problem  Only whole pizzas are allowed  The decisions variables can have only integer values  This restriction makes the problem more complicated  Software need much more time Tibor Holczinger53

54. Integer programming  If a linear model contains only integer variables, it is called integer programming (IP) model  If some variables are integer and some of them are continuous, it is called mixed integer linear programming (MILP) model  Among integer variables there are binary variables  They can have only two values 0 or 1 Tibor Holczinger54

55. Size of the model  Usually getting more integer variables needs more time to solve  Adding constraint to a model  Solving LP model takes more time  Solving MILP model can take less time  Smaller search space yields getting solution faster  Adding variable to a model  Solving LP model takes slightly more time  Solving MILP model can takes multiple of solution time Tibor Holczinger55

56. Solution method  Widely used branch and bound method  It uses simplex method as subroutine  In a relaxed model the integer variables can have non-integer values  The relaxed model is an LP model  The optimal solution of the relaxed model  Better than the original’s one  In maximization it provides an upper bound  In minimization it provides a lower bound  Not necessary a feasible solution of the original model Tibor Holczinger56

57. Solution procedure 1.  Solving the pizzeria example allowing only whole pizzas  Integer variables  Solution of the relaxed model  Objective value = 39666.667  x[margareta] = 48.3333  x[hawaii] = 13.3333  The solution is not integer  Further calculation is needed Tibor Holczinger57

58. Solution procedure 2.  Choosing a variable which is not integer in the solution  Let x[hawaii], which value was 13.3333  Do not let the variable get the value 13.3333  Branch the problem along x[hawaii]  Make two new relaxed models with constraints  x[hawaii] ≤ 13  x[hawaii] ≥ 14  Solve the two new models Tibor Holczinger58

59. Branching 1. Tibor Holczinger59

60. Evaluating solutions  In both cases the objective values are decreased  In case of x[hawaii] ≥ 14 the values of variables are integers, i.e. it is a feasible solution  In case of x[hawaii] ≤ 13 the objective value is better than the other  Further calculation can lead better solution than 39400 Tibor Holczinger60

61. Pruning  Let max denote the current best solution  The initial value is infinite in case of maximization  Update max in case of reaching new, better solution  If the solution of a partial problem is not better then max, it can be discarded  Cannot reach better solution than max  This step is called pruning  In the example the max is equal 39400 after the first branching Tibor Holczinger61

62. Branching 2  Branch the partial problem on the left side along x[margareta] Tibor Holczinger62

63. Evaluating solutions  In both cases the solution is worse than max  They can be pruned  There is no more partial problem where the solution is no integer  The algorithm ends  The optimal solution  Prepare 47 margareta and 14 hawaii pizzas  The income is 39400 Tibor Holczinger63

64. Branch and bound  The described method is called branch and bound (B&B) method  It can be seen why more variables leads to more solution time  Increasing the number of integer variables  Increasing the number of forks in the tree  Increasing the number of partial problems exponentially Tibor Holczinger64

65. Example  The GNU MathProg modeling language allows integer variables  Using of keyword integer var x{Pizzak} >= 0, integer; Tibor Holczinger65

66. Solution Solving LP relaxation... * 0: obj = 0.000000000e+00 infeas = 0.000e+00 (0) * 3: obj = 3.966666667e+04 infeas = 0.000e+00 (0) OPTIMAL SOLUTION FOUND Integer optimization begins... + 3: mip = not found yet <= +inf (1; 0) + 4: >>>>> 3.940000000e+04 <= 3.940000000e+04 0.0% (3; 0) + 4: mip = 3.940000000e+04 <= tree is empty 0.0% (0; 5) INTEGER OPTIMAL SOLUTION FOUND Tibor Holczinger66

67. Solution procedure  The solver solves the relaxed model first  After the line Integer optimization begins...  The remaining % of search space  The number of open partial problems  The number of branched partial problems  In real life problems the list can be much longer and the solution takes much more time  In this case the solver write periodically the elapsed time and the size of used memory Tibor Holczinger67