1 Introduction to Linear and Nonlinear Programming

2 Outline  Introduction - types of problems - types of problems - size of problems - size of problems - iterative algorithms and convergence - iterative algorithms and convergence  Basic properties of Linear Programs - introduction - introduction - examples of linear programming - examples of linear programming problems problems

3 Types of Problems  Three parts - linear programming - linear programming - unconstrained problems - unconstrained problems - constrained problems - constrained problems  The last two parts comprise the subject of nonlinear programming of nonlinear programming

4 Linear Programming  It is characterized by linear functions of the unknowns; the objective is linear in the unknowns; the objective is linear in the unknowns, and the constraints are the unknowns, and the constraints are linear equalities or linear inequalities. linear equalities or linear inequalities.  Why are linear forms for objectives and constraints so popular in problem formulation? constraints so popular in problem formulation? - a great number of constraints and objectives that arise in - a great number of constraints and objectives that arise in practice are indisputably linear (ex: budget constraint) practice are indisputably linear (ex: budget constraint) - they are often the least difficult to define - they are often the least difficult to define

5 Unconstrained Problems  Are unconstrained problems devoid of structural properties as to preclude their applicability as properties as to preclude their applicability as useful models of meaningful problems? useful models of meaningful problems? - if the scope of a problem is broadened to the consideration of - if the scope of a problem is broadened to the consideration of all relevant decision variables, there may then be no all relevant decision variables, there may then be no constraints constraints - many constrained problems are sometimes easily converted - many constrained problems are sometimes easily converted to unconstrained problems to unconstrained problems

6 Constrained Problems  Many complex problems cannot be directly treated in its entirety accounting for all treated in its entirety accounting for all possible choices, but instead must be decomposed possible choices, but instead must be decomposed into separate subproblems into separate subproblems  “ Continuous variable programming ”

7 Size of Problems  Three classes of problems - small scale: five or fewer unknowns and constraints - small scale: five or fewer unknowns and constraints - intermediate scale: from five to a hundred variables - intermediate scale: from five to a hundred variables - large scale: from a hundred to thousands variables - large scale: from a hundred to thousands variables

8 Iterative Algorithms and Convergence  Most algorithms designed to solve large optimization problems are iterative. optimization problems are iterative.  For LP problems, the generated sequence is of finite length, reaching the solution is of finite length, reaching the solution point exactly after a finite number of steps. point exactly after a finite number of steps.  For non-LP problems, the sequence generally does not ever exactly reach the solution point, but converges toward it. does not ever exactly reach the solution point, but converges toward it.

9 Iterative Algorithms and Convergence (cont ’ d)  Iterative algorithms 1. the creation of the algorithms themselves 1. the creation of the algorithms themselves 2. the verification that a given algorithm will in fact 2. the verification that a given algorithm will in fact generate a sequence that converges to a solution generate a sequence that converges to a solution 3. the rate at which the generated sequence of points 3. the rate at which the generated sequence of points converges to the solution converges to the solution  Convergence rate theory 1. the theory is, for the most part, extremely simple in nature 1. the theory is, for the most part, extremely simple in nature 2. a large class of seemingly distinct algorithms turns out to 2. a large class of seemingly distinct algorithms turns out to have a common convergence rate have a common convergence rate

10 LP Problems ’ Standard Form

11 LP Problems ’ Standard Form (cont ’ d)  Here x is an n-dimensional column vector, is an n-dimensional row vector, A is an is an n-dimensional row vector, A is an m×n matrix, and b is an m-dimensional m×n matrix, and b is an m-dimensional column vector. column vector.

12 Example 1 (Slack Variables)

13 Example 1 (Slack Variables) (cont ’ d)

14 Example 2 (Free Variables)  X 1 is free to take on either positive or negative values. negative values.  We then write X 1 =U 1 -V 1, where U 1 ≧ 0 and V 1 ≧ 0. and V 1 ≧ 0.  Substitute U 1 -V 1 for X 1, then the linearity of the constraints is preserved and all of the constraints is preserved and all variables are now required to be nonnegative. variables are now required to be nonnegative.

15 The Diet Problem  We assume that there are available at the market n different foods and that the i th food sells at a n different foods and that the i th food sells at a price per unit. In addition there are m basic price per unit. In addition there are m basic nutritional ingredients and, to achieve a balanced nutritional ingredients and, to achieve a balanced diet, each individual must receive at least units diet, each individual must receive at least units of the j th nutrient per day. Finally, we assume that each unit of food i contains units of the j th nutrient. of the j th nutrient per day. Finally, we assume that each unit of food i contains units of the j th nutrient.

16 The Diet Problem (cont ’ d)

17 The Transportation Problem Quantities a1, a2, …, a m, respectively, of a certain product are to be shipped from each of m locations and received in amounts b1, b2, …, b n, respectively, at each of n destinations. Associated with the Shipping of a unit of product from origin i to destination j is a unit shipping cost c ij. It is desired to determine the amounts x ij to be shipped between each origin-destination pair (i=1,2, …,m; j=1,2, …,n)

18 The Transportation Problem (cont ’ d)