§1.4 Algorithms and complexity For a given (optimization) problem, Questions: 1)how hard is the problem. 2)does there exist an efficient solution algorithm?  Computational complexity analysis (a worst case analysis)

§ Problems, Algorithm and Complexity Problem: a general question to be answered, usually possessing several unspecified parameters. Example: A Linear programming problem subject to A problem is described by giving (i)a general description of its parameters (ii)A statement of what properties the answer should satisfy

Example Traveling salesman problem Parameters: set of cities distance solution: an ordered sequence that minimizes

Measuring Algorithmic Efficiency Computation Time as the measure  Time Complexity Function for an algorithm: express its time requirements by giving the largest amount of time needed by the algorithm to solve a problem instance.  a function of the “size” of the instance Example size indices of a TSP # of cities # of interconnections

§ Polynomial Time Algorithms and Intractable Problems – A time complexity function is if for all n ≥ 0.  Algorithm An instance of a problem: a problem with all parameters specified E xample Polynomial time algorithm: Time complexity fcn. for some polynomial function p( ･ ). Exponential time algorithm That cannot be bounded by (may not be exponential, e.g

Algorithms: General, step-by-step procedures for solving problems Solve: An algorithm solves a problem  if that algorithm can be applied to any instance I of  and produce a solution to that instance I Example: A TSP solution algorithm step 1: Generate all the feasible paths step 2: Compare the distances among paths and save the shortest one  m! evaluations and comparisons for an m city instance 20! > Assume evaluations/sec.  >31 years  interested in finding the most “efficient” algorithm How to measure algorithmic efficiency?

Comparisons of the Two Classes of Algorithms Time Complexity function Size Second second second second second second.0001 second.0004 second.0009 second.0016 second.0025 second.0036 second.001 second.008 second.027 second.064 second.125 second.216 second.1 second 3.2 second 24.3 second 1.7 minutes 5.2 minutes 13.0 minutes.001 second 1.0 second 17.9 minutes 12.7 days 35.7 years 366 centuries.059 second 58 minutes 6.5 years 3855 centuries 2x centuries 1.3x centuries

Size of Largest Problem Instance Solvable in 1 Hour Time Complexity function With present Computer With computer 100 times faster With computer 1000 times faster

1.An algorithm is not a good algorithm if its complexity is exponential 2. A problem is not “well-solved” until a polynomial time algorithm is known for it 3. A problem is intractable if it is so hard that no polynomial time algorithm can possibly solve it Note: Time complexity is a wont case measure Exponential algorithms may be efficient for specific applications Algorithm and Problem Intractibility

Examples of Intractable problems 1) Hilbert’s tenth problem:  An integer solution ? 2) “Intractable problems in Control Theory” C.H. Papadimitriou and J. Tsitsiklis, SIAM J. control and Optimization, July, 1986 PP

§1.4.4 P and NP Problem, NP- Completeness deterministic algorithms the result of every operation is uniquely defined e.g. computer programs non-deterministic algorithms Results of operations can be any of a set of alternatives Example: Shopping list A shopping list can be viewed as a very simple nondeterministic algorithm. Every item on the list is a directive to find the indicated product, but the order in which to find them is not indicated.  P-class of problems: the set of all problems which can be solved by deterministic algorithms in polynomial time NP-Class of problems the set of all problems which can be solved by non-deterministic algorithms in polynomial time

NP-Completeness A problem  is NP-complete if  NP and every problem in NP can be reduced to  through polynomial time transformations (may not be deterministic) e.g. The satisfiability problem INSTANCE:Set U of variables, collection C of clauses over U. Question:Is there a satisfying truth assignment for C?  All NP-complete problems are equally difficult

Facts: (1) can be solved in polynomial time by a deterministic algorithm => NP=P (2) Q: NP problems intractable?  P=NP? no answer so far! Usually, if a problem is proven to be NP-complete =>give up finding optimal solution algorithm for it =>heuristics, sub-optimality or approximations.

Chapter 2 The Simplex algorithm §2.1 Basic Properties of LP §2.1.1 Forms of LP General LP(GLP) subject to (equalities) ( inequalities)

Proof: G=>S 1) ( for ) 2) Canonical form( C L P )Standard Form( S L P ) Three forms are equivalent ! ! = > need to stud y onl y one form

§2.1.2 Basic Feasible Solutions Goal: mathematically defines the concept of looking for the optimum among vertices Consider SLP = subject to A:m x n (m<n) Q: m ≥ n? Assume:  (A)=m A={ } B={ }  (B)=m I.e. Linearly indep. => is a basic solution if If a basic solution is also feasible, i.e. then it is a basic feasible solution (bfs)

Example min[ ] subject to and  B = a bfs ‚ B= = > basic not feasible

The fundamental theorem of LP Let (i) IF (ii) If optimal feasible solution => optimal bfs. Proof: (i) Let =>

The fundamental theorem of LP Let (i) IF (ii) If optimal feasible solution => optimal bfs. Proof: (i) Let => Case1 are linearly indep. => p ≤m if p=m => is a bfs if p can find from (linearly indep) => is a basic (degenerate) solution to case2 are linearly dependent => such that

Proof: (i) Let => Case1 are linearly indep. => p ≤ m if p=m => is a bfs if p can find from (linearly indep) => is a basic (degenerate) solution to case2 are linearly dependent => such that

Set Let otherwise (ii)Almost the same as ( i ) Read by yourself [Lue 8 4,p.2 0]

Implications of the Theorem (1) need only consider bfses for optimum (2) A:m x n => # of bfses ≤ # of bs ≤ Q: How to search efficiently? Next, geometric interpretation of properties of LP.d let The fundamental theorem of LP (i)IF (ii) If optimal feasible solution => optimal bfs.

Polytopes and LP 1. A polytope as the convex hull of a finite set of points Q: relation to D.O.? => can use a convex region to include a discrete region => can find a lower bound of the minimum 2. A polytope as the bounded intersection of half spaces => use linear inequalities to represent a polytope 3. 1) p(A)= m<n characterizes a polytope in 2) Any convex polytope can be alternatively viewed as the feasible region of an LP via a simple transformation. (i.e. can be represented by a set of linear ineqs.)

we now relate the vertex of a polytope to the bfs of the conesponding LP. vertex (or extreme point) => Let F be a convex set and IF it must hold that,then is called a vertex or an extreme point.

§2.1.3 Geometry of LP Hyperplane: ? Closed half space: ? Ploytope: – A bounded and nonempty intersection of a finite number of half spaces – 3 kinds of faces Example: Facet dim. n-1 Vertex dim. 0 Edgedim. 1 Z (001) (010) y x (100)

Convex Hull = Example v1 v2 v3 v4 v5

Equivalence of Vertices and bfses Theorem Let A: m x n, P(A)=m<n, and F= X is a vertex of F  x is a bfs of Proof: “=>” Let x be a vertex of F and assume that and for i > k => Show that are linearly indep. :

Assume liner dependence => such that set, => such that => => a contradiction to the fact that x is a vertex “<=“ try it by yourself

§2.2 The Revised Simplex Method The simplex method for LP : G.B. Dantzig 1951 Key idea: Phase1: find a bfs of Note that an LP may not have a solution Example min s.t.

phase 2 Allow one of the zero components of the bfs to become positive and force one of the original positive components to become zero. => How to pick “entering” and “leaving” component Cost Traditional form of the simplex method: Tableau => read by yourself Here we consider matrix form for conciseness of presentation and later developments.

SLP: s.t. Assume P(A)=m. Partition A=[B:D] B: m linearly indep. Columns of A(Assume the first m cols.) (SLP) subject to

If and a basic solution When from(2-4) (note that may not ≥0 Substituting(2-5) into the cost function Define as the relative cost vector => To minimize Z, we need only adjust How?