The Complexity of Optimization Problems

Summary -Complexity of algorithms and problems -Complexity classes: P and NP -Reducibility -Karp reducibility -Turing reducibility -NP-complete problems -Complexity of optimization problems -Classes PO and NPO -NP-hard optimization problems

Uniform and logarithmic cost -Uniform cost: the overall number of instructions executed by the algorithm before halting -Logarithmic cost:each instruction has a cost depending on the number of bits of the operands -E.g. product of two n-bit integer costs O(nlogn) -Same for space measure (but we will talk only of time measure)

Example: x y -Uniform cost: 2+3y -Logarithmic cost: aylogy+by 2 logx(logy+loglogx)+c begin r:=1; while y  0 do begin r:=r*x; y:=y-1 end; return r end.

Worst case analysis -Instances of the same size may result in different execution costs (e.g. sorting) -Cost of applying the algorithm on the worst case instance of a given size -Gives certainty that the algorithm will perform its task within the established time bound -It is easier to determine

Input size -Size of input: number of bits needed to present the specific input -Existence of encoding scheme which is used to describe any problem instance -For any pair of natural encoding schemes and for any instance x, the resulting strings are polynomially related -I.e., |e i (x)|  p i,j (|e j (x)|) and |e j (x)|  p i,j (|e i (x)|) -Avoid unary base encoding

Asymptotic Analysis -Let t(x) be the running time of algorithm A on input x. The worst case running time of A is given by t(n)=max(t(x) | x such that |x|  n) -Upper bound: A has complexity O(f(n)) if t(n) is O(f(n)) (that is, we ignore constants) -Lower bound: A has complexity  (f(n)) if t(n) is  (f(n))

Complexity of a problem -A problem P -has a complexity lower bound  (f(n)) if any algorithm for P has complexity  (f(n)) -has a complexity upper bound O(f(n)) if an algorithm for P exists with complexity O(f(n))

Decision problems -Set of instances partitioned into a YES-subset and a NO-subset -Given an instance x, decide which subset x belongs to -A decision problem P is solved by an algorithm A if, for every instance, A halts and returns YES if and only if the instance belongs to the YES-subset

Complexity Classes -For any function f(n), TIME(f(n)) is the set of decision problems which can be solved with a time complexity O(f(n)) -P = the union of TIME(n k ) for all k -EXPTIME = the union of TIME (2 n k ) for all k -P is contained in EXPTIME -It is possible to prove (by diagonalization) that EXPTIME is not contained in P

Examples -SATISFYING TRUTH ASSIGNMENT: given a CNF formula F and a truth assignment f, does f satisfy F ? -SATISFYING TRUTH ASSIGNMENT is in P -SATISFIABILITY (simply, SAT): given a CNF formula F, is F satisfiable? -SAT is in EXPTIME. -Open problem: SAT is in P?

Class NP -A problem P is in class NP if there exist a polynomial p and a polynomial-time algorithm A such that, for any instance x, x is a YES-instance if and only if there exists a string y with |y|  p(|x|) such that A (x,y) returns YES -y is said to be a certificate -Example: SAT is in NP (the certificate is a truth assignment that satisfies the formula) -P is contained in NP (the certificate is the computation of the polynomial-time algorithm)

Non-deterministic algorithms: SAT (v 1 or v 2 or (not v 3 )) and ((not v 1 ) or (not v 2 ) or v 3 ) begin for each variable v guess Boolean value f(v); if f satisfies F then return YES else return NO end.

Non-deterministic algorithms and NP Every problem in NP admits a polynomial-time non-deterministic algorithm begin guess string y with |y|  p(|x|); if A (x,y) returns YES then return YES else return NO end. Each computation path, which returns YES, is a certificate of polynomial length that can be checked in polynomial time Every problem that admits a polynomial- time non-deterministic algorithm is in NP

Karp reducibility -A decision problem P 1 is Karp reducible to a decision problem P 2 (in short, P 1  P 2 ) if there exists a polynomial-time computable function R such that, for any x, x is a YES-instance of P 1 if and only if R (x) is a YES-instance of P 2 -If P 1  P 2 and P 2 is in P, then P 1 is in P

Example: {0,1}-Linear programming -SAT  {0,1}-LINEAR PROGRAMMING -For each Boolean variable v of a CNF Boolean formula F, we introduce a {0,1}-valued variable z -For each clause l 1 or l 2 or … or l k of F, we introduce the inequality  1 +  2 + … +  k  1, where  i = z if l i = vand  i = (1-z) if l i = not v -E.g. (v 1 or v 2 or (not v 3 )) and ((not v 1 ) or (not v 2 ) or v 3 ) is transformed into the following two inequalities: z 1 +z 2 +(1-z 3 )  1and(1-z 1 )+(1-z 2 ) +z 3  1 -If f is a truth assignment, let g be the natural corresponding {0,1}-value assignment (0=FALSE,1=TRUE) -f satisfies F if and only g satisfies all inequalities

Turing reducibility -A decision problem P 1 is Turing reducible to a decision problem P 2 if there exists a polynomial-time algorithm R solving P 1 such that R may access to an oracle algorithm solving P 2 -If P 1  P 2 then P 1 is Turing reducible to P 2

Example: Equivalent formulas -SAT is Turing reducible to EQUIVALENT FORMULAS -Given a CNF Boolean formula F, query the oracle with input F and x and (not x) -If the oracle answers YES, then F is not satisfiable, otherwise F is satisfiable -It is not known whether SAT is Karp reducible to EQUIVALENT FORMULAS

NP-complete problems -A decision problem P is NP-complete if P is in NP and, for any decision problem P 1 in NP, P 1  P -If P is NP-complete and P is in P, then P=NP -NP-complete problems are the hardest in NP -P versus NP question can be solved by focusing on an NP- complete problem -Cook’s Theorem: SAT is NP-complete

Optimization problem -Optimization problem P characterized by -Set of instances I -Function SOL that associates to any instance the set of feasible solutions -Measure function m that, for any feasible solution, provides its positive integer value -Goal, that is, either MAX or MIN -An optimal solution is a feasible solution y* such that m(x,y*) = Goal{m(x,y) | y  SOL(x)} -For any instance x, m*(x) denotes optimal measure

MINIMUM VERTEX COVER -INSTANCE: Graph G=(V,E) -SOLUTION: A subset U of V such that, for any edge (u,v), either u is in U or v is in U -MEASURE: Cardinality of U

Three problems in one -Constructive problem: given an instance, compute an optimal solution and its value -We will study these problems -Evaluation problem: given an instance, compute the optimal value -Decision problem: given an instance and an integer k, decide whether the optimal value is at least (if Goal=MAX) or at most (if Goal=MIN) k

Class NPO -Optimization problems such that -I is recognizable in polynomial time -Solutions are polynomially bounded and recognizable in polynomial time -m is computable in polynomial time -Example: MINIMUM VERTEX COVER -Theorem : If P is in NPO, then the corresponding decision problem is in NP

Class PO -NPO problems solvable in polynomial time -Example: MINIMUM PATH -An optimization problem P is NP-hard if any problem in NP is Turing reducible to P -Theorem: If the decision problem corresponding to a NPO problem P is NP-complete, then P is NP-hard -Example: MINIMUM VERTEX COVER -Corollary: If P  NP then PO  NPO

Evaluating versus constructing -Decision problem is Turing reducible to evaluation problem -Evaluation problem is Turing reducible to constructive problem -Evaluation problem is Turing reducible to decision problem -Binary search on space of possible measure values -Is constructive problem Turing reducible to evaluation problem?

MAXIMUM SATISFIABILITY -INSTANCE: CNF Boolean formula, that is, set C of clauses over set of variables V -SOLUTION: A truth-assignment f to V -MEASURE: Number of satisfied clauses

Evaluating versus constructing: MAX SAT begin for each variable v begin k := MAX SAT eval (x); x TRUE := formula obtained by setting v to TRUE in x; x FALSE := formula obtained by setting v to FALSE in x; if MAX SAT eval (x TRUE ) = k then begin f(v) := TRUE; x := x TRUE end else begin f(v) := FALSE; x := x FALSE end; return f end. Theorem: if the decision problem is NP-complete, then the constructive problem is Turing reducible to the decision problem