1 UNIT -6 P, NP, and NP-Complete

2 Tractability u Some problems are intractable: as they grow large, we are unable to solve them in reasonable time u What constitutes reasonable time? »Standard working definition: polynomial time »On an input of size n the worst-case running time is O(n k ) for some constant k »O(n 2 ), O(n 3 ), O(1), O(n lg n), O(2 n ), O(n n ), O(n!) »Polynomial time: O(n 2 ), O(n 3 ), O(1), O(n lg n) »Not in polynomial time: O(2 n ), O(n n ), O(n!)

3 Polynomial-Time Algorithms u Are some problems solvable in polynomial time? »Of course: many algorithms we’ve studied provide polynomial-time solutions to some problems u Are all problems solvable in polynomial time? »No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given u Most problems that do not yield polynomial-time algorithms are either optimization or decision problems.

4 Optimization/Decision Problems u Optimization Problems »An optimization problem is one which asks, “What is the optimal solution to problem X?” »Examples: v 0-1 Knapsack v Fractional Knapsack v Minimum Spanning Tree u Decision Problems »An decision problem is one with yes/no answer »Examples: v Does a graph G have a MST of weight  W?

5 Optimization/Decision Problems u An optimization problem tries to find an optimal solution u A decision problem tries to answer a yes/no question u Many problems will have decision and optimization versions »Eg: Traveling salesman problem v optimization: find hamiltonian cycle of minimum weight v decision: is there a hamiltonian cycle of weight  k u Some problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time »Is there a polynomial-time algorithm that solves the problem?

6 The Class P P: the class of decision problems that have polynomial-time deterministic algorithms. »That is, they are solvable in O(p(n)), where p(n) is a polynomial on n »A deterministic algorithm is (essentially) one that always computes the correct answer Why polynomial? »if not, very inefficient »nice closure properties v the sum and composition of two polynomials are always polynomials too

7 Sample Problems in P u Fractional Knapsack u MST u Sorting u Others?

8 The class NP NP: the class of decision problems that are solvable in polynomial time on a nondeterministic machine (or with a nondeterministic algorithm) u (A determinstic computer is what we know) u A nondeterministic computer is one that can “guess” the right answer or solution »Think of a nondeterministic computer as a parallel machine that can freely spawn an infinite number of processes u Thus NP can also be thought of as the class of problems »whose solutions can be verified in polynomial time u Note that NP stands for “Nondeterministic Polynomial-time”

9 Sample Problems in NP u Fractional Knapsack u MST u Sorting u Others? »Hamiltonian Cycle (Traveling Salesman) »Graph Coloring »Satisfiability (SAT) v the problem of deciding whether a given Boolean formula is satisfiable

10 NP-complete problems u A decision problem D is NP-complete iff 1.D  NP 2.every problem in NP is polynomial-time reducible to D

11 NP-Hard and NP-Complete u If R is polynomial-time reducible to Q, we denote this R  p Q u Definition of NP-Hard and NP-Complete: »If all problems R  NP are polynomial-time reducible to Q, then Q is NP-Hard »We say Q is NP-Complete if Q is NP-Hard and Q  NP u If R  p Q and R is NP-Hard, Q is also NP-Hard (why?)

12 Boolean satisfiability problem u In computer science, the Boolean Satisfiability Problem (sometimes called Propositional Satisfiability Problem and abbreviated as SATISFIABILITY or SAT) is the problem of determining if there exists an interpretation that satisfies a given Boolean formula.computer scienceinterpretationsatisfiesBooleanformula u In other words, it asks whether the variables of a given Boolean formula can be consistently replaced by the values TRUE or FALSE in such a way that the formula evaluates to TRUE.evaluates to TRUE u If this is the case, the formula is called satisfiable.

13 u On the other hand, if no such assignment exists, the function expressed by the formula is identically FALSE for all possible variable assignments and the formula is unsatisfiable.identically FALSE u For example, the formula "a AND NOT b" is satisfiable because one can find the values a = TRUE and b = FALSE, which make (a AND NOT b) = TRUE. In contrast, "a AND NOTa" is unsatisfiable.

14 u SAT is one of the first problems that was proven to be NP-complete.NP-complete u This means that all problems in the complexity classNP, which includes a wide range of natural decision and optimization problems, are in a technical sense at most that difficult to solve as SAT.NP u There is no known algorithm that efficiently solves SAT, and it is generally believed that no such algorithm exists; yet this belief has not been proven mathematically, and resolving the question whether SAT has an efficient algorithm is equivalent to the P versus NP problem, which is the most famous open problem in the theory of computing.P versus NP problem

15 u Despite the fact that no algorithms are known that solve SAT efficiently, correctly, and for all possible input instances, many instances of SAT that occur in practice, such as in artificial intelligence, circuit design and automatic theorem proving, can actually be solved rather efficiently using heuristical (intelligent guesswork) SAT-solvers.artificial intelligencecircuit designautomatic theorem proving u Such algorithms are not believed to be efficient on all SAT instances, but experimentally these algorithms tend to work well for many practical applications.

16 Basic definitions and terminology u A propositional logic formula, also called Boolean expression, is built from variables, operators AND (conjunction, also denoted by ∧ ), OR (disjunction, ∨ ), NOT (negation, ¬), and parentheses. A formula is said to be satisfiable if it can be made TRUE by assigning appropriate logical values (i.e. TRUE, FALSE) to its variables.propositional logicvariablesconjunctiondisjunctionnegationlogical values u The Boolean satisfiability problem (SAT) is, given a formula, to check whether it is satisfiable. u This decision problem is of central importance in various areas of computer science, including theoretical computer science, complexitytheory, algorithmics, cryptography and artificial intelligence.decision problemcomputer sciencetheoretical computer sciencecomplexitytheoryalgorithmicscryptographyartificial intelligence

17 u There are several special cases of the Boolean satisfiability problem in which the formulas are required to have a particular structure. u A literal is either a variable, then called positive literal, or the negation of a variable, then called negative literal. A clause is a disjunction of literals (or a single literal). u A clause is called Horn clause if it contains at most one positive literal.Horn clause u A formula is in conjunctive normal form (CNF) if it is a conjunction of clauses (or a single clause).conjunctive normal form

18 u For example, "x 1 " is a positive literal, "¬x 2 " is a negative literal, "x 1 ∨ ¬x 2 " is a clause, and "(x 1 ∨ ¬x 2 ) ∧ (¬x 1 ∨ x 2 ∨ x 3 ) ∧ ¬x 1 " is a formula in conjunctive normal form, its 1st and 3rd clause are Horn clauses, but its 2nd clause is not. u The formula is satisfiable, choosing x 1 =FALSE, x 2 =FALSE, and x 3 arbitrarily, since (FALSE ∨ ¬FALSE) ∧ (¬FALSE ∨ FALSE ∨ x 3 ) ∧ ¬FALSE evaluates to (FALSE ∨ TRUE) ∧ (TRUE ∨ FALSE ∨ x 3 ) ∧ TRUE, and in turn to TRUE ∧ TRUE ∧ TRUE (i.e. to TRUE). In contrast, the CNF formula a ∧ ¬a, consisting of two clauses of one literal, is unsatisfiable, since for a=TRUE anda=FALSE it evaluates to TRUE ∧ ¬TRUE (i.e. to FALSE) and FALSE ∧ ¬FALSE (i.e. again to FALSE), respectively.

19 The Satisfiability (SAT) Problem u Satisfiability (SAT): »Given a Boolean expression on n variables, can we assign values such that the expression is TRUE? »Ex: ((x 1  x 2 )   ((  x 1  x 3 )  x 4 ))  x 2 »Seems simple enough, but no known deterministic polynomial time algorithm exists »Easy to verify in polynomial time!

20 Example: CNF satisfiability u This problem is in NP. Nondeterministic algorithm: »Guess truth assignment »Check assignment to see if it satisfies CNF formula u Example: (A  ¬ B  ¬ C )  ( ¬ A  B)  ( ¬ B  D  F )  (F  ¬ D) u Truth assignments: A B C D E F (how many more?) Checking phase: Θ(n)

21 Comple xity ©D.Mos hkovitz 21 Conclusion: SAT is NP- Complete For any language A in NP, testing membership in A can be reduced to... satisfiability problem

22 Comple xity ©D.Mos hkovitz 22 Co-NP Revisiting the Map NP P NPC SAT

23 Comple xity ©D.Mos hkovitz 23 Looking Forward From now on, in order to show some NP problem is NP-Complete, we merely need to reduce SAT to it. any NP problem can be reduced to... SAT new NP problem can be reduced to... We’ve already shown this will imply the new problem is in NPC

24 Comple xity ©D.Mos hkovitz and Beyond! Moreover, every NP-Complete problem we discover, provides us with a new way for showing problems to be NP-Complete. any NP problem can be reduced to... Known NP-hard problem new NP problem can be reduced to...

25

26 Review: P And NP Summary u P = set of problems that can be solved in polynomial time »Examples: Fractional Knapsack, … u NP = set of problems for which a solution can be verified in polynomial time »Examples: Fractional Knapsack,…, Hamiltonian Cycle, CNF SAT, 3-CNF SAT u Clearly P  NP u Open question: Does P = NP? »Most suspect not »An August 2010 claim of proof that P ≠ NP, by Vinay Deolalikar, researcher at HP Labs, Palo Alto, has flaws

Chapter 8 The Theory of NP-Completeness

28 u KINDS OF PROBLEMS u Decision problems u Optimization problems 8- 28

Decision problems u The solution is simply “Yes” or “No”. u Optimization problems are more difficult. u e.g. the traveling salesperson problem »Optimization version: Find the shortest tour »Decision version: Is there a tour whose total length is less than or equal to a constant c ?

Solving an optimization problem by a decision algorithm : u Solving TSP optimization problem by a decision algorithm : »Give c 1 and test (decision algorithm) Give c 2 and test (decision algorithm)  Give c n and test (decision algorithm) »We can easily find the smallest c i

u CATEGORIES OF PROBLEMS Undecidable problems Intractable problems Tractable problems

32 Non deterministic Algorithms u Deterministic Algorithms : Result of every operation is uniquely defined. u Nondeterministic Algorithms : Results of operations are not uniquely defined but are limited to specified sets of possibilities. u To specify nondeterministic algorithms, we introduce three new functions: u Choice(S) – arbitrarily chooses one of the elements of set S. u Failure() – signals an unsuccessful completion. u Success() – signals a successful completion. Nondeterministic machine : A machine capable of executing a non deterministic algorithm

33 Nondeterministic search j := Choice(1, n); if A[j] = x then { write(j); success(); } write(0); Failure(); 8- 33

34 Class P u A decision problem D is solvable in polynomial time or in the class P, if there exists an algorithm A such that u A takes instances of D as inputs. u A always outputs the correct answer “Yes” or “No”. u There exists a polynomial p such that the execution of A on inputs of size n always terminates in p(n) or fewer step 8- 34

35 Class NP u A decision problem solvable by non deterministic algorithms in polynomial Time or in the class NP if there exists an algorithm A such that u A takes as inputs potential witnesses for “yes” answers to problem D. u A correctly distinguishes true witnesses from false witnesses. u There exists a polynomial p such that for each potential witnesses of each instance of size n of D, the execution of the algorithm A takes at most p(n) steps

36 P and NP u P is the set of all decision problems solvable by deterministic algorithms in polynomial time. u NP is the set of all decision problems solvable by nondeterministic algorithms in polynomial time. u P is subset of NP 8- 36

37 Polynomial Time Reduction u Problem L1 reduces to L2 (L1 α L2) if and only if there is a way to solve L1 by a deterministic polynomial time algorithm using a deterministic algorithm that solves L2 in polynomial time. u If there is a polynomial time algorithm for L2, then we can solve L1 in polynomial time. α is a transitive relation. Example: u The Hamiltonian cycle problem is polynomial-time reducible to the decision version of TSP

38 NP-Hard and NP-Complete u NP-HARD : Problem P is said to be NP-hard, u If all problems Q Є NP, are reducible to P. u Informally, P is “as hard as” all the problems in NP. u NP-COMPLETE : Problem P is said to be NP-Complete, u If P is NP-hard, and u P Є NP. Informally, P is one of the hardest problems in NP. If P α R, and P is NP-Complete, then R is also NP-Complete

u P: the class of problems which can be solved by a deterministic polynomial algorithm. u NP : the class of decision problem which can be solved by a non-deterministic polynomial algorithm. u NP-hard: the class of problems to which every NP problem reduces. u NP-complete (NPC): the class of problems which are NP-hard and belong to NP.

Some concepts of NPC  Definition of reduction: Problem A reduces to problem B (A  B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time. u Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case. u It does not seem to have any polynomial time algorithm to solve the NPC problems.

u The theory of NP-completeness always considers the worst case. u The lower bound of any NPC problem seems to be in the order of an exponential function. u Not all NP problems are difficult. (e.g. the MST problem is an NP problem.)  If A, B  NPC, then A  B and B  A. u Theory of NP-completeness: If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

The satisfiability problem u The satisfiability problem »The logical formula : x 1 v x 2 v x 3 & - x 1 & - x 2 the assignment : x 1 ← F, x 2 ← F, x 3 ← T will make the above formula true. (-x 1, -x 2, x 3 ) represents x 1 ← F, x 2 ← F, x 3 ← T

u If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable. u An unsatisfiable formula : x 1 v x 2 & x 1 v -x 2 & -x 1 v x 2 & -x 1 v -x 2

u Definition of the satisfiability problem: Given a Boolean formula, determine whether this formula is satisfiable or not. u A literal : x i or -x i  A clause : x 1 v x 2 v -x 3  C i u A formula : conjunctive normal form (CNF) C 1 & C 2 & … & C m

Nondeterministic algorithms u A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking u If the checking stage of a nondeterministic algorithm is of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm. u NP problems : (must be decision problems) »e.g. searching, MST sorting satisfiability problem (SAT) traveling salesperson problem (TSP)

Decision problems u Decision version of sorting: Given a 1, a 2,…, a n and c, is there a permutation of a i s ( a 1, a 2, …,a n ) such that ∣ a 2 –a 1 ∣ + ∣ a 3 –a 2 ∣ + … + ∣ a n –a n-1 ∣＜ c ? u Not all decision problems are NP problems »E.g. halting problem : v Given a program with a certain input data, will the program terminate or not? v NP-hard v Undecidable

Nondeterministic operations and functions [Horowitz 1998] u Choice(S) : arbitrarily chooses one of the elements in set S u Failure : an unsuccessful completion u Success : a successful completion u Nonderministic searching algorithm: j ← choice(1 : n) /* guessing */ if A(j) = x then success /* checking */ else failure

u A nondeterministic algorithm terminates unsuccessfully iff there does not exist a set of choices leading to a success signal. u The time required for choice(1 : n) is O(1).  A deterministic interpretation of a non-deterministic algorithm can be made by allowing unbounded parallelism in computation.

Nondeterministic sorting B ← 0 /* guessing */ for i = 1 to n do j ← choice(1 : n) if B[j] ≠ 0 then failure B[j] = A[i] /* checking */ for i = 1 to n-1 do if B[i] > B[i+1] then failure success

Nondeterministic SAT /* guessing */ for i = 1 to n do x i ← choice( true, false ) /* checking */ if E(x 1, x 2, …,x n ) is true then success else failure

Cook’s theorem NP = P iff the satisfiability problem is a P problem. u SAT is NP-complete. u It is the first NP-complete problem.  Every NP problem reduces to SAT. Stephen Arthur Cook (1939~)