P, NP, and NP-Complete Suzan Köknar-Tezel

CSC 551 Design and Analysis of Algorithms Giving credit where credit is due These lecture notes are based on slides by Dr. Cusack http://www.cse.unl.edu/~goddard/Courses/CSCE310J/Lectures/Lecture10-NPcomplete.pdf Dr. Leubke http://www.cs.virginia.edu/~luebke/cs332/index.html Dr. Goddard http://www.cse.unl.edu/~goddard/Courses/CSCE310J And Wikipedia I have modified them and added new slides

Tractability Some problems are intractable As they grow large, we are unable to solve them in reasonable time What is reasonable time? Standard working definition: polynomial time On input of size n, the worst-case running time is (nk) Polynomial time: (n2), (n3), (1) (nlgn) Not in polynomial time: (2n), (nn), (n!)

Polynomial-Time Algorithms Are some problems solvable in polynomial time? Of course: every algorithm we’ve studied provides polynomial-time solution to some problem Are all problems solvable in polynomial-time? No Most are optimization problems

NP-completeness Near the end of 60’s Growing list of problems with no efficient solution Remarkable discovery: many of these problems were interrelated If one solved in polynomial time, all will be solved in polynomial time NP-completeness

Our new focus: showing a given problem cannot be solved efficiently!

Optimization/Decision Problems Optimization problems They ask: “What is the optimal solution to problem X?” Examples 0-1 Knapsack Fractional Knapsack Minimum Spanning Tree Decision problems They ask: “Is there a solution to problem X with property Y?” Example Does graph G have a MST of weight ≤ W?

Optimization/Decision Problems An optimization problem tries to find an optimal solution A decision problem tries to answer a yes/no question Many problems will have decision and optimization versions E.g: MST Optimization: find an MST of graph G Decision: does graph G have a MST of weight < W?

Our new focus: showing a given problem cannot be solved efficiently! We will phrase optimization problems as decision problems If the decision version (which is intuitively easier to solve) cannot be solved effciiently, the optimization version also cannot be solevd effciiently.

The Class P P: the class of decision problems that have polynomial-time deterministic algorithms That is, they are solvable in (nk) A deterministic algorithm is (essentially) one that always computes the correct answer Sample problems in P Fractional knapsack MST Single-source shortest path Sorting

The Class NP NP (nondeterministic polynomial-time): the class of decision problems that are verifiable in polynomial time.

Sample Problems in NP Fractional Knapsack MST Single-source shortest path Sorting Others? Knapsack Hamiltonian Cycle Problem Satisfiability (SAT) Conjunctive Normal Form (CNF) SAT 3-CNF SAT

Hamiltonian Cycle Problem (HCP) A hamiltonian-cycle of an undirected graph is a simple cycle that contains every vertex exactly once and returns to the starting vertex The Hamiltonian-Cycle Problem: given an undirected graph G, does it have a hamiltonian cycle? The HCP is in NP A solution is easy to verify in polynomial time (how?) 13

The MILLION DOLLAR Question Does P = NP? This is literally a million dollar question The Millennium Problems The Clay Mathematics Institute of Cambridge, Massachusetts (CMI) has designated 7 problems as Millennium Problems http://www.claymath.org/millennium/ Each problem has a prize of $1,000,000.00 to whomever solves it P = NP is one of the seven problems

P and NP What do we mean when we say a problem is in P? A: A solution can be found in polynomial time What do we mean when we say a problem is in NP? A: A solution can be verified in polynomial time What is the relation between P and NP? A: P  NP, but no one knows whether P = NP 15

NP-Complete Problems NPC problems are the “hardest” problems in NP If any one NPC problem can be solved in polynomial time… … then every NPC problem can be solved in polynomial time… … and in fact every problem in NP can be solved in polynomial time (which would show that P = NP) Thus: solve the hamiltonian-cycle in (n100) time, you’ve proved that P = NP. Retire rich & famous.

Reduction The crux of NP-Completeness is reducibility Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P What do you suppose “easily” means? This rephrasing is called transformation Intuitively: If P reduces to Q easily, P is “no harder to solve” than Q

Reducibility An example: P: Given a set of Booleans, is at least one TRUE? Q: Given a set of integers, is their sum positive? Transformation: (x1, x2, …, xn) (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

Using Reductions If P is polynomial-time reducible to Q, we denote this P ≤P Q Definition of NP-Complete: A decision problem Q is NPC iff Q  NP (What does this mean?) Every problem in NP is polynomial-time reducible to Q I.e. R ≤P Q for every R  NP

NP-Hard Problems Definition: P is NP-Hard iff Every problem in NP is polynomial-time reducible to P I.e. R ≤P P for every R  NP So, an alternative definition for NPC: P is NPC iff P is NP-Hard and P  NP Important: For a problem to be NP-hard it does not have to be in class NP 20

The Relationship between P, NP, and NP-Hard If there is a polynomial algorithm for any NP-hard problem Then there are polynomial algorithms for all problems in NP And hence P = NP If P  NP, then NP-hard problems have no solutions in polynomial time If P = NP, it does not resolve whether the NP-hard problems can be solved in polynomial time

The Relationship between P, NP, and NP-Hard NPC P NP NP-Hard P = NP = NPC possibilities If P  NP If P = NP

Proving NP-Completeness Steps to prove a problem Q is NPC? Prove Q  NP Pick a known NPC problem P Reduce P to Q Describe a transformation that maps instances of P to instances of Q, such that “yes” for Q = “yes” for P Prove the transformation works Prove it runs in polynomial time NP-hard

Why reduction to single NPC problem P is okay? If P is NPC, then every problem in NP is polynomial-time reducible to P Transitivity: If P can be polynomial reducible to Q, then every problem in NP is polynomial-time reducible to Q Hence, If P ≤P Q and P is NPC, Q is also NPC This is the key idea for today

Why Prove NP-Completeness? Though nobody has proven that P  NP, if you prove a problem NP-Complete, most people accept that it is probably intractable Therefore it can be important to prove that a problem is NP-Complete Don’t need to come up with an efficient algorithm Can instead work on approximation algorithms

NPC Problems We will now look at some NPC problems In some cases we will also look at the transformations

Hamiltonian Cycle Problem A hamiltonian-cycle of an undirected graph is a simple cycle that contains every vertex exactly once and returns to the starting vertex The Hamiltonian-Cycle Problem: given an undirected graph G, does it have a hamiltonian cycle? The HCP is in NPC

Traveling salesman problem (TSP) The well-known traveling salesman problem: Optimization variant: a salesman must travel to n cities, visiting each city exactly once and finishing where he begins. How to minimize travel time? Model as complete graph with cost c(i,j) to go from city i to city j How would we turn this into a decision problem? A: ask if  a TSP with cost <= k

Hamiltonian Cycle  TSP The steps to prove TSP is NP-Complete: Prove that TSP  NP (Argue this) Reduce the undirected hamiltonian cycle problem to the TSP So if we had a TSP-solver, we could use it to solve the hamilitonian cycle problem in polynomial time How can we transform an instance of the hamiltonian cycle problem to an instance of the TSP? Can we do this in polynomial time?

Transformation: Hamiltonian Cycle  TSP Let G = (V,E) be a graph with n nodes Ham. cycle problem Construct an instance of TSP as follows: Let G’ = (V,VxV) be a complete graph on the vertices of G The edge weights c(u,v) are 1 if the edge (u,v) is in E and 2 otherwise The bound k is n where n is the number of vertices in V

Transformation: Hamiltonian Cycle  TSP No Ham. Cycle 1 u v u v 1 1 1 2 x w x w 2 The best tour has c of 5, which is greater than n, so NO ham. cycle

Transformation: Hamiltonian Cycle  TSP Ham. Cycle Present 1 u v u v 1 1 1 1 x w x w 2 The best tour has c of 4, which is equal to n, so  ham. cycle

Transformation: Hamiltonian Cycle  TSP A hamiltonian cycle in G is a tour in G’ with cost n. If there are no hamiltonian cycles in G, any tour in G’ must cost at least n+1 So G has a hamiltonian cycle iff G’ has a traveling salesperson tour of weight n Is this a polynomial-time transformation?

The Satisfiability (SAT) Problem Given a boolean expression on n variables, can we assign values such that the expression is TRUE? Ex: ((x1x2)  ((x1x3)  x4))  x2 Simple enough but no known deterministic polynomial time algorithm exists Easy to verify in polynomial time This is the first NPC problem Proven by Stephen Cook in 1971

Conjunctive Normal Form (CNF) Even if the form of the boolean expression is simplified, no known polynomial time algorithm exists Literal: An occurrence of a boolean or its negation A boolean formula is in CNF if it is an AND of clauses, each of which is an OR of literals Ex: (x1   x2)  (x1  x3  x4)  (x5 ) 3-CNF: Each clause has exactly 3 distinct literals Ex: (x1  x2  x3 )  (x1  x3  x4)  (x5  x3  x4 ) Notice: TRUE if at least one literal in each clause is true

3SAT Problem Satisfiability of Boolean formulas in 3-CNF form (the 3SAT Problem) is NP-Complete Proof: Nope The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others Thus by proving 3SAT NP-Complete we can prove many seemingly unrelated problems NP-Complete

3SAT  Clique What is a clique of a graph G? A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G The clique problem: how large is the maximum-size clique in a graph? Can we turn this into a decision problem? A: Yes, we call this the k-clique problem Is the k-clique problem within NP?

3SAT  Clique What should the reduction do? A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

Transformation: 3SAT  Clique The reduction: Let B = C1  C2  …  Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals For each clause put a triple of vertices in the graph, one for each literal Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation

Transformation: 3SAT  Clique B = (x  y  z)  (x  y  z )  (x  y  z ) y x z x x y y z z

Transformation: 3SAT  Clique Prove the reduction works: If B has a satisfying assignment, then each clause has at least one literal (vertex) that evaluates to 1 Picking one such “true” literal from each clause gives a set V’ of k vertices. V’ is a clique (Why?) If G has a clique V’ of size k, it must contain one vertex in each triple (clause) (Why?) We can assign 1 to each literal corresponding with a vertex in V’, without fear of contradiction

Transformation: 3SAT  Clique B = (x  y  z)  (x  y  z )  (x  y  z ) y x z Assume the satisfying assignment is: x = 1 y = 1 z = 0 So we know we have chosen k vertices, and those k vertices must be a clique x x y y z z

Transformation: 3SAT  Clique B = (x  y  z)  (x  y  z )  (x  y  z ) y x z Assume there is a clique of size 3 Make the assignment: x = 1 y = 1 z = 1 and we know that each triple MUST contain one of these and thus B is satisfied x x y y z z

General Comments Literally hundreds of problems have been shown to be NP-Complete Some reductions are profound, some are comparatively easy, many are easy once the key insight is given

Other NP-Complete Problems Subset-sum: Given a set of integers, does there exist a subset that adds up to some target T? 0-1 knapsack: when weights not just integers Hamiltonian path: Obvious Graph coloring: can a given graph be colored with k colors such that no adjacent vertices are the same color? Etc…

Review: P and NP What do we mean when we say a problem is in P? A: A solution can be found in polynomial time What do we mean when we say a problem is in NP? A: A solution can be verified in polynomial time What is the relation between P and NP? A: P  NP, but no one knows whether P = NP

Review: NP-Complete What, intuitively, does it mean if we can reduce problem P to problem Q? P is “no harder than” Q How do we reduce P to Q? Transform instances of P to instances of Q in polynomial time s.t. Q: “yes” iff P: “yes” What does it mean if Q is NP-Hard? Every problem PNP p Q What does it mean if Q is NP-Complete? Q is NP-Hard and Q  NP

Review: Proving Problems NP-Complete How do we usually prove that a problem R is NP-Complete? A: Show R NP, and reduce a known NP-Complete problem Q to R