2 CSC 551 Design and Analysis of Algorithms Giving credit where credit is dueThese lecture notes are based on slides byDr. CusackDr. LeubkeDr. GoddardAnd WikipediaI have modified them and added new slides
3 Tractability Some problems are intractable As they grow large, we are unable to solve them in reasonable timeWhat is reasonable time? Standard working definition: polynomial timeOn input of size n, the worst-case running time is (nk)Polynomial time: (n2), (n3), (1) (nlgn)Not in polynomial time: (2n), (nn), (n!)
4 Polynomial-Time Algorithms Are some problems solvable in polynomial time?Of course: every algorithm we’ve studied provides polynomial-time solution to some problemAre all problems solvable in polynomial-time?NoMost are optimization problems
5 NP-completeness Near the end of 60’s Growing list of problems with no efficient solutionRemarkable discovery:many of these problems were interrelatedIf one solved in polynomial time, all will be solved in polynomial timeNP-completeness
6 Our new focus: showing a given problem cannot be solved efficiently!
7 Optimization/Decision Problems Optimization problemsThey ask: “What is the optimal solution to problem X?”Examples0-1 KnapsackFractional KnapsackMinimum Spanning TreeDecision problemsThey ask: “Is there a solution to problem X with property Y?”ExampleDoes graph G have a MST of weight ≤ W?
8 Optimization/Decision Problems An optimization problem tries to find an optimal solutionA decision problem tries to answer a yes/no questionMany problems will have decision and optimization versionsE.g: MSTOptimization: find an MST of graph GDecision: does graph G have a MST of weight < W?
9 Our new focus: showing a given problem cannot be solved efficiently! We will phrase optimization problems as decision problemsIf the decision version (which is intuitively easier to solve) cannot be solved effciiently, the optimization version also cannot be solevd effciiently.
10 The Class PP: the class of decision problems that have polynomial-time deterministic algorithmsThat is, they are solvable in (nk)A deterministic algorithm is (essentially) one that always computes the correct answerSample problems in PFractional knapsackMSTSingle-source shortest pathSorting
11 The Class NPNP (nondeterministic polynomial-time): the class of decision problems that are verifiable in polynomial time.
12 Sample Problems in NP Fractional Knapsack MST Single-source shortest pathSortingOthers?KnapsackHamiltonian Cycle ProblemSatisfiability (SAT)Conjunctive Normal Form (CNF) SAT3-CNF SAT
13 Hamiltonian Cycle Problem (HCP) A hamiltonian-cycle of an undirected graph is a simple cycle that contains every vertex exactly once and returns to the starting vertexThe Hamiltonian-Cycle Problem: given an undirected graph G, does it have a hamiltonian cycle?The HCP is in NPA solution is easy to verify in polynomial time (how?)13
14 The MILLION DOLLAR Question Does P = NP?This is literally a million dollar questionThe Millennium ProblemsThe Clay Mathematics Institute of Cambridge, Massachusetts (CMI) has designated 7 problems as Millennium ProblemsEach problem has a prize of $1,000, to whomever solves itP = NP is one of the seven problems
15 P and NP What do we mean when we say a problem is in P? A: A solution can be found in polynomial timeWhat do we mean when we say a problem is in NP?A: A solution can be verified in polynomial timeWhat is the relation between P and NP?A: P NP, but no one knows whether P = NP15
16 NP-Complete Problems NPC problems are the “hardest” problems in NP If any one NPC problem can be solved in polynomial time…… then every NPC problem can be solved in polynomial time…… and in fact every problem in NP can be solved in polynomial time (which would show that P = NP)Thus: solve the hamiltonian-cycle in (n100) time, you’ve proved that P = NP. Retire rich & famous.
17 Reduction The crux of NP-Completeness is reducibility Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of PWhat do you suppose “easily” means?This rephrasing is called transformationIntuitively: If P reduces to Q easily, P is “no harder to solve” than Q
18 Reducibility An example: P: Given a set of Booleans, is at least one TRUE?Q: Given a set of integers, is their sum positive?Transformation: (x1, x2, …, xn) (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE
19 Using ReductionsIf P is polynomial-time reducible to Q, we denote this P ≤P QDefinition of NP-Complete: A decision problem Q is NPC iffQ NP (What does this mean?)Every problem in NP is polynomial-time reducible to QI.e. R ≤P Q for every R NP
20 NP-Hard Problems Definition: P is NP-Hard iff Every problem in NP is polynomial-time reducible to PI.e. R ≤P P for every R NPSo, an alternative definition for NPC:P is NPC iff P is NP-Hard and P NPImportant: For a problem to be NP-hard it does not have to be in class NP20
21 The Relationship between P, NP, and NP-Hard If there is a polynomial algorithm for any NP-hard problemThen there are polynomial algorithms for all problems in NPAnd hence P = NPIf P NP, then NP-hard problems have no solutions in polynomial timeIf P = NP, it does not resolve whether the NP-hard problems can be solved in polynomial time
22 The Relationship between P, NP, and NP-Hard NPCPNPNP-HardP = NP = NPCpossibilitiesIf P NPIf P = NP
23 Proving NP-Completeness Steps to prove a problem Q is NPC?Prove Q NPPick a known NPC problem PReduce P to QDescribe a transformation that maps instances of P to instances of Q, such that “yes” for Q = “yes” for PProve the transformation worksProve it runs in polynomial timeNP-hard
24 Why reduction to single NPC problem P is okay? If P is NPC, then every problem in NP is polynomial-time reducible to PTransitivity: If P can be polynomial reducible to Q, then every problem in NP is polynomial-time reducible to QHence, If P ≤P Q and P is NPC, Q is also NPCThis is the key idea for today
25 Why Prove NP-Completeness? Though nobody has proven that P NP, if you prove a problem NP-Complete, most people accept that it is probably intractableTherefore it can be important to prove that a problem is NP-CompleteDon’t need to come up with an efficient algorithmCan instead work on approximation algorithms
26 NPC Problems We will now look at some NPC problems In some cases we will also look at the transformations
27 Hamiltonian Cycle Problem A hamiltonian-cycle of an undirected graph is a simple cycle that contains every vertex exactly once and returns to the starting vertexThe Hamiltonian-Cycle Problem: given an undirected graph G, does it have a hamiltonian cycle?The HCP is in NPC
28 Traveling salesman problem (TSP) The well-known traveling salesman problem:Optimization variant: a salesman must travel to n cities, visiting each city exactly once and finishing where he begins. How to minimize travel time?Model as complete graph with cost c(i,j) to go from city i to city jHow would we turn this into a decision problem?A: ask if a TSP with cost <= k
29 Hamiltonian Cycle TSP The steps to prove TSP is NP-Complete:Prove that TSP NP (Argue this)Reduce the undirected hamiltonian cycle problem to the TSPSo if we had a TSP-solver, we could use it to solve the hamilitonian cycle problem in polynomial timeHow can we transform an instance of the hamiltonian cycle problem to an instance of the TSP?Can we do this in polynomial time?
30 Transformation: Hamiltonian Cycle TSP Let G = (V,E) be a graph with n nodesHam. cycle problemConstruct an instance of TSP as follows:Let G’ = (V,VxV) be a complete graph on the vertices of GThe edge weights c(u,v) are 1 if the edge (u,v) is in E and 2 otherwiseThe bound k is n where n is the number of vertices in V
31 Transformation: Hamiltonian Cycle TSP No Ham. Cycle1uvuv1112xwxw2The best tour has c of 5, which isgreater than n, so NO ham. cycle
32 Transformation: Hamiltonian Cycle TSP Ham. Cycle Present1uvuv1111xwxw2The best tour has c of 4, which isequal to n, so ham. cycle
33 Transformation: Hamiltonian Cycle TSP A hamiltonian cycle in G is a tour in G’ with cost n.If there are no hamiltonian cycles in G, any tour in G’ must cost at least n+1So G has a hamiltonian cycle iff G’ has a traveling salesperson tour of weight nIs this a polynomial-time transformation?
34 The Satisfiability (SAT) Problem Given a boolean expression on n variables, can we assign values such that the expression is TRUE?Ex: ((x1x2) ((x1x3) x4)) x2Simple enough but no known deterministic polynomial time algorithm existsEasy to verify in polynomial timeThis is the first NPC problemProven by Stephen Cook in 1971
35 Conjunctive Normal Form (CNF) Even if the form of the boolean expression is simplified, no known polynomial time algorithm existsLiteral: An occurrence of a boolean or its negationA boolean formula is in CNF if it is an AND of clauses, each of which is an OR of literalsEx: (x1 x2) (x1 x3 x4) (x5 )3-CNF: Each clause has exactly 3 distinct literalsEx: (x1 x2 x3 ) (x1 x3 x4) (x5 x3 x4 )Notice: TRUE if at least one literal in each clause is true
36 3SAT ProblemSatisfiability of Boolean formulas in 3-CNF form (the 3SAT Problem) is NP-CompleteProof: NopeThe reason we care about the 3-CNF problem is that it is relatively easy to reduce to othersThus by proving 3SAT NP-Complete we can prove many seemingly unrelated problems NP-Complete
37 3SAT Clique What is a clique of a graph G? A: a subset of vertices fully connected to each other, i.e. a complete subgraph of GThe clique problem: how large is the maximum-size clique in a graph?Can we turn this into a decision problem?A: Yes, we call this the k-clique problemIs the k-clique problem within NP?
38 3SAT Clique What should the reduction do? A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable
39 Transformation: 3SAT Clique The reduction:Let B = C1 C2 … Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literalsFor each clause put a triple of vertices in the graph, one for each literalPut an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation
40 Transformation: 3SAT Clique B = (x y z) (x y z ) (x y z )yxzxxyyzz
41 Transformation: 3SAT Clique Prove the reduction works:If B has a satisfying assignment, then each clause has at least one literal (vertex) that evaluates to 1Picking one such “true” literal from each clause gives a set V’ of k vertices. V’ is a clique (Why?)If G has a clique V’ of size k, it must contain one vertex in each triple (clause) (Why?)We can assign 1 to each literal corresponding with a vertex in V’, without fear of contradiction
42 Transformation: 3SAT Clique B = (x y z) (x y z ) (x y z )yxzAssume the satisfyingassignment is:x = 1y = 1z = 0So we know we havechosen k vertices,and those k verticesmust be a cliquexxyyzz
43 Transformation: 3SAT Clique B = (x y z) (x y z ) (x y z )yxzAssume there isa clique of size 3Make the assignment:x = 1y = 1z = 1and we know thateach triple MUSTcontain one of theseand thus B is satisfiedxxyyzz
44 General CommentsLiterally hundreds of problems have been shown to be NP-CompleteSome reductions are profound, some are comparatively easy, many are easy once the key insight is given
45 Other NP-Complete Problems Subset-sum: Given a set of integers, does there exist a subset that adds up to some target T?0-1 knapsack: when weights not just integersHamiltonian path: ObviousGraph coloring: can a given graph be colored with k colors such that no adjacent vertices are the same color?Etc…
46 Review: P and NP What do we mean when we say a problem is in P? A: A solution can be found in polynomial timeWhat do we mean when we say a problem is in NP?A: A solution can be verified in polynomial timeWhat is the relation between P and NP?A: P NP, but no one knows whether P = NP
47 Review: NP-CompleteWhat, intuitively, does it mean if we can reduce problem P to problem Q?P is “no harder than” QHow do we reduce P to Q?Transform instances of P to instances of Q in polynomial time s.t. Q: “yes” iff P: “yes”What does it mean if Q is NP-Hard?Every problem PNP p QWhat does it mean if Q is NP-Complete?Q is NP-Hard and Q NP
48 Review: Proving Problems NP-Complete How do we usually prove that a problem R is NP-Complete?A: Show R NP, and reduce a known NP-Complete problem Q to R