1. P and NP

2. Computational Complexity
Recall from our sorting examples at the start of class that we could prove that any sort would have to do at least some minimal amount of work (lower bound)
We proved this using decision trees (ch 7.8)
(the following decision tree slides are repeated)

3. Yes No Yes No No Yes Yes No Yes No a < b b < c b < c a,b,c
// Sorts an array of 3 items
void sortthree(int s[]) {
a=s[1]; b=s[2]; c=s[3];
if (a < b) {
if (b < c) {
S = a,b,c;
} else {
if (a < c) {
S = a,c,b;
S = c,a,b; }}
} else if (b < c) {
S = b,a,c;
S = b,c,a;
else {
S = c,b,a; }
a < b
Yes No
b < c
b < c
Yes No No
Yes
a,b,c
a < c
a < c
c,b,a
Yes No
Yes No
a,c,b
c,a,b
b,a,c
b,c,a

4. Decision Trees
A decision tree can be created for every comparison-based sorting algorithm
The following is a decision tree for a 3 element Exchange sort
Note that “c < b” means that the Exchange sort compares the array item whose current value is c with the one whose current value is b – not that it compares s[3] to s[2].

5. Yes No No Yes No Yes No Yes Yes Yes Yes No b < a c < a c < b
a < b
c < b No
Yes
Yes
Yes
Yes No
c,b,a
b,c,a
b,a,c
c,a,b
a,c,b
a,b,c

6. Decision Trees So what does this tell us…
Note that there are 6 leaves in each of the examples given (each N=3)
In general there will be N! leaves in a decision tree corresponding to the N! permutations of the array
The number of comparisons (“work”) is equal to the depth of the tree (from root to leaf)
Worst case behavior is the path from the root to the deepest leaf

7. Decision Trees
Thus, to get a lower bound on the worst case behavior we need to find the shortest tree possible that can still hold N! leaves
No comparison-based sort could do better
A tree of depth d can hold 2d leaves
So, what is the minimal d where 2d >= N!
Solving for d we get d >= log2(N!)
The minimal depth must be at least log2(N!)

8. Decision Trees According to Lemma 7.4 (p. 291):
log2(N!) >= n log2(n) – 1.45n
Putting that together with the previous result
d must be at least as great as (n log2(n) – 1.45n)
Applying Big-O
d must be at least O(n log2(n))
No comparison-based sorting algorithm can have a running time better than O(n log2(n))

9. Decision Trees for other problems?
Unfortunately, this technique only applies directly to comparison-based sorts
What if the problem is Traveling Salesperson?
The book has only shown exponential time (or worse) algorithms for this problem
What if your boss wants a faster implementation?
Do you try and find one?
Do you try and prove one doesn’t exist?

10. Polynomial-Time Algorithms
A polynomial-time algorithm is one whose worst-case running time is bounded above by a polynomial function
Poly-time examples: 2n, 3n, n5, n log(n), n100000
Non-poly-time examples: 2n, n, n!
Poly-time is important because for large problem sizes, all non-poly-time algorithms will take forever to execute

11. Intractability
In Computer Science, a problem is called intractable if it is impossible to solve it with a polynomial-time algorithm
Let me stress that intractability is a property of the problem, not just of any one algorithm to solve the problem
There can be no poly-time algorithm that solves the problem if the problem is to be considered intractable
And just because one non-poly-time algorithm exists for the problem does not make it intractable

12. Three Categories of Problems
We can group problems into 3 categories:
Problems for which poly-time algorithms have been found
Problems that have been proven to be intractable
Proven that no poly-time algorithms exist
Problems that have not been proven to be intractable, but for which poly-time algorithms have never been found
No one has found a poly-time algorithm, but no one has proven that one doesn’t exist either
The interesting thing is that most problems in CS fall into the 1st & 3rd categories and very few into the second

13. Poly-time Category
Any problem for which we have found a poly-time algorithm
Sorting, searching, matrix multiplication, chained matrix multiplication, shortest paths, minimal spanning tree, etc.

14. Intractable Category Two types of problems
Those that require a non-polynomial amount of output
Determining all Hamiltonian Circuits
(n – 1)! Circuits in worst case
Those that produce a reasonable amount of output, but the processing time is just too long
Very few of these
Some are undecidable problems
Halting Problem
Presburger Arithmetic

15. Unknown Category
Not proven to be Intractable but no Poly-time algorithm known
Many problems belong in the category
0-1 Knapsack, Traveling Salesperson, m-coloring, CNF-Satisfiability, etc.
In general, any problem that we had to solve using backtracking or bounded backtracking falls into this category

16. The Theory of NP
There is a close and interesting relationship among many of the problem in the Unknown Category
It will be more convenient to develop this theory restricting ourselves to decision problems
Problems that have a yes/no answer
We can always convert non-decision problems into decision problems
In the Traveling Salesperson Problem instead of just asking for the optimal tour, we can instead ask if the optimal tour is no greater than some number d
In graph coloring instead of just asking the minimal number of colors we can instead ask if the minimal number is less than m

17. The Set P
The set P is the set of all decision problems that can be solved by polynomial-time algorithms
What problems are in P?
Obviously all the ones we have found poly-time solutions for (sorting, etc)
What about problems like Traveling Salesperson?

18. The Set NP For Traveling Salesperson: For M-Coloring:
Verify a tour has a total weight of no greater than x.
For M-Coloring:
Verify that a coloring of a graph uses at most M colors.
NP = Set of all decision problems that can be verified by a polynomial time algorithm
NP = Set of all decision problems that can be solved by a polynomial-time non-deterministic algorithm

19. Non-deterministic Algorithms
2 Stages
Guessing (Nondeterministic) stage
Simply guesses a solution to the instance
Verification (Deterministic) stage
Takes a proposed solution from the guessing stage and decides yes/no
Note that the purpose of this type of algorithm is for theory and classification – there are usually much better ways to actually implement the algorithm
Non-deterministic algorithm solves a decision problem if …

20. The Set NP
The set NP is the set of all decision problems that can be solved by a polynomial-time non-deterministic algorithm
The verification stage can be accomplished in poly-time

21. NP There are thousands of problems that have been proven to be in NP
Further note that all problems in P are also in NP
The guessing stage can do anything it wants
The verification stage can just run the algorithm
The only problems proven to not be in NP are the intractable ones
And there are only a few of these

22. P and NP NP P Here is the way the picture of the sets is usually drawn
We know that P is a subset of NP
We don’t know if it is a proper subset NP P

23. P and NP
No one has ever proven that there exists a problem in NP that is not in P
So, NP – P could be an empty set
If it is then we say that P = NP
The question of whether P = NP is one of the more intriguing and important questions in all of Computer Science

24. P = NP?
To prove that P  NP we would have to find a single problem in NP that is not in P
To prove P = NP we would have to find a poly-time algorithm for each problem in NP
If you prove either you get an A in the class, not to mention famous
And if you prove P = NP then you also become rich!

25. NP-Complete Problems
Recall: “To prove P = NP we would have to find a poly-time algorithm for each problem in NP”
This can be greatly simplified by looking at the class of NP-Complete Problems.
Examples: CNF-Sat, TSP, 0-1 Knapsack, M-Coloring, Clique,…

26. CNF-Satisfiability Problem
Given a logical expression in CNF, determine whether there is some truth assignment (some set of assignments of true and false to the variables) that makes the whole expression true
(x1  x2)  (x2  x3)  (x2)
Yes, x1 = T, x2 = F, x3 = F
(x1  x2)  x1  x2 No

27. CNF-Satisfiability Problem
It is easy to write a poly-time algorithm that takes as input a logical expression in CNF and a set of true assignments a verifies if the expression is true for that assignment
Therefore, the problem is in the set NP
Further, no one has ever found a poly-time algorithm for this entire problem and no one has ever proven that it cannot be solved in poly-time
So we do not know if it is in P or not

28. CNF-Satisfiability Problem
However, in 1971 Stephen Cook published a paper proving that if CNF-Satisfiability is in P, then P = NP
WOW! If we can just prove that this simple problem is in P then suddenly all NP problems are in P
And all sorts of things like encryption will break!

29. NP-Complete Problems
Class of NP problems that all have the same “difficulty”
To prove an NP problem is NP-Complete you must show that every other problem in NP can be “reduced” to this problem.
Cook showed that CNF-Sat was NP-Complete

30. More NP-Complete Problems
Now we can use transitivity of a reduction to get:
A problem C is NP-complete if:
It is in NP and
For some other NP-complete problem B, B reduces to C
Researchers have spent the last 30 years creating transformation for these problems and we now have a list of hundreds of NP-complete problems
If any of these NP-complete problems can be proven to be in P then P = NP
And, additionally, we also have a way to solve all the NP problems (poly-time transformations to the poly-time problem)

31. The State of P and NP All this sounds promising, but…
Over the last 40 years no one has been able prove that any problem from NP is not in P
Over the last 40 years no one has been able to prove that any problem from NP-complete is in P
Proving either of these things would give us an answer to the open problem of P = NP?
Most people seem to believe that P  NP