1. CS154, Lecture 13: P vs NP

2. The EXTENDED Church-Turing Thesis
 Polynomial-Time Turing Machines
Everyone’s Intuitive Notion of Efficient Algorithms
More generally: TM can simulate every “reasonable” model of computation with only polynomial increase in time
This extended church-turing thesis has merit: whatever a single-tape TM can do in polytime, a random access machine can do in polytime, a multitape TM can do in polytime, and so on.
A controversial thesis! Potential counterexamples: quantum algorithms

3. Nondeterministic Turing Machines
…are just like standard TMs, except:
1. The machine may proceed according to several possible transitions (like an NFA)
2. The machine accepts an input string if there exists an accepting computation history for the machine on the string

5. Definition: A nondeterministic TM is a 7-tuple T = (Q, Σ, Γ, , q0, qaccept, qreject), where:
Q is a finite set of states
Σ is the input alphabet, where   Σ
Γ is the tape alphabet, where   Γ and Σ  Γ
 : Q  Γ → 2(Q  Γ  {L,R})
q0  Q is the start state
qaccept  Q is the accept state
qreject  Q is the reject state, and qreject  qaccept

6. Accepting computation histories for N on w
Defining Acceptance for NTMs
Let N be a nondeterministic Turing machine
An accepting computation history for N on w is a sequence of configurations C0,C1,…,Ct where
1. C0 is the start configuration q0w,
2. Ct is an accepting configuration,
3. Each configuration Ci yields Ci+1
Accepting computation histories for N on w
do not have to be unique!
Def. N(w) accepts in t time  Such a history exists
N has time complexity T(n) if for all n, for all inputs of length n and for all histories, N halts in T(n) time

7. TIME(t(n))  NTIME(t(n)) Is TIME(t(n)) = NTIME(t(n)) for all t(n)?
Definition: NTIME(t(n)) =
{ L | L is decided by a O(t(n)) time nondeterministic Turing machine }
TIME(t(n))  NTIME(t(n))
Is TIME(t(n)) = NTIME(t(n)) for all t(n)?
Indeed, we can think of the usual, deterministic Turing machines as being a “special-case” of the nondeterministic model, where you just so happen to only have one possible transition out of each state and symbol you read.
It is known that TIME(O(n)) != NTIME(O(n)) for multitape TMs

8. What problems can we efficiently solved nondeterministically, but not

9. The Clique Problem k-clique = complete subgraph on k nodes a d f c b e
Finding cliques is a useful operation in social networks, computing similarities between biological structures, finding error correcting codes, there are all kinds of applications.
For example, this graph here has a 5-clique…
k-clique = complete subgraph on k nodes

10. The Clique Problem Find a clique of 1 million nodes?
Finding cliques is a useful operation in social networks, computing similarities between proteins, finding error correcting codes, there are all kinds of applications.
For example, this graph here has a 5-clique…
Find a clique of 1 million nodes?

11. CLIQUE = { (G,k) | G is an undirected graph with a k-clique }
Assume a reasonable encoding of graphs (example: the adjacency matrix is reasonable)
CLIQUE = { (G,k) | G is an undirected graph with a k-clique }
Theorem: CLIQUE  NTIME(nc) for some c > 1
N((V,E),k): Nondeterministically guess a subset S of V with |S| = k
For all u, v in S, if (u,v) is not in E then reject
Accept
Now it would seem that maybe this problem may not be all that hard. After all, it certainly LOOKS easy!

12. The Hamiltonian Path Problemc d f h i g
There are a lot of problems where you would like to search for a small object that’s hiding in another object.
NP includes all of these problems, and more.
There are many books written solely on the Hamiltonian Path problem and its relative, the Traveling Salesperson problem, where distances are added to the edges. There are applications all over industry in unexpected ways – I have seen an application where the order in which holes are to be punched on a circuit board was determined by solving a problem like Hamiltonian Path, and the difference saved a company tens of millions of dollars.
A Hamiltonian path traverses through each node exactly once

13. HAMPATH = { (G,s,t) | G is a directed graph with
HAMPATH = { (G,s,t) | G is a directed graph with a Hamiltonian path from s to t }
Theorem: HAMPATH  NTIME(nc) for some c > 1
N((V,E),s,t): Nondeterministically guess a sequence v1, …, v|V| of vertices
If vi = vj for some i ≠ j, reject
For all i = 1,…,|V|-1,
if (vi,vi+1) is not in E then reject
If (v1 = s & vn = t) then accept else reject

14.  NP = NTIME(nk) k  N Nondeterministic Polynomial Time
The class of problems that can be “feasibly solved” by nondeterministic Turing machines is called NP.
NP = “Nondeterministic Polynomial Time”

15. L = { x |  y ϵ Σ* [|y| ≤ |x|k and V(x,y) accepts ] }
Theorem: L  NP  There is a constant k and polynomial-time TM V such that
L = { x |  y ϵ Σ* [|y| ≤ |x|k and V(x,y) accepts ] }
Proof:
If L = { x | y |y| ≤ |x|k and V(x,y) accepts } then L  NP
Define the NTM N(x): Guess y of length at most |x|k
Run V(x,y) and output answer
Then, L(N) is the set of x s.t. [|y| ≤ |x|k & V(x,y) accepts]
(2) If L  NP then L = { x | y |y| ≤ |x|k and V(x,y) accepts }
Before I go any further discussing NP, I want to show you an extremely useful and enlightening characterization of NP, in terms of deterministic polynomial time.
Recall we had this characterization of recognizable languages…
So we have this analogy: P ~~ feasibly decidable, NP ~~ feasibly recognizable
Suppose N is a poly-time NTM that decides L. Define V(x,y) to accept iff y encodes an accepting computation history of N on x

16. there are polynomial-length proofs
A language L is in NP
if and only if
there are polynomial-length proofs
(aka. certificates or witnesses) for membership in L
CLIQUE =
{ (G,k) |  subset of nodes S such that S is a k-clique in G }
So we can rephrase CLIQUE and HAMPATH
NP = “Nifty Proof”
HAMPATH =
{ (G,s,t) |  Hamiltonian path in graph G
from node s to node t }

17. Boolean Formula Satisfiability
logical
operations
 recedes
 precedes 
parentheses
 =
(x  y)  z
Boolean variables (0 or 1)
A totally different NP problem, but still very fundamental problem, comes from Boolean logic.
not recedes and precedes or
To give you a good example of what nondeterminism can do, let’s briefly review Boolean formulas and logic.
To give another example, look at this formula, is it satisfiable?
YES, set x=0, y=0, z=1.

18. Boolean Formula Satisfiability
 =
(x  y)  z
A satisfying assignment is a setting of the variables that makes the formula true
x = 1, y = 1, z = 1 is a satisfying assignment for 
(in fact, any assignment with z = 1 is satisfying)
 = (x  y)  (z  x)
A totally different NP problem, but still very fundamental problem, comes from Boolean logic.
To give you a good example of what nondeterminism can do, let’s briefly review Boolean formulas and logic.
To give another example, look at this formula, is it satisfiable?
YES, set x=0, y=0, z=1. 1

19. A Boolean formula is satisfiable if there is a true/false setting to the variables that makes the formula true
a  b  c  d
YES
(x  y)  x NO
SAT = {  |  is a satisfiable Boolean formula }

20. A 3cnf-formula has the form:
(x1  x2  x3)  (x4  x2  x5)  (x3  x2  x1)
clauses
literals
*Why* do we define 3SAT and SAT like this?
Don‘t we want to *find* satisfying assignments, not just decide if they exist? Given an algorithm for deciding 3SAT, we can get another algorithm that prints satisfying assignments when they exist, and runs in about the same time. (put on homework)
1st one: This is a 3cnf formula: we can have repetitions of variables
2nd one: this is technically not 3cnf, one of its clauses has 2 variables… but we can make it 3cnf by simply duplicating a variable, so we generally don’t worry about clauses with 1 or 2 variables.
3rd one: this is also a 3cnf, and it can be simplified: when we have an x1 and not(x1) in the same clause, that clause has to be true no matter what we assign, so we could remove the third clause and have an equivalent formula.
4th one: this is not 3cnf, but we can make it 3cnf… by replacing x3 with the clause (x3 or x3 or x3), not(x2) by (not(x2) or not(x2) or not(x2))…
3SAT = {  |  is a satisfiable 3cnf-formula }

21. 3SAT = {  |  is a satisfiable 3cnf-formula }
Theorem: 3SAT  NP
We can express 3SAT as
3SAT =
{  |  is in 3cnf and  string y that encodes a satisfying assignment to  }
The number of variables of  is at most ||, so |y| ≤ ||.
Then, argue that the language
3SAT-CHECK = {(,y) |  is in 3cnf and y is a satisfying assignment to }
is in P.
(Similarly, SAT  NP)
For multitape TMs we can get that 3SAT is in NTIME[n log n].

22. NP = Problems with the property that, once you have the solution, it is “easy” to verify the solution
When  ϵ SAT,
or (G, k) ϵ CLIQUE,
or (G,s,t) ϵ HAMPATH,
Can prove that with a short proof that can easily been verified
How about when Phi is NOT in SAT? In that case, I don’t know how to efficiently prove to you that a formula is NOT satisfiable. In order to do that, I have to convince you that for every possible assignment to the formula \Phi, the formula evaluates to false on *all* of them. And I have to do it with something SHORT.
There are 2^n possible assignments on n variables, but the requirement is that the proof is only poly(n) bits long.
What if  ∉ SAT? (G, k) ∉ CLIQUE? Or (G,s,t) ∉ HAMPATH?

23. P = the problems that can be efficiently solved
NP = the problems where proposed solutions can be efficiently verified
Is P = NP? can problem solving be automated?
Clay Math Institute in the year 2000: “millennium problems”

24. Mathematicians may be out of a job
If P = NP:
Mathematicians may be out of a job
Cryptography as we know it may be impossible
In principle, every aspect of life could be efficiently and globally optimized … life as we know it would be different!
Math out of a job:
Given a formal system F, define
Short-Provability_{F} = {T01^k | Theorem T has a proof in F of length <= k}
For most formal systems F, this problem is contained in NP: Given a proof P of T of length |T|^2, we can verify if it is a proof in polynomial in |P| time.
If P = NP, we can always find such proofs without exhaustive search over all proofs. You can tune this for specific formal systems to ask: is there a proof of the Riemann hypothesis that can be written down in less than 1000 pages? Or take any open problem in mathematics. Does it have a 100 page proof? If P=NP then such a question could be efficiently answered in practice, if the polynomial time algorithm had nice leading constants and a nice exponent.
Modern cryptography relies on the existence of things like one-way functions – functions which are easy to compute but hard to invert. But if P=NP then any polytime computable function can be inverted in polytime: Given y, guess x and verify that f(x) = y.
All this looks too good to be true!
But how on earth can we prove it? We have to rule out all possible polynomial time algorithms for solving generic search problems.
It’s likely that P != NP, and I also think it’s likely that the proof will be a proof by contradiction: someone will assume P=NP and derive so many fantastic consequences that they get a contradiction.
So when I say “it’s hard to imagine how P=NP” that’s because I don’t know how to prove P != NP. Perhaps if you can imagine better than the rest of us, you will find the contradiction that we have all been missing!
However, we can still make progress in understanding computation by assuming P != NP, and asking what can we still do in the world.
It may well be false, but many of the theorems we prove assuming the conjecture will still give us information about how best to solve problems, given what we know.
Conjecture: P  NP

25. Polynomial Time Reducibility
f : Σ*  Σ* is a polynomial time computable function
if there is a poly-time Turing machine M that on every input w, halts with just f(w) on its tape
Language A is poly-time reducible to language B, written as A P B, if there is a poly-time computable f : Σ*  Σ* so that:
w  A  f(w)  B
Recall that we found relationships between undecidable problems by using computable reductions. We showed that many problems can be expressed in terms of other problems.
We can try to get a handle on the relationships between NP problems by defining an efficient notion of a computable reduction, a polytime reduction.
And the definition is very similar to that of a computable reduction, except we throw in “polytime”
f is a polynomial time reduction from A to B
Note there is a k such that for all w, |f(w)| ≤ |w|k

26. f converts any string w into a string f(w) such that w  A  f(w)  Bnc n w
f(w) f n nc
f converts any string w into a string f(w) such that w  A  f(w)  B

27. Theorem: If A P B and B P C, then A P Cg C n nc
ncd
Suppose f runs in time n^c, g runs in time n^d.
The first is that this <= relation acts like a real inequality relation.
Why is the composition of f with g a poly-time reduction from A to C?
Because the composition of a polynomial with another polynomial, is still a polynomial.

28. Theorem: If A P B and B  P, then A  P
Proof:
Let MB be a poly-time TM that decides B.
Let f be a poly-time reduction from A to B.
We build a machine MA that decides A as follows:
MA = On input w,
1. Compute f(w)
2. Run MB on f(w), output its answer
w  A  f(w)  B

29. Theorem: If A P B and B  NP, then A  NP
Proof:
Analogous…

30. Theorem: If A P B and B  P, then A  P
Theorem: If A P B and B  NP, then A  NP
Corollary: If A P B and A ∉ P, then B ∉ P
So we could potentially use this theorem to prove positive results and negative results, analogous to what we saw in computability theory.
We could prove that some problem cannot be solved in polynomial time: if we have a “hard” problem A and can reduce it in polynomial time to a problem B, then B must also be “hard”