1. CS151 Complexity Theory
Lecture 3
April 10, 2017

2. Padding and succinctness
Two consequences of measuring running time as function of input length:
“padding”
suppose L  EXP, and define
PADL = { x#N : x  L, N = 2|x|k }
TM that decides PADL: ensure suffix of N #s, ignore #s, then simulate TM that decides L
running time now polynomial !
April 10, 2017

3. Padding and succinctness
converse (intuition only): “succinctness”
suppose L is P-complete
intuitively, some inputs are “hard” -- require full power of P
SUCCINCTL has inputs encoded in different form than L, some exponentially shorter
if “hard” inputs are exponentially shorter, then candidate to be EXP-complete
April 10, 2017

4. Succinct encodings
succinct encoding for a directed graph G= (V = {1,2,3,…}, E):
a succinct encoding for a variable-free Boolean circuit:
1 iff (i, j)  E i j
1 iff wire from gate i to gate j
type of gate i
type of gate j i j
April 10, 2017

5. An EXP-complete problem
Succinct Circuit Value: given a succinctly encoded variable-free Boolean circuit (gates , , , 0, 1), does it output 1?
Theorem: Succinct Circuit Value is EXP-complete.
Proof:
in EXP (why?)
L arbitrary language in EXP, TM M decides L in 2nk steps
April 10, 2017

6. An EXP-complete problem
tableau for input x = x1x2x3…xn:
Circuit C from CVAL reduction has size O(22nk).
TM M accepts input x iff circuit outputs 1
x _ _ _ _ _
height, width 2nk
April 10, 2017

7. An EXP-complete problem
Can encode C succinctly:
if i, j within single STEP circuit, easy to compute output
if i, j between two STEP circuits, easy to compute output
if one of i, j refers to input gates, consult x to compute output
1 iff wire from gate i to gate j
type of gate i
type of gate j i j
April 10, 2017

8. Summary Remaining TM details: big-oh necessary.
First complexity classes:
L, P, PSPACE, EXP
First separations (via simulation and diagonalization):
P ≠ EXP, L ≠ PSPACE
First major open questions:
L = P P = PSPACE
First complete problems:
CVAL is P-complete
Succinct CVAL is EXP-complete ? ?
April 10, 2017

9. Summary
EXP
PSPACE P L
April 10, 2017

10. Nondeterminism: introduction
A motivating question:
Can computers replace mathematicians?
L = { (x, 1k) : statement x has a proof of length at most k }
April 10, 2017

11. Nondeterminism: introduction
Outline:
nondeterminism
nondeterministic time classes
NP, NP-completeness, P vs. NP
coNP
NTIME Hierarchy
Ladner’s Theorem
April 10, 2017

12. Nondeterminism Recall deterministic TM Q finite set of states
∑ alphabet including blank: “_”
qstart, qaccept, qreject in Q
transition function:
δ : Q x ∑ ! Q x ∑ x {L, R, -}
April 10, 2017

13. Nondeterminism nondeterministic Turing Machine:
Q finite set of states
∑ alphabet including blank: “_”
qstart, qaccept, qreject in Q
transition relation
∆  (Q x ∑) x (Q x ∑ x {L, R, -})
given current state and symbol scanned, several choices of what to do next.
April 10, 2017

14. Nondeterminism
deterministic TM: given current configuration, unique next configuration
qstartx1x2x3…xn
qstartx1x2x3…xn
x L
x  L
qaccept
qreject
nondeterministic TM: given current configuration, several possible next configurations
April 10, 2017

15. Nondeterminism asymmetric accept/reject criterion qstartx1x2x3…xn
“computation path”
x  L
x  L
“guess”
qaccept
qreject
asymmetric accept/reject criterion
April 10, 2017

16. Nondeterminism all paths terminate
time used: maximum length of paths from root
space used: maximum # of work tape squares touched on any path from root
April 10, 2017

17. Nondeterminism
NTIME(f(n)) = languages decidable by a multi-tape NTM that runs for at most f(n) steps on any computation path, where n is the input length, and f :N ! N
NSPACE(f(n)) = languages decidable by a multi-tape NTM that touches at most f(n) squares of its work tapes along any computation path, where n is the input length, and f :N ! N
April 10, 2017

18. Nondeterminism Focus on time classes first: NP = k NTIME(nk)
NEXP = k NTIME(2nk)
April 10, 2017

19. Poly-time verifiers Very useful alternate definition of NP:
“witness” or “certificate”
Very useful alternate definition of NP:
Theorem: language L is in NP if and only if it is expressible as:
L = { x| 9 y, |y| ≤ |x|k, (x, y)  R }
where R is a language in P.
poly-time TM MR deciding R is a “verifier”
efficiently verifiable
April 10, 2017

20. R = {(φ, A) : A is a sat. assign. for φ}
Poly-time verifiers
Example: 3SAT expressible as
3SAT = {φ : φ is a 3-CNF formula for which  assignment A for which (φ, A)  R}
R = {(φ, A) : A is a sat. assign. for φ}
satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ)
R is decidable in poly-time
April 10, 2017

21. L = { x | 9 y, |y| ≤ |x|k, (x, y)  R }
Poly-time verifiers
L = { x | 9 y, |y| ≤ |x|k, (x, y)  R }
Proof: () give poly-time NTM deciding L
phase 1: “guess” y with |x|k nondeterministic steps
phase 2: decide if
(x, y)  R
April 10, 2017

22. Poly-time verifiers Proof: () given L  NP, describe L as:
L = { x | 9 y, |y| ≤ |x|k, (x, y)  R }
L is decided by NTM M running in time nk
define the language
R = { (x, y) : y is an accepting computation history of M on input x}
check: accepting history has length ≤ |x|k
check: R is decidable in polynomial time
check: M accepts x iff y, |y| ≤ |x|k, (x, y)  R
April 10, 2017

23. Why NP? not a realistic model of computation
problem requirements
object we are seeking
not a realistic model of computation
but, captures important computational feature of many problems:
exhaustive search works
contains huge number of natural, practical problems
many problems have form:
L = { x | 9 y s.t. (x,y) 2 R}
efficient test: does y meet requirements?
April 10, 2017

24. Why NP? Why not EXP? too strong! important problems not complete.
April 10, 2017

25. Relationships between classes
Easy: P µ NP, EXP µ NEXP
TM special case of NTM
Recall: L  NP iff expressible as
L = { x | 9 y, |y| · |x|k s.t. (x,y) 2 R}
NP µ PSPACE (try all possible y)
The central question:
P = NP
finding a solution vs. recognizing a solution ?
April 10, 2017

26. NP-completeness
Circuit SAT: given a Boolean circuit (gates , , ), with variables y1, y2, …, ym is there some assignment that makes it output 1?
Theorem: Circuit SAT is NP-complete.
Proof:
clearly in NP
April 10, 2017

27. NP-completeness Given L  NP of form L = { x | 9 y s.t. (x,y) 2 R}… xn y1 y2 … ym
1 iff (x,y)  R
CVAL reduction for R
hardwire input x; leave y as variables
April 10, 2017

28. NEXP-completeness
Succinct Circuit SAT: given a succinctly encoded Boolean circuit (gates , , ), with variables y1, y2, …, ym is there some assignment that makes it output 1?
Theorem: Succinct Circuit SAT is NEXP-complete.
Proof:
same trick as for Succinct CVAL EXP-complete.
April 10, 2017

29. Complement classes In general, if C is a complexity class
co-C is the complement class, containing all complements of languages in C
L  C implies (* - L)  co-C
(* - L)  C implies L  co-C
Some classes closed under complement:
e.g. co-P = P
April 10, 2017

30. coNP Is NP closed under complement? Can we transform this machine:
x  L
Can we transform this machine:
x  L
x  L
x  L
qaccept
qreject
into this machine?
qaccept
qreject
April 10, 2017

31. L = { x | 9 y, |y| ≤ |x|k, (x, y)  R }
coNP
“proof system” interpretation:
Recall: L  NP iff expressible as
L = { x | 9 y, |y| ≤ |x|k, (x, y)  R }
“proof verifier”
“proof”
languages in NP have “short proofs”
coNP captures (in its complete problems) problems least likely to have “short proofs”.
e.g., UNSAT is coNP-complete
April 10, 2017

32. coNP NP ≠ coNP P = NP implies NP = coNP Belief:
another major open problem
April 10, 2017

33. NTIME Hierarchy Theorem
Theorem (Nondeterministic Time Hierarchy Theorem):
For every proper complexity function f(n) ≥ n, and g(n) = ω(f(n+1)),
NTIME(f(n)) ( NTIME(g(n)).
April 10, 2017

34. NTIME Hierarchy Theorem
inputs
Proof attempt :
(what’s wrong?) Y
Turing Machines n
(M, x): does NTM M accept x in f(n) steps? Y n n Y n
D : n Y n Y Y n Y
April 10, 2017

35. NTIME Hierarchy Theorem
Let t(n) be large enough so that can decide if NTM M running in time f(n) accepts 1n, in time t(n).
I’m responsible for dealing with NTM Mi 1n
1t(n)
. . . M1 y n y ? ? ? ?
...
...
Mi-1 n y n ? ? ? ? Mi n y y ? ? ? ?
. . .
D :
April 10, 2017

36. NTIME Hierarchy Theorem
Enough time on input 1t(n) to do the opposite of Mi(1n): 1n
1t(n)
. . . Mi y ? ? ? ?
D :
. . . n
April 10, 2017

37. NTIME Hierarchy Theorem
For k in [n…t(n)] can to do same as Mi(1k+1) on input 1k 1n
1t(n)
. . . Mi y ? ? ? ?
D :
. . . n
April 10, 2017

38. NTIME Hierarchy Theorem
Did we diagonalize against Mi?
if L(Mi) = L(D) then:
equality along all arrows.
contradiction. 1n
1t(n)
. . . Mi y ? ? ? ?
D :
. . . n
April 10, 2017

39. NTIME Hierarchy Theorem
General scheme:
interval [1...t(1)] kills M1
interval [t(1)…t(t(1))] kills M2
interval [ti-1(1)…ti(1)] kills Mi
Running time of D on 1n: f(n+1) + time to compute interval containing n
conclude D in NTIME(g(n)) (g(n) = ω(f(n+1)))
April 10, 2017

40. Ladner’s Theorem
Assuming P ≠ NP, what does the world (inside NP) look like?
NP:
NP:
NPC
NPC P P
April 10, 2017

41. Ladner’s Theorem
Theorem (Ladner): If P ≠ NP, then there exists L  NP that is neither in P nor NP-complete.
Proof: “lazy diagonalization”
deal with similar problem as in NTIME Hierarchy proof
April 10, 2017

42. Ladner’s Theorem Can enumerate (TMs deciding) all languages in P.
enumerate TMs so that each machine appears infinitely often
add clock to Mi so that it runs in at most ni steps
April 10, 2017

43. Ladner’s Theorem
Can enumerate (TMs deciding) all NP-complete languages.
enumerate TMs fi computing all polynomial-time functions
machine Ni decides language SAT reduces to via fi if fi is reduction, else SAT (details omitted…)
April 10, 2017

44. Ladner’s Theorem Our goal: L  NP that is neither in P nor NP-completeMi : : M0 L
inputs N0 : : Ni
April 10, 2017

45. Ladner’s Theorem Top half, assuming P ≠ NP: focus on Mi
for any x, can always find some z ≥ x on which Mi and SAT differ (why?) Mi : : M0
SAT
input x
input z
April 10, 2017

46. Ladner’s Theorem Bottom half, assuming P ≠ NP: TRIV = Σ* input z
focus on Ni
for any x, can always find some z ≥ x on which Ni and TRIV differ (why?)
input x
TRIV N0 : : Ni
April 10, 2017

47. Ladner’s Theorem Try to “merge”: on input x, either . . .
answer SAT(x)
answer TRIV(x)
if can decide which one in P, L  NP Mi
SAT
TRIV : : M0 ≠ ≠
. . . L ≠ ≠ N0 : : Ni
April 10, 2017

48. Ladner’s Theorem General scheme: f(n) slowly increasing function
f(|x|) even: answer SAT(x)
f(|x|) odd: answer TRIV(x)
notice choice only depends on length of input… that’s OK
. . .
SAT L
f(|x|) 1 1 1 2 2 2 2
. . .
TRIV
April 10, 2017

49. Ladner’s Theorem 1st attempt to define f(n)
“eager f(n)”: increase at 1st opportunity
Inductive definition: f(0) = 0; f(n) =
if f(n-1) = 2i, trying to kill Mi
if 9 z < 1n s.t. Mi(z) ≠ SAT(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1)
if f(n-1) = 2i+1, trying to kill Ni
if 9 z < 1n s.t. Ni(z) ≠ TRIV(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1)
April 10, 2017

50. Ladner’s Theorem Problem: eager f(n) too difficult to compute
on input of length n,
look at all strings z of length < n
compute SAT(z) or Ni(z) for each !
Solution: “lazy” f(n)
on input of length n, only run for 2n steps
if enough time to see should increase (over f(n-1)), do it; else, stay same
(alternate proof: give explicit f(n) that grows slowly enough…)
April 10, 2017

51. Ladner’s Theorem
I’m sup-posed to ensure Mi is killed
I finally have enough time to check input z
I notice z did the job, increase f to k+1
Key: n eventually large enough to notice completed previous stage Mi ≠
. . . L
. . . f 1 1 k k k k k
input z < x
input x
suppose k = 2i
April 10, 2017

52. Ladner’s Theorem Inductive definition of f(n) f(0) = 0
f(n): for n steps compute f(0), f(1), f(2),…
. . .
. . . L
. . .
. . . f 1 1 k k k
got this far in n steps
input x, |x| = n
April 10, 2017

53. Ladner’s Theorem if k = 2i:
for n steps try (lex order) to find z s.t. SAT(z) ≠ Mi(z) and f(|z|) even
if found, f(n) = f(n-1)+1 else f(n-1)
if k = 2i + 1:
for n steps try (lex order) to find z s.t. TRIV(z) ≠ Ni(z) and f(|z|) odd
April 10, 2017

54. L = { x | x  SAT if f(|x|) even,
Ladner’s Theorem
Finishing up:
L = { x | x  SAT if f(|x|) even,
x  TRIV if f(|x|) odd }
L  NP since f(|x|) can be computed in O(n) time
April 10, 2017

55. Ladner’s Theorem suppose Mi decides L suppose Ni decides L
f gets stuck at 2i
L  SAT for z : |z| > no
implies SAT  P. Contradiction.
suppose Ni decides L
f gets stuck at 2i+1
L  TRIV for z : |z| > no
implies L(Ni)  P. Contradiction.
(last of diagonalization…)
April 10, 2017