1. Complexity Theory Lecture 6
Give the handouts (course web page)
Lecturer: Moni Naor

2. Recap Last week: Probabilistic Complexity Plus Alternation This Week:
Schwatz-Zippel
Approximating Counting Problems
Plus
Alternation
This Week:
Non-Uniform Complexity Classes
Polynomial Time Hierarchy

3. Machines that Take Advice
Turing Machine with advice: extra read only tape
Advice string
Machine M decides language L with advice A(n) if:
X 2 L implies M(x,A(|x|) = `yes’
X  L implies M(x,A(|x|) = `no’
Advice A(n) is specific to the length
Complexity Class with advice: C is a complexity class
and f:N  N.
Language L 2 C/f(n) if there is a TM M and advice A(n) that decides language L and
|A(n)| · f(n)
Example: P/Poly(n) = [k P/nK

4. P/Poly Every unary language is in P/poly
{1index(x) | xL} where L is undecidable
Every unary language is in P/poly
There are undecidable languages in P/poly
Observation: a language is in P/poly iff it has a polynomial sized circuit
There exists a sequence of circuits C1, C2, … and a polynomial p(n) such that:
circuit Cn computes f on inputs of size n
circuit Cn is of size at most p(n)
What is the power of P/poly
is NP 2 P/poly?
Homework: show that if NP 2 P/log then P=NP

5. Circuits Let B be a collection of Booleans functions
A Circuit for on n Boolean variables x1, x2, … xn over Basis B is a DAG with
Each source is labeled with a literal
Unique sink
Each internal node is labeled with a function from B and has the appropriate indegree
A circuit compute a function in the natural way
Which (families of) functions have circuits whose size is bounded by polynomial in the input size?
B ={:, Ç, Æ}
Claim: all f 2 P have polynomial size circuits
Proof: via the `usual’ tableau. Size of circuit T(n)S(n).
Using Oblivious Turing Machines T(n) log T(n)
Claim: most Boolean function f on n variables do not have polynomial sized circuits
For any polynomial p(n) most functions need more p(n) gates
Need (2n/n) gates to compute all functions on n variables, which is tight
Can determine the inputs to a given gate by a log space machine

6. Polynomial Sized Circuits for BPP
Theorem: any f 2 BPP has a polynomial size circuit
There exists a sequence of circuits C1, C2, … and a polynomial p(n) such that:
circuit Cn computes f on inputs of size n
circuit Cn is of size at most p(n)
Several proofs (various derandomization methods):
Construction of hitting set (for f 2 RP)
Claim that a sequence r1, r2, … rn exists where for each x 2 L at least one ri says `yes’
Simulating large sample spaces by small ones
Amplification
Reduce the error so that a single assignment will be good for all x

7. Pr[[x Ax] · x Pr[Ax] < 2n 2-2n=1
Derandomization I
Theorem: any f 2 BPP has a polynomial size circuit
Simulating large sample spaces
Want to find a small collection of strings on which the PTM behaves as on the large collection
If the PTM errs with probability at most , then should err on at most + of the small collection
Choose m random strings
For input x event Ax is more than (+) of the m strings fail the PTM
Pr[Ax] · e-22m < 2-2n
Pr[[x Ax] · x Pr[Ax] < 2n 2-2n=1
Collection that should resemble probability of success on ALL inputs
Bad 
Good 1-
Chernoff

8. Derandomization A major research question:
How to make the construction of
Small Sample space `resembling’ large one
Hitting sets
Efficient.
Successful approach: randomness from hardness
(Cryptographic) pseudo-random generators
Complexity oriented pseudo-random generators

9. Alternation
Non-determinism: a configuration leads to acceptance if there is an accepting leaf in the computation
Similar to Or
Can also consider leads to acceptance if all leaves are accepting
Similar to And
What if we alternate between the modes

10. Alternating Turing Machines
An Alternating Turing Machines (ATM) is a non-deterministic TM whose state S ={SAND [ SOR} are partitioned into two sets
For input x consider the tree of computations where each node is a configuration
The ATM is accepting if the root leads to an accepting configuration:
A leaf is accepting or not
A node in state s 2 SAND leads to acceptance if both its children leads to acceptance
A node in state s 2 SOR leads to acceptance if one of its children leads to acceptance

11. Alternating Time Classes
ATIME(f(n))= class of language decided by an ATM where:
All computations halt after at most f(|x|) steps
ASPACE(f(n))= class of language decided by an ATM where:
All computation halt
Never use more than f(|x|) cells
AL = ASPACE(log n)
Theorem: AL = P
Theorem: by simulating circuit
Important point: if f 2 P then the circuit is constructable in log space
Simulate the circuit from the output towards the inputs

12. Oracle Machines
Recall: Oracle Turing Machine (OTM) have a special query tape where can ask membership queries and obtain answers
Oracle for A – can ask whether x 2 A
Oracle Machine with oracle for A: MA
Can consider time classes with oracle CA
PA ={L|there is an OTM MA that decides L in polynomial time}
NPA ={L|there is an OTM MA that decides L in non deterministic polynomial time}
Is PSAT = NPSAT ? Is PPSPACE = NPPSPACE ?
Can consider Time Classes with oracle from another time class
PNP ={L|there is an OTM M AND a language A2 L where MA decides L in polynomial time}
Homework

13. Motivation: minimizing circuits
Central problem in logic synthesis:
What is the complexity of this problem?
NP-hard?
Is it in NP, coNP, PSPACE?
Complete for any of these classes? 
Given a Boolean circuit C
Is there a circuit C’ of size smaller than C computing the same function that C does?      x1 x2 x3 … xn

14. The Polynomial-Time Hierarchy
Can define many complexity classes using oracles
Concentrate on classes that
have natural complete problems
have a natural interpretation in terms of alternating quantifiers
help state consequences and containments

15. The Polynomial Time Hierarchy Version I: Oracles
0P = 0P = 0P =P
i+1P = PiP
Si+1P = NPiP
Pi+1P = coNPiP
PH = [i ¸ 0 SiP
Notation sometimes iP

16. Polynomial time hierarchy
First level of the hierarchy:
1P = P 0P = P
S1P = NP 0P = NP
P1P = CoNP 0P = CoNP
If we believe the first level classes to be different from each other (and from the zero-level class(es)), are other levels also different?
Does equality at one level imply one at the other levels?
Are there interesting problems higher than the first level?

17. Polynomial time hierarchy
Proposition: iPiP  i+1P  i+1Pi+1P
Proof:
iP iP  i+1P=PiP
Example NPcoNP  PNP=2P
We have an oracle for L  iP
Ask the oracle and return answer for iP
Ask the oracle and return the opposite for iP
(which is co-iP).

18. Polynomially Balanced Relations
A binary relation R µ {0,1}n £ {0,1}n is polynomially balanced if
(x,y) 2 R, then |y|· |x|k for some k
Characterization of NP in terms of witnesses:
Claim: L 2 NP iff there is a polynomially balanced relation such that
R2 P
L={x| 9 y such that (x,y) 2 R}
Claim: L 2 CoNP iff there is a polynomially balanced relation R2 P such that
L={x| 8 y such that |y| · |x|k, (x,y) 2 R}

19. The Polynomial Time Hierarchy Version II: Quantifiers
Theorem: Let L be a language and i ¸ 1. L 2 SiP iff there is a polynomially balanced relation R such that
R 2 Pi-1P
and
L={x| 9 y such that (x,y) 2 R}
Proof: by induction on i
Interesting direction: show how to build a certificate checkable in Pi-1P from computation of NP machine with Si-1P oracle

20. Proof of Equivalence Proof of Theorem:
induction on i
Have seen base case
Oracle definition implies quantifier definition:
By definition ΣiP = NPΣi-1P = NPPi-1P
Given L={x| 9 y such that (x,y)2R} and R2Pi-1P to see why it is in ΣiP:
Guess y, ask oracle if (x, y)  R
Can do since ΣiP = NPPi-1P

21. Proof of Equivalence Quantifier definition implies oracle definition:
Given L  Σi = NPΣi-1 decided by an Oracle NTM M running in time nk
First try: R = {(x, y) | y describes a valid path of M’s computation}
Valid path = leading to qaccept
Problem: how to recognize valid computation path when it depends on results of oracle queries?

22. Proof of Equivalence
R = { (x, y) | y describes valid path of M’s computation}
Try
valid path = step-by-step description including correct yes/no answer for each A-oracle query zj (A  Σi-1P)
e.g: z1 A, z2 2 A, z3 A, … zm A
verify “no” queries in Pi-1P:
e.g: z1 A  z3 A  …  zm A
for each “yes” query zj , by induction.
9 wj, |wj| ≤ |zj|k with (zj, wj)  R’ for some R’  Pi-2P
For each “yes” query zj put wj in description of path y

23. Proof of Equivalence Single relation R in Pi-1P : (x, y)  R IFF
all “no” zj A and
all “yes” zj have (zj, wj)  R’ and
y is a path leading to qaccept.
Key Point: the AND of Pi-1 predicates is in Pi-1.

24. The Polynomial Time Hierarchy Version II
Corollary: Let L be a language and i ¸ 1. L 2 PiP iff there is a polynomially balanced relation R such that
{(x,y)|(x,y) 2 R} 2 Si-1P
and
L={x| 8y with |y| · |x|k we have (x,y) 2 R}
Proof: recall that Pi P is CoSiP
Corollary: Let L be a language and i ¸ 1. L 2 SiP iff there is a polynomially balanced and polynomially decidable (i+1)-ary relation R such that
L={x| 9 y1 8 y2 9 y3 …Q yi such that (x,y1, y2, … yi) 2 R}
Quantifier Q is 9 if i is odd and Q is 8 if i is even
Complete problem for iP

25. Collapse of the Hierarchy
Theorem: If for some i ¸ 1
SiP=PiP,
then for all j ¸ i
SjP=PjP
Proof: suffices to show SiP=PiP implies SiP=Si+1P
Corollary : If NP=CoNP then the polynomial hierarchy collapses to the first level
Similarly if P=NP

26. Interesting Languages in the hierarchy
Minimum equivalent problems
Minimum circuit: given a circuit C is it true that there is no circuit C’ with fewer gates that computes the same function as C
Claim: Minimum circuit is in P2P:
8 C’, |C’| < |C| 9 x such that C(x)  C’(x)
Minimum circuit is not known to be lower in the hierarchy or complete for P2P

27. Complete Problems for S2P
Minimum DNF: given DNF f, integer k; is there a DNF f‘ of size at most k computing same function f does?
Example:
x1x2x3  x1x3  x4  x1x2  x1x3  x4
Theorem: (Umans 98): Minimum DNF is Σ2P-complete

28. Complete Problems for D2P
Odd TSP: given a weighted graph G,
is the length of the shortest TSP tour in G odd?
Theorem: Odd TSP is Δ2P-complete
Homework: Show Odd TSP is in Δ2P

29. Oracles vs. Algorithms Given a polynomial time algorithm for SAT
Can you solve Minimum Circuit efficiently?
What other consequences?
Given an oracle for SAT
same input/output behavior as algorithm!
Key issue: is access to a program as a black box give you more power then given the code itself
Recent developments in cryptography show a difference (Barak 2001)

30. BPP in the Hierarchy Do not know whether BPP µ NP
Do not know whether BPP≠EXP
Theorem: BPP µ P2P S2P
Sufficient to show BPP µ S2P and then by symmetry BPP µ P2P
L 2 BPP: Poly time PTM M:
x  L  Pry[M(x,y) accepts] ≥ 2/3
x  L  Pry[M(x,y) rejects] ≥ 2/3
Consider a game between  and  players
 player wants to prove x  L and  player wants to prove x  L
Need to choose y cooperatively
First attempt:
 player chooses ρ 2 {0,1}n
 player chooses t 2 {0,1}n
Set y = ρ  t run M(x,y)
There is a winning strategy for  player as long as Pry[M(x,y) accepts] <1

31. [i=1,2,3 { t | M(x,y) accepts for y = ρi  t} = {0,1}n
BPP in the Hierarchy
Since game was unfair to  player give him several chances
Several (parallel) instances of the game
Force the  player to use the same choice in all instances
Second attempt:
 player chooses ρ1, ρ1,… ρm where each ρi 2 {0,1}n
 player chooses t 2 {0,1}n
Set yi = ρi  t run M(x,y1), M(x,y2),…, M(x,ym) accept if any of the runs accepts
Each ρi invalidates 2/3rds of possible t 2 {0,1}n
Invalidates = not a useful response for  player
Due to randomization by Xor
By a hitting set argument there is a set of ρ1, ρ2,… ρm that invalidates all t 2 {0,1}n for m= n log3 2
But what happens when x  L?
There might be {ρ1, ρ2, ρ3 } such that
[i=1,2,3 { t | M(x,y) accepts for y = ρi  t} = {0,1}n
Happens when bad witnesses are a coset of an ECC
Each of size at most 1/3

32. 1/ 2n’ |[i=1m { t | M(x,y) accepts for y = ρi  t}|
BPP in the Hierarchy
Can apply error reduction: Poly time PTM M’
use n’ random bits (|y’| = n’)
fraction of strings y’ for which M’(x, y’) is incorrect is at most 1/n’
Now by the hitting set argument there is a set of ρ1, ρ2,… ρm that invalidates all t 2 {0,1}n for m= n’ /log n’
But when x  L for any choice of {ρ1, ρ2,…, ρm }
1/ 2n’ |[i=1m { t | M(x,y) accepts for y = ρi  t}|
< n’ /log n’ ¢ 1/n’
Final form
 player chooses ρ1, ρ1,… ρm where each ρi 2 {0,1}n’
 player chooses t 2 {0,1}n’
Set yi = ρi  t run M’(x,y1), M’(x,y2),…, M’(x,ym) accept if any of the runs accepts
clearly a S2P process
Each of size at most 1/n’

33. More on BPP in the Hierarchy Simultaneous Provers
To prove BPP ½ S2P we considered a game between  and  players
 player wants to prove x  L and  player wants to prove x  L
In S2P setting first  makes a move and then  player (based on ’s move)
What if both players have to move simultaneously?
Want strategy where
 player wins whenever x  L even if  player makes move after  player
 player wins whenever x  L even if  player makes move after  player
Homework:
Phrase the above requirements in terms of a polynomial time relation R(x,y,z)
Show that for all L  BPP such a strategy exists
Canetti’s class S_2^P

34. Polynomial sized circuits for SAT
We know that P = NP implies SAT has poly-sized circuits (SAT  P/poly ).
Showing SAT does not have poly-size circuits is a research direction for proving P ≠ NP
Suppose SAT has poly-size circuits
Can we show implies P = NP ?
Instead show: SAT  P/poly  PH collapses (to second level)
similar implication as if SAT  P

35. L={x| |x|=n implies (x,Cn(x)) 2 R}
Self Reducibility
Lemma: If SAT  P/poly then there exists a polynomial p(n) and family of circuits C1, C2 … such that Cn gets an n sized formula  and outputs
A satisfying assignment if  is satisfiable
Null if  is not satisfiable
Corollary: If SAT  P/poly and R is polynomial time balanced relation, then for
L = { x | z (x,z) 2 R}
there is a family of circuits C1, C2 … such that
L={x| |x|=n implies (x,Cn(x)) 2 R}
Can define new polynomial relation R’ such that (x,C) 2 R’ iff (x,C(x))2R and C is of the right size.
Proof in Papadimitriou seems to be wrong

36. Collapse of The Hierarchy
Theorem: if SAT  P/poly then PH collapses to the second level.
Proof:
Show that P2P µ Σ2P
If L  P2P then we know:
L = { x | y z (x, y, z) 2 R }
for R  P.
“z (x, y, z)  R?” is in NP
Let Cn be the SAT solver – produces certificate z
z (x, y, z)  R iff (x, y, Cn(x,y))  R
L = { x | Cn y (x, y, Cn(x,y)) 2 R }
= { x | Cn y (x, y, Cn) 2 R’ }  S2P

37. Approximate Counting is in the Hierarchy
Theorem: For any f 2 #P there is a g 2 Δ3P such that on input x and  g(x, ) outputs a (1+ ) approximation to f(x)
(1-)g(x, ) · f(x) · (1+)g(x, )
Proof: key point – method for proving that sets are of certain size
The set in question:
A = Accept(M(x)) µ {0,1}m
A function f is in #P if there exists a NTM M
running in polynomial time where M(x) has f(x) accepting paths for all x 2 {0,1}n

38. Pair-wise Independent Hash Functions
A family of hash functions
H={h|h:{0,1}m  {0,1}ℓ }
is pair-wise independent if: for all x ≠ y, for random h 2RH the random variables h(x), h(y) are uniformly distributed and independent
In particular: Pr[h(x)=h(y)]= 1/2ℓ
Already saw the approximate version for distributed equality testing
Pr[h(x)=h(y)]· 
Constructing H:
Construction based on matrix multiplication.
Treat x as a m £ 1 vector.
The function is defined by a random m£ ℓ matrix over GF[2].
H={hA|A 2 GF[2]m£ ℓ and hA(x)=Ax }

39. Methods for proving sizes of sets
Let H be family of hash functions where for h 2 H h:{0,1}m  {0,1}ℓ
if h 2 H is 1-1 on A then clearly |A| · 2ℓ
a bit difficult to expect full uniqueness
Relaxed property:
if there are h1, h2, … hℓ 2 H are such that
for all x 2 A there is an 1 · i · l where for all x’ 2 A\{x} we have hi(x)  hi(x’)
then we know |A| < ℓ 2ℓ
Claim: for a pair-wise independent family H, if |A|· 2ℓ-1 , then there exist h1, h2, … hℓ 2 H with the no collisions property. Denote with UH(A,ℓ).
Claim: for A=Accept(M) we have UH(A,ℓ) 2 Σ2P
Can eliminate the quantifier on i by explicitly going over all possibilities!
Conclusion: with oracles calls to UH(A,ℓ) can find smallest ℓ for which UH(A,ℓ) holds
Yields a 2ℓ approximation for f(x)
Can use amplification of f(x) by running M k times (from accepting positions).
New machine M will have fk(x) by accepting paths
No collisions property

40. Proof of No Collisions Claim
For any x 2 A and any x’ 2 A\{x} if h is chosen at random from H
Pr[h(x)=h(x’)]= 1/2ℓ
By the Union bound
Pr[h(x) is unique] ¸ |A| ¢ 1/2ℓ ¸ 1/2
Hitting set: Construct h1, h2, … hℓ one by one, each time covering at least ½ of elements in A that have not been unique so far
Let A’ be the set not covered so far
For any x 2 A’ and any x’ 2 A\{x} for random h 2R H
Pr[h(x) is unique] ¸ |A| ¢ 1/2ℓ ¸ 1/2
Note: this is thefull A

41. Approximate Counting and Uniform Generation
For self reducible problems:
exact counting implies exact uniform generation of witnesses/solutions
Approximate counting: almost uniform generation of solutions
Theorem: if P=NP then given Boolean circuit C possible to sample almost uniformly at random satisfying assignment to C

42. The classes we discussed
EXP
The classes we discussed
PSPACE #P
PSPACE:
Complete problems: TQBF, 2-person games, generalized geography, …
3rd level:
Complete problems: VC dimension for succinct sets
Containment: Approximate #P…
2nd level:
Complete problems: Min DNF, Succinct Set Cover
Containment: Min Circuit, BPP…
1st level:
Complete problems: SAT, UNSAT,…lots…
Containment: factoring, graph isomorphism… PH
Σ3P
Π3P
Δ3P
Σ2P
Π2P
Δ2P NP
coNP
BPP P

43. Some History Polynomial Time Hierarchy: Meyer-Stockmeyer
BPP in the Hierarchy: Lautmann, Sipser
Approximate Counting in the Hierarchy: Stockmeyer
SAT  P/poly implies collapse: Karp-Lipton, Sipser
Stockmeyer’s papers available:
Survey on problems high in the hierarchy: Schaefer and Umans