1. Pseudorandomness for Approximate Counting and Sampling
Ronen Shaltiel
University of Haifa
Chris Umans
Caltech

2. What is this talk about? Main technical result:
We define and construct “pseudorandom objects” associated with:
Approximate counting of accepting instances of a given circuit.
Random sampling of accepting instances of a given circuit.
But in fact it all relates to derandomization and it’s a long story:
Once upon a time there was an evil magician called Merlin and a handsome prince called Arthur. One day as Arthur was tossing coins he came about a beautiful NP statement…

3. This talk is about derandomization
Derandomization of procedures that use both randomness and nondeterminism.
Arthur-Merlin games (by derandomization we mean AM=NP).
Approximate counting of accepting instances.
random sampling of accepting instances
Goal: Get rid of randomness (we don’t expect to get rid of nondeterminism).
Under what assumptions?
We derandomize some randomized procedures using assumptions that seem weaker than those we are “supposed to use”.

4. Approximate counting and sampling of accepting instances
Two common computational tasks used frequently in complexity:
approximate counting:
given circuit C on n bits
output approximation of |C-1(1)|:
random sampling:
output random x in C-1(1)
Solvable using randomness and nondeterminism [Sto,JVV,BGP].
What do we mean by derandomizing a sampling procedure?
objects of interest (C recognizes)
{0,1}n

5. Derandomization: Hardness versus Randomness
Initiated by [BM,Yao].
Assumption: hard functions exist.
Conclusion: Derandomization.
A lot of works: [BM82,Y82,HILL,NW88,BFNW93, I95,IW97,IW98,KvM99,STV99,ISW99,MV99, ISW00,SU01,U02,TV02,KI03,GST03]

6. Pseudo-Random Generators
Use a short “seed” of very few truly random bits to generate a long string of pseudo-random bits. A
input
output
pseudo-random
bits
PRG
seed
few truly random bits
many “pseudo-random” bits
Pseudo-randomness: no efficient algorithm can distinguish truly random bits from pseudo-random bits.
Nisan-Wigderson setting: The test A can’t run PRG. (i.e., for tests that runs in time n3 the PRG is allowed to run in time n5). A
input
random bits
output

7. Hardness versus Randomness
Assumption: hard functions exist.
Exists pseudo-random generator
Conclusion: Derandomization.

8. Algorithm for function a contradiction
The meta-argument
Assume (for contradiction) that A that is not fooled by PRG
Hard function
PRG
Derandomization
Proof takes a distinguishing A and uses it to construct a circuit/algorithm for the supposedly hard function. A
input
output
pseudo-random
bits
PRG
seed A
The hardness assumption is against procedures at least as complex as A.
Algorithm for function a contradiction
Meta-Argument: We can’t derandomize the probabilistic version of a complexity class C without a lower bound against C.

9. A brief survey: Achieving the meta argument
Meta-Argument: We can’t derandomize the probabilistic version of a complexity class C without a lower bound against C.
Actually, we usually require a lower bound against the nonuniform version of C of size 2Ω(n) [KvM99].
Assumption: There is a function in E=DTIME(2O(n)) that cannot be computed for size 2Ω(n) circuits of a certain type.
Class
Prob. Class
Lower bound for nonuniform size 2Ω(n):
Ref. P
BPP
deterministic circuits.
[IW97] NP AM
PNP-circuits
[KvM99]
Nondeterministic circuits
[MV99,SU01]

10. Different types of nondeterminsim….
SAT
Adaptive SAT circuit P
NP  coNP NP
coNP
PNP||
PNP
Nonadaptive SAT circuit
SAT ….
ordinary circuit
PNP : Poly-time with access to a SAT oracle.
PNP|| : Poly-time with nonadaptive access to a SAT oracle.

11. Our results

12. Beating the Meta-argument
Class
Prob. class
Nonunform class P
BPP
Det. Circuits NP AM
Nondet circuits
PNP||
BPPNP||
Nonadaptive SAT circuits
PNP
BPPNP
Adaptive SAT circuits
Arthur-Merlin
counting
S2P
sampling
Prvs results: Each can be derandomized using respective hardness.
Our results: All can be derandomized using only hardness for non-deterministic circuits. (Same assumption as the one for AM).
This results beat the meta-argument!
It is known that S2P contains PNP.
We’ve “derandomized” S2P using a lower bound for a weaker circuit class than supposed to!

13. A little bit more formally…
Theorem: Assume that there is a problem in E=DTIME(2O(n)) that cannot be computed by size 2Ω(n) (SV-)nondeterministic circuits then:
AM=NP (known result [MV99,SU01], new proof)
Approximate counting and “sampling” can be done in PNP||.
S2P=PNP
BPPpath=PNP||
The learning algorithm of Bshouty et al. can be derandomized.
More…
Remarks:
E can sometimes be replaced by stronger classes: NE  coNE, ENP|| ,ENP.

14. Main technical result
Theorem: (boosting hardness): if E requires size 2Ω(n) nondeterministic circuits then E requires size 2Ω(n) PNP||-circuits.
Contra-positive: (downward collapse): If E has PNP||-circuits of size s(n) then E has nondeterministic circuits of size s(n)O(1).
(E can be replaced by PSPACE, P#P, ENP, E||NP, NEXP  coNEXP)

15. Quick survey on assumptions implying AM = NP
 L worst-case hard for PNP||-circuits
 L average-case hard for PNP||-circuits
KvM
KvM
PRG for PNP||-circuits
 L worst-case hard for non-det. circuits
 L average-case hard for non-det. circuits AK
 PRG for non-det. circuits SU
AM = NP MV
 HSG for co-non-det. circuits
this paper
All assumptions are equivalent.

16. Strong PRGs from weak assumptions
 L worst-case hard for PNP||-circuits
KvM
PRG for PNP||-circuits
 L worst-case hard for non-det. circuits
“Boosting hardness”
 PRG for (co-) non-det. circuits SU
AM = NP MV
 HSG for co-non-det. circuits
this paper
PRG for stronger circuits than “supposed to”.

17. The current picture of nondeterministic hardness
 L worst-case hard for PNP||-circuits
 HSG for co-non-det. circuits
 PRG for (co-) non-det. circuits
PRG for PNP||-circuits
AM = NP
KvM
“Boosting hardness” MV SU
 L worst-case hard for non-det. circuits
this paper
 L worst-case hard for adap. PNP-circuits
PRG for adap PNP-circuits
KvM
open problem

18. Proof of main result

19. We have to use that f is complete for E
Outline of proof
Assumption:  small PNP||-circuit C for a complete f in E: (for simplicity assume that it makes only one SAT query).
Goal: Show that f has small nondeterministic circuit C’:
Note: in general can’t replace small PNP||-circuit with small nondeterministic circuit (implies, e.g., coNP  NP/poly)
ordinary circuit
SAT C
ordinary circuit ….
Naïve attempt for simulating a SAT query in a nondeterministic circuit:
Guess whether the query is answered by “yes” or “no”.
If query is answered by “yes”: guess satisfying assignment and verify.
If query is answered by “no”: ?????????

20. w.l.o.g. a function in E is a low degree multivariate polynomial
Theorem: (low degree extension) [BF] There is a function family fn:Fqn F for q=nO(1) that is complete for E.

21. Simulating C by a randomized nondeterministic circuit C’
low degree f
ordinary circuit x
SAT x1 x2 xq C x3
ordinary circuit x4 x5 ….
Fqn
On input x: Pass a random low degree curve through x.
Field size polynomial => curve has poly many points x1,..,xq.
Suppose we construct a nondeterministic circuit C’ that computes f(x1),..,f(xq) with at most an  fraction of errors.
Then we can compute f(x)!
Because f restricted to curve is a low degree univariate polynomial. Use Reed-Solomon decoding.

22. Using nonuniformity (following [FF91,SU01,BT03,..])
All points y in Fd s.t. the SAT query on y is answered “yes”.
low degree f
ordinary circuit x
SAT x1 x2 xq C x3
ordinary circuit x4 x5 ….
Fqn
On input x: Pass a random low degree curve through x.
Let p = fraction of y’s in Fd s.t. the SAT query on y is answered “yes”.
Hardwire p to circuit C’.
Points on random curve are k-wise independent for k=poly.
∀x with high probability (over curve) the fraction of xi‘s on curve s.t. the SAT query on xi is answered “yes” is p .

23. Simulating C on all xi‘s on curve with only few errors.
By choosing large enough poly degree for curve.
There exists a fixed choice of random bits that is good for all x’s. f
ordinary circuit x
SAT x1 x2 xq C x3
ordinary circuit x4 x5 ….
Fqn
the fraction of xi‘s on curve s.t. the SAT query on xi is answered “yes” is p .
Goal: Simulate C(x1),..,C(xq) with at most -fraction of errors. For every xi we simulate C up to the SAT query.
Guess fraction of p- xi‘s on curve and witnesses showing that all queries of xi‘s are answered “yes”.
Assume queries of other points on curve are answered “no”.
<2 errors.

24. Applications

25. Story so far… Class Prob. class Nonunform class P BPP Det. Circuits NPAM
Nondet circuits
PNP||
BPPNP||
Nonadaptive SAT circuits
PNP
BPPNP
Adaptive SAT circuits
Arthur-Merlin
counting
sampling
S2P
Goal: Derandomize using only hardness for nondeterministic circuits.
We’ve seen: can boost hardness: From nondeterministic circuits to nonadaptive SAT circuits.
This gives: new proof for AM=NP.
“Implies”: derandomizing counting and sampling.
What does it mean to derandomize sampling?

26. A pseudorandom object for sampling accepting instances
Standard sampling:
sample random x in {0,1}n.
Sampling accepting instances:
given circuit C on n bits.
sample random x in C-1(1)
Discrepancy set:
Output x1,..,xpoly(n) in {0,1}n
No circuit of size (say n2) can distinguish a random xi from a random x.
{0,1}n
C-1(1) x
Sampling accepting instances:
given circuit C on n bits.
sample random x in C-1(1)
Conditional discrepancy set:
given circuit C on n bits.
Output x1,..,xpoly(n) in C-1(1)
No circuit of size (say n2) can distinguish a random xi from a random accepting x.
{0,1}n
C-1(1) x

27. More applications Class Prob. class Nonunform class P BPP
Det. Circuits NP AM
Nondet circuits
PNP||
BPPNP||
Nonadaptive SAT circuits
PNP
BPPNP
Adaptive SAT circuits
Arthur-Merlin
counting
sampling
S2P
Goal: Derandomize using only hardness for nondeterministic circuits.
We’ve seen: new proof for AM=NP.
We’ve seen: can boost hardness: From nondeterministic circuits to nonadaptive SAT circuits.
“Implies”: derandomizing counting and sampling.
Under the same hardness assumption S2P=PNP.

28. Derandomizing S2P S2P  ZPPNP [Cai] Cai’s proof gives that:
Every S2P language has an algorithm that runs in PNP and uses conditional discrepancy sets.
Theorem: if ENP requires exponential size nondeterministic circuits, then
S2P = PNP.

29. Conclusions (SV-)nondeterministic hardness assumption sufficient for:
conditional discrepancy set generators are “pseudorandom object” for sampling accepting instances.
(SV-)nondeterministic hardness assumption sufficient for:
AM = NP (and all assumptions are equivalent)
placing approximate counting in PNP||
placing sampling in PNP||
Placing S2P in PNP.
Use given assumptions in stronger ways!

30. Open questions strengthen downward collapse to adaptive case?
current result: “If E  PNP||/poly then E  NP/poly”
open problem: “If E  PNP/poly then E  NP/poly”
uniform version?
open problem: “If E  PNP|| then E  AM”
Our techniques give E  AM/log.
Improvement by [KF05], E  NP/log.
More examples of beating the meta-argument. Can it be done for weaker classes?

31. That’s it…
Thank You!

32. Tool: low degree extension
Every language L  E has a low-degree extension L  E.
extend to f:Fqd  Fq
f has low total degree (≤ hd)
f can be computed in E and is a robust version of f.
f:{0,1}n  {0,1}
H  Fq (e.g. H={0,1}).
think of f as f:Hd  Fq
Identify f with low-degree polynomial p:Hd Fq Hd
Fqd

33. A pseudorandom generator for sampling
objects of interest (C recognizes)
Approximate counting:
given circuit C
output approximation of |C-1(1)|:
Namely: a number r s.t.
|C-1(1)|(1-) ≤ r ≤ |C-1(1)|(1+)
Theorem: in PNP|| if E requires exponential size (SV-)nondeterministic circuits.
{0,1}n

34. Derandomizing Approximate counting
objects of interest (C recognizes)
Approximate counting:
given circuit C
output approximation of |C-1(1)|:
Namely: a number r s.t.
|C-1(1)|(1-) ≤ r ≤ |C-1(1)|(1+)
Theorem: in PNP|| if E requires exponential size (SV-)nondeterministic circuits.
{0,1}n

35. Approximate counting and sampling
given circuit C
output approximation of |C-1(1)|:
|C-1(1)|(1-) ≤ r ≤ |C-1(1)|(1+)
Note: PRGs for det circuits give:
|C-1(1)| -  ≤ r ≤ |C-1(1)| + 
Theorem: in PNP|| if E requires exponential size (SV-)nondeterministic circuits.
objects of interest (C recognizes)
{0,1}n

36. Proof sketch
Start from weak assumption (hardness for (SV-)nondeterministic circuits).
Use boosting theorem to obtain PRG against PNP|| circuits.
Algorithm for counting works in “BPPNP||“.
Replace random bits with pseudorandom bits (careful: counting is not a decision problem).

37. Probabilistic procedure for Approximate counting [S,JVV,BGP]
try random hash fn. h into 1, 2, 3, … bits
NP query: y that has too many preimages?
with high probability when 2k  |C-1(1)| no y has too many preimages. Output 2k.
{0,1}1
{0,1}k …
{0,1}n
{0,1}n

38. Derandomized procedure for Approximate counting
try hash functions h into 1, 2, 3, … bits that are the outputs of a PRG fooling PNP||-circuits
NP query: “y that has too many preimages?”
when 2k  |C-1(1)| no y has too many preimages with high probability over all hash functions.
therefore many hash functions that are outputs of the PRG will pass the NP test. Output 2k.
{0,1}1
{0,1}k …

39. Pseudorandom Sampling
objects of interest (C recognizes)
Discrepancy set generator:
given s, output T  {0,1}n s.t.
for all circuits D of size s:
|Prx[D(x) = 1] - Prt[D(t) = 1]| ≤ 
Conditional discrepancy set generator:
given C, s, output T  {0,1}n s.t.
|Prx[D(x)=1|C(x) = 1] - PrtT[D(t)=1|C(t)=1]| ≤ 
{0,1}n

40. Sampling Conditional discrepancy set generator:
given C, s, output T  {0,1}n s.t.
for all circuits D of size s:
|Prx[D(x)=1|C(x)=1] - PrtT[D(t)=1|C(t)=1]| ≤ 
Theorem: in PNP|| if E requires exponential size SV-nondeterministic circuits.

41. Proof sketch
Start from weak assumption (hardness for SV nondeterministic circuits).
Use boosting theorem to obtain PRG against PNP|| circuits.
Algorithm for sampling works in “BPPNP“.
Observe that adaptive NP queries are used mainly to find NP witnesses. (Given NP statement find witness).
Replace with non-adaptive witness finding [BCGL90] to get “BPPNP||”.
Replace random bits with pseudorandom bits.

42. Sy = {x : C(x) = 1 and h(x) = y} (note: |Sy| ≤ n2)
Random sampling
pick random y, use NP oracle to enumerate:
Sy = {x : C(x) = 1 and h(x) = y} (note: |Sy| ≤ n2)
pick random i in {1,2,…, n2}
output ith item in list, or “fail” if no ith item (requires adaptive queries).
{0,1}1
{0,1}k …
2k  |C-1(1)|
{0,1}n
{0,1}n

43. Pseudorandom Sampling
as before, using nonadaptive NP queries, can obtain hash function h:{0,1}n  {0,1}k such that 2k  |C-1(1)| and no y has > n2 preimages.
idea: use NP oracle to enumerate:
Sy = {x : C(x) = 1 and h(x) = y}
for those y that are the outputs of a PRG fooling PNP||-circuits
Note: Seems that we require fooling PNP circuits!
{0,1}1
{0,1}k …

44. Sy = {x : C(x) = 1 and h(x) = y}
Sampling
idea: use NP oracle to enumerate:
Sy = {x : C(x) = 1 and h(x) = y}
for those y that are the outputs of a PRG fooling NP||-circuits
Two issues:
need to convert small circuit that catches this conditional discrepancy set into small NP||-circuit that catches the PRG.
enumeration step seems to require adaptive use of NP oracle.

45. Non-adaptive witness finding
can deal with both issues using non-adaptive NP witness finding
usual technique: given (x1, x2, …, xn)
2 queries: is (c1, x2, …, xn) satisfiable for c1=0,1
if satisfiable for c1=0, then
2 queries: is (0, c2, …, xn) satisfiable for c2=0,1
else
2 queries: is (1, c2, …, xn) satisfiable for c2=0,1
etc…
at most 2n adaptive queries total

46. Non-adaptive witness finding
usual technique if unique satisfying assignment: given (x1, x2, …, xn)
is (c1, x2, …, xn) satisfiable for c1=0,1
is (x1, c2, …, xn) satisfiable for c2=0,1 …
is (x1, x2, …, cn) satisfiable for cn=0,1
assemble into single satisfying assignment
2n non-adaptive queries total

47. Non-adaptive witness finding
Valiant-Vazirani: randomized procedure
given (x1, x2, …, xn), produce 1, 2, …, n
with high probability this is a “good” set: at least one i has a unique satisfying assignment
Key observation (in KvM): there is a small circuit that given 1, 2, …, n uses non-adaptive NP queries to decide if input is a “good” set
the output of a PRG fooling NP||-circuits includes a “good” set
use non-adaptive procedure from previous slide in parallel on all formulas in the output of the PRG

48. Putting it all together
“pseudorandom object for sampling”
Conditional discrepancy set generator:
given C, s, output T  {0,1}n s.t.
for all circuits D of size s:
|Prx[D(x)=1|C(x)=1] - PrtT[D(t)=1|C(t)=1]| ≤ 
Theorem: in PNP|| if E requires exponential size SV-nondeterministic circuits.

49. Applications S2P = those languages L expressible as x  L: x  L:
x  L  y z R(x, y, z) = 1
x  L  z y R(x, y, z) = 0
given x, form matrix:
x  L:
x  L:
cell (y, z) = R(x, y, z) y z

50. Applications Background BPP  S2P known: PNP  S2P S2P  ZPPNP (Cai)
Theorem: if ENP requires exponential size SV-nondeterministic circuits, then
S2P = PNP.
Proof idea: Cai’s argument can be viewed as non-randomized reduction to sampling.
Note: This is the strongest example we have of breaking the barrier.
Moral: Make better use of assumptions.

51. Applications
BPPpath = those languages L with a nondeterministic TM M for which
x  L  at least 2/3 of M’s computation paths accept
x  L  at least 2/3 of M’s computation paths reject
note: paths need not be same length
[HHT97]: P||NP  BPPpath

52. Note: non-adaptive needed to get exact characterization.
Applications
Theorem: if E||NP requires exponential size SV-nondeterministic circuits, then
BPPpath = P||NP.
Proof:
perform approximate counting of computation paths
Note: non-adaptive needed to get exact characterization.

53. Conclusions SV-nondeterministic hardness assumption sufficient for:
conditional discrepancy set generators are “pseudorandom object” for sampling
relative error approximators are “pseudorandom object” for approximate counting
SV-nondeterministic hardness assumption sufficient for:
AM = NP (and all assumptions are equivalent)
placing approximate counting in PNP||
placing sampling in PNP||
Placing S2P in PNP.
Use given assumptions in stronger ways!

54. Open questions strengthen downward collapse to adaptive case?
“If E has small adaptive SAT-oracle circuits then E has small SV-nondeterministic circuits.”
current result: “If E  PNP||/poly then E  NP/poly”
open problem: “If E  PNP/poly then E  NP/poly”
uniform version?
open problem: “If E  PNP|| then E  AM”
Our techniques give E  AM/log.
Improvement by [KF05], E  NP/log.
More examples of breaking the barrier. Can it be broken for weaker classes?

55. That’s it…
Thank You!