Sarah R. Allen Ryan O’Donnell David Witmer Carnegie Mellon University

Average-case complexity of CSPs. E.g., random 3SAT.

Random 3SAT n variables, m = m(n) constraints, each an OR of 3 uniformly random literals m 4.267n satisfiable (w.v.h.p.) unsatisfiable (w.v.h.p.) Algorithmic task: find a satisfying assignment Algorithmic task: find a refutation (proof of unsatisfiability)

What is this “refutation” task? 1. Algorithm should (w.h.p.) output a proof, encoded in some formal language like ZFC, that the CSP instance is unsatisfiable. 2. Algorithm should output “unsatisfiable” or “no comment”. It should never be wrong, and it should output “unsatisfiable” w.h.p. Potential example: Algorithm solves an LP/SDP relaxation, outputs “unsatisfiable” iff its value < m.

Random 3SAT n variables, m = m(n) constraints, each an OR of 3 uniformly random literals m 4.267n find a sat. assignmentfind a refutation 3.52n provably doable [HS03,KKL03]

Random 3SAT n variables, m = m(n) constraints, each an OR of 3 uniformly random literals m 4.267n find a sat. assignmentfind a refutation 3.52n provably doable [HS03,KKL03] heuristically doable (?) [MPR15] provably doable [FGK01] ???

Perhaps when, say, m = n 1.4, there’s no poly-time refutation algorithm. Cool! An efficient way to randomly generate simple, hard-to-solve algorithmic tasks! Potential application 1: Cryptography. [Gol00, ABW09, Applebaum…] Potential application 2: Hardness of learning. [Daniely–Linial–Shalev-Shwartz13+]

For applications, investigating random CSP(P) for different predicates P matters. E.g., for certain kinds of k-ary predicates P, if refuting random CSP(P) is hard when m = n C … App. 1: Gives evidence for PRGs in NC 0 with stretch n ↦ n C App. 2: Shows hardness of PAC-learning concept classes “related to” P, assuming C → ∞ as k → ∞

Refuting random CSPs is heavily studied for kSAT, less studied for other predicates.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. The “complexity” of P, an integer we’ll define shortly. It’s always at most k, and “often” small, like 2, 3, 4.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. There’s some evidence that this is the “right” answer. We’ll say more at the end of the talk.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. Ignoring polylog(n) factors.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. Recovers the known n 1.5 result for 3SAT ; cmplx(OR 3 ) = 3.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. New result for kSAT, when k ≥ 5 is odd: we refute provided m ≫ n k/2 ; previous best was m ≫ n ⌈ k/2 ⌉.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. Our results also hold for non-Boolean predicates.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. Running time is n O(k). The algorithm can be “k rounds of the SOS SDP hierarchy”.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. We even provide a δ-refutation, i.e., a proof that OPT ≤ (1−δ)m, for some constant δ = δ(P) > 0.

Our main theorem: A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. If you take m ≫ n k/2, we provide “strong refutation”: i.e., proof that OPT ≤ E[P]·m + o(m). E.g., that OPT ≤ ⅞·m + o(m) for 3SAT, OPT ≤ ½·m + o(m) for kXOR.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. 3SAT example: P = OR 3 = Does it support a 2-wise uniform distribution? Yes.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. 3SAT example: P = OR 3 = Does it support a 2-wise uniform distribution? Yes. Uniform distrib. on these 4 strings.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. 3SAT example: P = OR 3 = Does it support a 2-wise uniform distribution? Yes. Does it support a 3-wise uniform distribution? No. ∴ cmplx(OR 3 ) = 3. We can refute once m ≫ n 1.5

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. 2-out-of-4-SAT: P = Does it support a 2-wise uniform distribution?No. (If it did, by symmetrization that distribution could be the uniform distribution over weight-2, length-4 strings, but that’s not 2-wise uniform.)

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. 2-out-of-4-SAT: P = Does it support a 2-wise uniform distribution?No. ∴ cmplx(P) = 2. We can refute random instances once m ≫ n.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. Does it support a 2-wise uniform distribution?No. ∴ cmplx(P) = 2. We can refute random instances once m ≫ n. The exact same thing holds for any j-out-of-k-SAT. (Refutation fact was already known [BB02].)

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. [DLS14]: A very strong conjecture implying that for many k-ary predicates P, random CSP(P) instances with n ω k→∞ (1) constraints are not refutable in polynomial time. Instantiated with 3 families (P k ), derived 3 hardness-of-learning results. Unfortunately, we show all these predicates have cmplx(P k ) ≤ 4.

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. [DLS14]: A very strong conjecture implying that for many k-ary predicates P, random CSP(P) instances with n ω k→∞ (1) constraints are not refutable in polynomial time. Instantiated with 3 families (P k ), derived 3 hardness-of-learning results. Fortunately, [DS14,Dan15] mostly recovered the results based on more plausible assumptions.

A condition similar to m ≫ n cmplx(P) / 2 also arises in work on finding satisfying assignments in random CSPs with a planted solution [FPV14]. Warning / Remark Nevertheless, I assure you: solving planted CSPs & refuting random CSPs are quite distinct algorithmic tasks.

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random CSP(P) instances, provided m ≫ n cmplx(P) / 2. Plan for the rest of the talk Part 1: Strong refutation for kXOR Part 2: Strong refutation for any k-CSP, provided m ≫ n k/2 Part 3: Proof of the main theorem

Part 1: Strong refutation for kXOR

Given a random kXOR CSP with m ≫ n k/2, want to certify “OPT ≤ ½ · m +o(m)” (whp). [ CCF10 ] strongly refute any CSP with m ≫ n ⌈ k/2 ⌉. For kXOR specifically, m ≫ n ⌈ k/2 ⌉ follows pretty easily from approximation algs literature. For 3SAT, strong refutation provided m ≫ n 1.5 achieved by [ CGL07 ]. Messy, but if you stare at it long enough, it’s clear it generalizes to 3XOR. This, as well as the generalization to kXOR, done independently by [ BM15 ].

Given a random kXOR CSP with m ≫ n k/2, want to certify “OPT ≤ ½ · m +o(m)” (whp). The weaker bound m ≫ n ⌈ k/2 ⌉ follows from the k = 2 case, and a very trivial trick. The m ≫ n k/2 bound requires adding in a very clever trick (from [ FGK01,CGL07 ]). Sadly, only time for a sketch of the k = 2 case.

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ ½ · m +o(m)” (whp). n variables, x 1, …, x n ∈ {0,1} m constraints, each like Switch to ±1 notation.

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ ½ · m +o(m)” (whp). n variables, x 1, …, x n ∈ {−1,+1} m constraints, each like Let OPT denote (# sat − # unsat) instead; i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ o(m)” (whp). n variables, x 1, …, x n ∈ {−1,+1} m constraints, each like Let OPT denote (# sat − # unsat) instead; i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n, want to certify “|OPT| ≤ o(m)” (whp). n variables, x 1, …, x n ∈ {−1,+1} m constraints, each like Let OPT denote (# sat − # unsat) instead; i.e., sum of right-hand sides achieved.

Given a random 2XOR CSP with m ≫ n, want to certify “|OPT| ≤ o(m)” (whp). n variables, x 1, …, x n ∈ {−1,+1} m constraints, each like Let and instead include each possible constraint independently with prob. p.

Given a random 2XOR CSP with p ≫ 1/n, want to certify “|OPT| ≤ o(pn 2 )” (whp). n variables, x 1, …, x n ∈ {−1,+1} m constraints, each like ~pn 2 Let A be the random n×n symmetric matrix with entries {−1,0,+1} depending on the x i,x j constr. Given assignment x ∈ {−1,+1} n, it has (# sat − # unsat) = x T Ax ≤ n||A||

Given a random 2XOR CSP with p ≫ 1/n, want to certify “|OPT| ≤ o(pn 2 )” (whp). Let A be the random n×n symmetric matrix with entries {−1,0,+1} depending on the x i,x j constr. Given assignment x ∈ {−1,+1} n, it has |# sat − # unsat| = |x T Ax| ≤ n||A|| Done if ||A|| ≤ o(pn) whp. (Can efficiently certify.) A good old-fashioned random matrix fact. ||A|| ≤ whp, by the Trace Method [ FK81 ].

Part 1: Strong refutation for kXOR Part 2: Strong refutation for any k-CSP, provided m ≫ n k/2 ✔

m k Def: Given an assignment x ∈ {0,1} n,

m k the table distribution D x is the probability distribution on {0,1} k given by choosing a uniformly random row from the above table.

m k Def: Given an assignment x ∈ {0,1} n, the table distribution D x is the probability distribution on {0,1} k given by choosing a uniformly random row from the above table. Note: Has nothing to do with the predicate P.

Def: Given an assignment x ∈ {0,1} n, the table distribution D x is the probability distribution on {0,1} k given by choosing a uniformly random row from the above table. Def: An instance I is quasirandom if for all assignments x ∈ {0,1} n, the table distribution D x is o(1)-close to the uniform distribution on {0,1} k. Fact: If a CSP(P) instance I is quasirandom, then OPT( I ) ≤ E[P]·m + o(m).

Given a random k-ary CSP with m ≫ n k/2, want to certify it’s quasirandom (whp). By “Vazirani XOR Lemma”, a table distribution D x is o(1)-close to uniform if and only if D x has o(1)-correlation with XOR of S coordinates for all ∅ ≠ S ⊆ [k]. ∴ can certify this using the |S|-XOR refutation algorithm from Part 1, for all 2 k −1 subsets S.

Part 1: Strong refutation for kXOR Part 2: Strong refutation for any k-CSP, provided m ≫ n k/2 ✔ ✔ Part 3: Main theorem, refuting random CSP(P) provided m ≫ n cmplx(P) / 2

cmplx(P): least t ≥ 2 such that P does not support a t-wise uniform distribution. Suppose P does not support a t-wise uniform distribution.

⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory). Given CSP(P) instance, instead of certifying quasirandomness by strongly refuting S-XOR for all S ⊆ [k]…

Suppose P does not support a t-wise uniform distribution. ⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory). Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t. This is doable (whp) provided m ≫ n t/2. This certifies that all table distributions D x are o(1)-close to t-wise uniform, and hence (δ−o(1))-far from being supported on P.

Suppose P does not support a t-wise uniform distribution. ⇒ every t-wise uniform distribution is δ-far from being supported on P (for some constant δ = δ(P) > 0, by basic LP theory). Given CSP(P) instance, strongly refute S-XOR for all |S| ≤ t. This is doable (whp) provided m ≫ n t/2. This certifies that all table distributions D x are o(1)-close to uniform, and hence (δ−o(1))-far from being supported on P. I.e., it certifies that OPT ≤ (1−δ+o(1))m.

Part 1: Strong refutation for kXOR Part 2: Strong refutation for any k-CSP, provided m ≫ n k/2 ✔ ✔ Part 3: Main theorem, refuting random CSP(P) provided m ≫ n cmplx(P) / 2 ✔

Open directions Give evidence that n cmplx(P) / 2 is optimal. I.e., if P supports a (t−1)-wise uniform distribution, try to show various efficient refutation systems fail on random instances with m ≪ n t/2 constraints. [ BW99 ]: Resolution fails on kXOR and kSAT. [ Sch08 ]: SOS fails on kXOR and kSAT. [ BGMT12,TW13,OW14,MWW15 ]: Sherali–Adams + fails. [ FPV15 ]: Certain “statistical algorithms” fail. [ BCK15 ]: For t=3 (pairwise unif.), “pruned” random instances with m = O(n), SOS fails.