Hardness of Reconstructing Multivariate Polynomials. Parikshit Gopalan U. Washington Parikshit Gopalan U. Washington Subhash Khot NYU/Gatech Rishi Saket Gatech/NYU Rishi Saket Gatech/NYU

Curve Fitting Problem: Given data points, find a low degree polynomial that fits best. Easy if there is a perfect fit. Well studied problem …

Curve Fitting through the ages

!

Statistics: Least Squares

Polynomial Reconstruction Coding Theory Computational Learning Cryptography PCPs Pseudorandom -ness

The Reconstruction Problem Input: Degree d. PointsValues x1x1x1x1 f(x 1 ) xixixixi f(x i ) xmxmxmxm f(x m ) Output: A degree d polynomial that best fits the data. In this talk: Finite fields, Hamming distance.

The Reconstruction Problem Input: Degree d, set S, values f(x) for x 2 S. Output: A degree d polynomial that best fits the data. Parameters that matter: 1. Degree d, Field F. 2. Set S. 3. How good is the best fit? (error-rate )

Algorithms for Reconstruction Univariate Case [Sudan, Guruswami-Sudan]: Multivariate Case [Goldreich-Levin, Goldreich- Rubinfeld-Sudan, Arora-Sudan, Sudan-Trevisan-Vadhan]: Can tolerate very high error rate. Are these algorithms optimal?

Hardness Results: Univariate Case Degree d polynomials, n points in F. [Guruswami-Vardy]: NP-hard to tell if some degree d poly. has d +2 agreements. [Guruswami-Sudan]: Can tell if some degree d poly. has (nd) 0.5 agreement.

Hardness Results: Multivariate Case Linear polynomials over F 2 [Hastad] : NP-hard to tell if Some linear poly. satisfies 1- fraction of points. Some linear poly. satisfies 1- fraction of points. Every linear poly. satisfies less than fraction of points. Every linear poly. satisfies less than fraction of points. Extends to any F and d =1. Implies something for d < F. d ¸ 2 over F 2 : Nothing known.

Our Results Over F 2 for any d, NP-hard to tell whether Some linear polynomial satisfies 1- fraction of points. Some linear polynomial satisfies 1- fraction of points. Every degree d polynomial satisfies at most d + fraction of points. Every degree d polynomial satisfies at most d + fraction of points. SZ Lemma: For a degree d poly P 0 over F 2, Pr x [ P(x) 0] ¸ 2 -d. d=1 ½ + ½ + d=2 ¾ + ¾ +

Our Results Over F q for any d, NP-hard to tell whether Some linear polynomial satisfies 1- fraction of points. Some linear polynomial satisfies 1- fraction of points. Every degree d polynomial satisfies at most c(d,q) + fraction of points. Every degree d polynomial satisfies at most c(d,q) + fraction of points. c(d,q): Schwartz-Zippel for polynomials of total degree d over F q.

Overview of Reduction Reducing from Label-Cover. Reducing from Label-Cover. Dictatorship Testing. Dictatorship Testing. Consistency Testing. Consistency Testing. Putting it all together. Putting it all together.

Label Cover Graph: G(V,E), |V| =n. Labels: [k] Edges: e ½ [k] £ [k] Goal: Find a labeling satisfying all edges. Thm [PCP + Raz]: It is NP-hard to tell if Some labeling satisfies all edges. Every labeling satisfies · frac. of edges. 1 2 n 3

X 1 1 X 1 2 … X 1 k The Reduction X 2 1 X 2 2 … X 2 k X 3 1 X 3 2 … X 3 k X n 1 X n 2 … X n k Henceforth d =2, field = F 2. Constraints: Points in {0,1} nk + values. Yes Case: Some L satisfies most constraints. No Case: No Q satisfies many constraints.

The Reduction X 1 1 X 1 2 … X 1 k X 2 1 X 2 2 … X 2 k X 3 1 X 3 2 … X 3 k X n 1 X n 2 … X n k If l(v) is a good labelling, then L = v X v l(v) will satisfy most points.

The Reduction X 1 1 X 1 2 … X 1 k X 2 1 X 2 2 … X 2 k X 3 1 X 3 2 … X 3 k X n 1 X n 2 … X n k If l(v) is a good labelling, then L = v X v l(v) will satisfy most points. Any Q that does ¾ + gives a labelling satisfying fraction of edges.

Dictatorship: Q 1 = Q(X 1 1,…,X 1 k,0,..,0). Q 1 looks like a Dictator X 1 j. Will settle for small list. Consistency: Some pair of labels in the list satisfy. 3, 71, 99 17, 45 Overview of Reduction Constant independent of k.

Dictatorship: Q 1 = Q(X 1 1,…,X 1 k,0,..,0). Q 1 looks like a Dictator X 1 j. Will settle for small list. Can enforce this for frac. of vertices. Consistency: Some pair of labels in the list satisfy. Can enforce this for all edges. 3, 71, 99 17, 45 Overview of Reduction

3, 71, 99 17, 45 Overview of Reduction If Q does ¾ + Small list for frac. of vertices. Consistency for all edges. Assign random labels from list. Satisfies constant fraction of edges.

Overview of Reduction Dictatorship Testing. Dictatorship Testing. Consistency Testing. Consistency Testing. Putting it all together. Putting it all together.

Overview of Reduction Dictatorship Testing. Dictatorship Testing. Consistency Testing. Consistency Testing. Putting it all together. Putting it all together.

Dictatorship Testing for low- degree Polynomials. Input: Q(X 1,…,X k ) of degree 2. Goal: Design a test s.t Every dictatorship X i passes w.p close to 1. Every dictatorship X i passes w.p close to 1. If Q does better than ¾, it is close to a dictatorship. If Q does better than ¾, it is close to a dictatorship. Test: Pick a random point x 2 {0,1} k. Check if Q(x) = y. Mini reconstruction problem! Small List

Dictatorship Testing for low- degree Polynomials. Dictatorships Quadratic polys. All polys.

Dictatorship Testing [Hastad, Bourgain, MOO] Dictatorships All polys. Hard to do with just 2 queries.

Dictatorship Testing for low- degree Polynomials. Dictatorships Quadratic polys. Poly. is of low degree. Allowed one query (!)

Dictatorship Test (0,…,0) (1,…,1) Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Uniform dist: Quadratic polys. are 3:1 balanced. -biased: Dictatorships are highly skewed. Is there a converse? Each i =1 independently. w.p

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. X i passes w.p 1-. X i X j passes w.p X 1 (X 1 + … + X ) + X 2 (X + …) passes w.p 1 - 2

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Define G(Q) to be the graph of Q. Q = X 1 X 2 + X 2 X 3, G(Q) = Thm: If Q passes w.p ¾ +, then G(Q) has no large matchings

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Thm: If Q passes w.p ¾ +, then G(Q) has no large matchings. 1. Large matching: Independent monomials. 2. Only small matchings: Small vertex cover. X 1 L 1 + X 2 L 2

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Thm: If Q does better than ¾, then G(Q) has no large matchings. QQ X i = 0 w.p 1- 2 X i 2 R {0,1} c = ? 0 If G(Q) has a large matching, then Q 0 w.h.p. If Q 0, then c =1 w.p ¸ ¼ (SZ lemma). If Q does well, G(Q) has no large matchings.

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Thm: If Q does better than ¾, then G(Q) has no large matchings. If G(Q) has a large matching, then Q 0 w.h.p. Each edge survives w.p 4 2. Events for each matching edge are independent.

Dictatorship Test Dictatorship Test: Pick 2 {0,1} k from the -biased distribution. Check if Q( ) = 0. Define G(Q) to be the graph of Q. Q = X 1 X 2 + X 2 X 3, G(Q) = Thm: If Q passes w.p ¾ +, then G(Q) has no large matchings Small List: Vertex set of a maximal matching.

Dictatorship: Assign a small list to a vertex. Consistency: Some pair of labels in the list satisfy. 3, 71, 99 17, 45 Overview of Reduction

Overview of Reduction Dictatorship Testing. Dictatorship Testing. Consistency Testing. Consistency Testing. Putting it all together. Putting it all together.

Consistency Testing l(x) = l(y)

Consistency Testing X 1 X 2 … X k Y 1 Y 2 … Y k l(x) = l(y) Given Q(X 1,…,X k,Y 1,…,Y k ) s.t Q(X i ) and Q(Y j ) both pass the dict. Test. Want Q(X 1,..,X k,0,…,0) = Q(0,…,0,Y 1,…,Y k ). Test: Q(r,0) = Q(0,r) for r 2 R {0,1} k. Two queries!

Consistency via Folding X 1 X 2 … X k Y 1 Y 2 … Y k l(x) = l(y) Yes case: Q = X i + Y i for some i. All of them vanish over H = (r,r). Constant on each coset of H. Enforce this on Q even in the No case. H

Consistency via Folding Def: Q is folded over subspace H µ {0,1} k if Q is constant on every coset of H. Examples: Linear polys., juntas. H Thm: Q is folded over H iff for some nice basis ( 1,…, t, 1,..., k-t ), Q = R( 1,…, t ) is a t-junta for t = k – dim(H) In the nice basis ( 1,…, t, 1,..., k-t ) i s: coset of H, j s: position in coset.

Template for Folding Want Q folded over a subspace H. Compute nice basis ( i, j ). Ask for R( 1,…, t ). To test if Q(x) = y o Let x = ( ); test R( ) = y. For analysis: Rewrite R( ) as Q(x). Now Q is folded. H {0,1} n /H

Consistency via Folding l(x) = l(y) Fold over H = (r,r) for r 2 {0,1} k. Polys. folded over H can be written as: Q(X 1,…,X k,Y 1,…,Y k ) = R(X 1 +Y 1, …, X k + Y k ) Gives Q(X 1,…,X k ) = Q(Y 1,…,Y k ).

Dictatorship: Assign a small list to a vertex. Consistency: Some pair of labels in the list satisfy. 3, 71, 99 17, 45 Overview of Reduction

Consistency via Folding l(x) = l(y) Fold over H = (r,r) for r 2 {0,1} k. Polys. folded over H can be written as: Q(X 1,…,X k,Y 1,…,Y k ) = R(X 1 +Y 1, …, X k + Y k ) Gives Q(X 1,…,X k ) = Q(Y 1,…,Y k ). List of X i s: Vertex set of maximal matching. Every two maximal matchings intersect.

Consistency Test 1.Graphs of restrictions are the same. 2.Graph has no large matchings. 3. List = Vertex set of maximal matching. G(Q(X 1,…,X k )) G(Q(Y 1,…,Y k ))

Consistency Test 1.Graphs of restrictions are the same. 2.Graph has no large matchings. 3. List = Vertex set of maximal matching. G(Q(X 1,…,X k )) G(Q(Y 1,…,Y k ))

Summary of Reduction Each constraint gives H ½ {0,1} nk. Fold over the span of all H. Run Dict. test on every vertex. No explicit consistency tests. If Q passes w.p ¾ +, fraction of vertices do well on Dict. test. Consistency for all edges by folding.

Overview of Reduction Dictatorship Testing. Dictatorship Testing. Consistency Testing. Consistency Testing. Putting it all together. Putting it all together.

Projections … X 1 1 X 1 2 … X 1 k X 2 1 X 2 2 … X 2 k X 3 1 X 3 2 … X 3 k X n 1 X n 2 … X n k Can handle equality, permutations. Need perfect completeness: no UGC. Have to deal with projections.

Projections … 1, 2, k (l u ) = (l v ) 1, 2, k 1,,t [k] ! [t]

Projections … 1, 2, k (l u ) = (l v ) 1, 2, k 1,,t [k] ! [t]

Projections … Decoding is a vertex cover for G(Q i ). Need to show that every two vertex covers intersect.

Projections … Do every two vertex covers of G intersect? No:

Projections … Do every two vertex covers of G intersect? … but in any three VCs, some pair intersects. No:

Hypergraph Label Cover 1, 2, k 1,,t [k] ! [t] 1, 2, k [k] ! [t] Strongly satisfied: all 3 projections agree. Weakly satisfied: some 2 agree. Thm: NP-hard to tell if all edges are strongly satsified or at most are weakly satified.

Main Theorem Over F 2 for any d, NP-hard to tell whether Some linear polynomial satisfies 1- fraction of points. Some linear polynomial satisfies 1- fraction of points. Every degree d polynomial satisfies at most d + fraction of points. Every degree d polynomial satisfies at most d + fraction of points.

Better Hardness? Problem: Can we improve soundness to ? Bottleneck: Dictatorship test. Present analysis is optimal in general: Q = (X X k )(X k+1 + … +X 2k ) passes w.p ¾. Can assume that Q is balanced.

Thank You!

Curve Fitting in Deep Space

!

G : graph on k vertices. Alice and Bob have G. Want to pick a common vertex. G Consistency Testing

G : graph on k vertices. Pick a maximal matching; output a random vertex. G Consistency Testing

Overall Reduction X 1 1 X 1 2 … X 1 k X 2 1 X 2 2 … X 2 k X 3 1 X 3 2 … X 3 k Fold over consistency constraints. Test dictatorship on every vertex. If fraction do better than ¾, many edges are satisfied.