December 2, 2009 IPAM: Invariance in Property Testing 1 Invariance in Property Testing Madhu Sudan Microsoft/MIT TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA

Modern challenge to Algorithm Design Data = Massive; Computers = Tiny Data = Massive; Computers = Tiny How can tiny computers analyze massive data? How can tiny computers analyze massive data? Only option: Design sublinear time algorithms. Only option: Design sublinear time algorithms. Algorithms that take less time to analyze data, than it takes to read/write all the data. Algorithms that take less time to analyze data, than it takes to read/write all the data. Can such algorithms exist? Can such algorithms exist? December 2, 2009 IPAM: Invariance in Property Testing 2

Yes! Polling … Is the majority of the population Red/Blue Is the majority of the population Red/Blue Can find out by random sampling. Can find out by random sampling. Sample size / margin of error Sample size / margin of error Independent of size of population Independent of size of population Other similar examples: (can estimate other moments …) Other similar examples: (can estimate other moments …) December 2, 2009 IPAM: Invariance in Property Testing 3

Recent “novel” example Can test for homomorphisms: Can test for homomorphisms: Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Test: Test: Pick x,y in G uniformly, ind. at random; Pick x,y in G uniformly, ind. at random; Verify f(x) ¢ f(y) = f(x ¢ y) Verify f(x) ¢ f(y) = f(x ¢ y) Completeness: accepts homomorphisms w.p. 1 Completeness: accepts homomorphisms w.p. 1 (Obvious) (Obvious) Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. (Not obvious) (Not obvious) December 2, 2009 IPAM: Invariance in Property Testing 4

[Blum, Luby Rubinfeld ’90] [Rubinfeld, S. ’92, ‘96] [Goldreich Goldwasser Ron ‘96] Informally: Informally: “Efficiently” test if “data” satisfies some “property”, in “essence’ “Efficiently” test if “data” satisfies some “property”, in “essence’ Formally: Formally: Data: f: D  R Data: f: D  R Property: P µ {g: D  R} Property: P µ {g: D  R} Efficient: f given as a Efficient: f given as a Tester should make few queries to f. Tester should make few queries to f. Essentially: Essentially: Accept f 2 P w.p. 1; Accept f 2 P w.p. 1; Reject f “far” from P w.h.p. Reject f “far” from P w.h.p. Property Testing December 2, 2009 IPAM: Invariance in Property Testing 5 x f(x) f

Distance: Far/Close Distance = (normalized) Hamming distance Distance = (normalized) Hamming distance δ(f,g) = Prob x 2 D [ f(x) ≠ g(x) ] δ(f,g) = Prob x 2 D [ f(x) ≠ g(x) ] δ(f,P) = Min g 2 P [δ(f,g)] δ(f,P) = Min g 2 P [δ(f,g)] (q, ², δ)-tester for P: (q, ², δ)-tester for P: Makes q queries to f. Makes q queries to f. Accepts w.p. probability ¼ 1 if f 2 P Accepts w.p. probability ¼ 1 if f 2 P Reject w.p. probability ² if δ(f,P) ¸ δ Reject w.p. probability ² if δ(f,P) ¸ δ Ideally: q = O(1) and ² (δ) > 0, 8 δ > 0. Ideally: q = O(1) and ² (δ) > 0, 8 δ > 0. December 2, 2009 IPAM: Invariance in Property Testing 6

[BLR] Lemma Let Rej(f) = Prob x,y 2 G [f(x) ¢ f(y) ≠ f(x ¢ y)] Let Rej(f) = Prob x,y 2 G [f(x) ¢ f(y) ≠ f(x ¢ y)] Lemma: If Rej(f) < 2/9 Lemma: If Rej(f) < 2/9 then δ(f, Hom) = O(Rej(f)). then δ(f, Hom) = O(Rej(f)). Motivated by Program Checking: Motivated by Program Checking: E.g. to check if (complex) program multiplies matrices correctly: E.g. to check if (complex) program multiplies matrices correctly: Verify it is linear in each argument Verify it is linear in each argument Use this to check correctness. Use this to check correctness. December 2, 2009 IPAM: Invariance in Property Testing 7

Independently [Babai Fortnow Lund ‘90] Multilinearity testing: Is a function f: F m  F essentially a degree 1 polynomial in each of the m variables? Multilinearity testing: Is a function f: F m  F essentially a degree 1 polynomial in each of the m variables? Let Rej(f) = Prob ℓ [f| ℓ is not affine] Let Rej(f) = Prob ℓ [f| ℓ is not affine] where ℓ is a random axis parallel line. where ℓ is a random axis parallel line. [BFL] Lemma: [BFL] Lemma: If Rej(f) < 1/poly(m), then If Rej(f) < 1/poly(m), then δ(f, MultiLin) = O(Rej(f)). δ(f, MultiLin) = O(Rej(f)). Implications to Complexity (precursor to “Probabilistically Checkable Proofs”) Implications to Complexity (precursor to “Probabilistically Checkable Proofs”) December 2, 2009 IPAM: Invariance in Property Testing 8

Low-degree testing [Rubinfeld, S. ‘92-’96] Is a function f: F m  F essentially a polynomial of degree d? Is a function f: F m  F essentially a polynomial of degree d? Let Rej(f) = Prob ℓ [f| ℓ is not of degree d] Let Rej(f) = Prob ℓ [f| ℓ is not of degree d] where ℓ is a random line (not axis parallel). where ℓ is a random line (not axis parallel). Lemma ([ALMSS]): Lemma ([ALMSS]): 9 ² > 0 s.t. 8 d,m, sufficiently large F 9 ² > 0 s.t. 8 d,m, sufficiently large F if Rej(f) < ² if Rej(f) < ² then δ (f,Degree-d) = O(Rej(f)) then δ (f,Degree-d) = O(Rej(f)) December 2, 2009 IPAM: Invariance in Property Testing 9

Low-degree testing & Derivatives Let f a (x) = f(x+a) – f(a). Let f a (x) = f(x+a) – f(a). Let f a,b = (f a ) b Let f a,b = (f a ) b Let Rej’(f) = E a,x [ I(f a,a,a,… (x)) ] Let Rej’(f) = E a,x [ I(f a,a,a,… (x)) ] where I(a) = 1 if a = 0 and 0 otherwise. where I(a) = 1 if a = 0 and 0 otherwise. Variant of low-degree test implies that if the (d+1) st derivative in random direction usually vanishes, then f is close to a degree d polynomial Variant of low-degree test implies that if the (d+1) st derivative in random direction usually vanishes, then f is close to a degree d polynomial December 2, 2009 IPAM: Invariance in Property Testing 10

Low-degree testing (Strong form) Is a function f: F m  F essentially a polynomial of degree d? Is a function f: F m  F essentially a polynomial of degree d? Let ρ (f) = Exp ℓ [ δ (f| ℓ, Univ-Deg(d))] Let ρ (f) = Exp ℓ [ δ (f| ℓ, Univ-Deg(d))] where ℓ is a random line. where ℓ is a random line. Note: Rej(f)/F · ρ (f) · Rej(f) Note: Rej(f)/F · ρ (f) · Rej(f) Lemma ([ALMSS]): Lemma ([ALMSS]): 9 ² > 0 s.t. 8 d,m, sufficiently large F 9 ² > 0 s.t. 8 d,m, sufficiently large F if ρ (f) < ² if ρ (f) < ² then δ (f,Degree-d) = O( ρ (f)) then δ (f,Degree-d) = O( ρ (f)) December 2, 2009 IPAM: Invariance in Property Testing 11

Low-degree testing (Stronger form) Is a function f: F m  F essentially a polynomial of degree d? Is a function f: F m  F essentially a polynomial of degree d? Let ρ (f) = Exp ℓ [ δ (f| ℓ, Univ-Deg(d))] Let ρ (f) = Exp ℓ [ δ (f| ℓ, Univ-Deg(d))] where ℓ is a random line. where ℓ is a random line. Note: Rej(f)/F · ρ (f) · Rej(f) Note: Rej(f)/F · ρ (f) · Rej(f) Lemma (Arora + S. ‘97, Raz+Safra ’97) Lemma (Arora + S. ‘97, Raz+Safra ’97) 8 d,m, ² > 0, sufficiently large F 8 d,m, ² > 0, sufficiently large F if ρ (f) < 1 - ² if ρ (f) < 1 - ² then δ (f,Degree-d) = 1 – O( ² ) then δ (f,Degree-d) = 1 – O( ² ) December 2, 2009 IPAM: Invariance in Property Testing 12

Motivations: [BLR] Linearity test: Program checking [BLR] Linearity test: Program checking [BFL], [ALMSS]: Probabilistically checkable proofs [BFL], [ALMSS]: Probabilistically checkable proofs There exists a format for writing proofs that can be checked for correctness with constant queries and constant error probability There exists a format for writing proofs that can be checked for correctness with constant queries and constant error probability Uses low-degree testing & linearity testing. Uses low-degree testing & linearity testing. [GGR]: Should be studied for algorithm design. [GGR]: Should be studied for algorithm design. December 2, 2009 IPAM: Invariance in Property Testing 13

1996-today Graph property testing [GGR, …, Alon, Shapira, Newman, Szegedy, Fisher] Graph property testing [GGR, …, Alon, Shapira, Newman, Szegedy, Fisher] Almost total understanding of graphical property testing … Regularity lemma. Almost total understanding of graphical property testing … Regularity lemma. Graph limits approach … (Borgs, Chayes, Lovasz, Sos, Szegedy, Vesztergombi) Graph limits approach … (Borgs, Chayes, Lovasz, Sos, Szegedy, Vesztergombi) Algebraic Property Testing: Algebraic Property Testing: Many stronger results Many stronger results Fewer new properties Fewer new properties [Alon-Kaufman-Krivelevich-Litsyn-Ron, Kaufman- Ron, Jutla-Patthak-Rudra-Zuckerman] [Alon-Kaufman-Krivelevich-Litsyn-Ron, Kaufman- Ron, Jutla-Patthak-Rudra-Zuckerman] Low-degree testing over small fields (F 2 ) Low-degree testing over small fields (F 2 ) December 2, 2009 IPAM: Invariance in Property Testing 14

Low-degree testing over GF(2) [AKKLR] = Alon-Kaufman-Krivelevich-Litsyn-Ron [AKKLR] = Alon-Kaufman-Krivelevich-Litsyn-Ron Let F = F 2 Let F = F 2 Is a function f: F m  F essentially a polynomial of degree d? Is a function f: F m  F essentially a polynomial of degree d? Let Rej(f) = Prob A [f| A is a degree d poly] Let Rej(f) = Prob A [f| A is a degree d poly] A is a random (d+1)-dim. affine subspace. A is a random (d+1)-dim. affine subspace. U d+1 (f) = (½ - Rej(f)) 2 -d U d+1 (f) = (½ - Rej(f)) 2 -d Lemma [AKKLR] Lemma [AKKLR] 9 ² > 0 s.t. If Rej(f) 0 s.t. If Rej(f) < ² ¢ 2 -d then δ (f,Degree-d) = O(Rej(f)) then δ (f,Degree-d) = O(Rej(f)) (Very weak “inverse Gowers” theorem) December 2, 2009 IPAM: Invariance in Property Testing 15

1996-today Graph property testing [GGR, …, Alon, Shapira, Newman, Szegedy, Fisher] Graph property testing [GGR, …, Alon, Shapira, Newman, Szegedy, Fisher] Almost total understanding of graphical property testing … Regularity lemma. Almost total understanding of graphical property testing … Regularity lemma. Graph limits approach … (Borgs, Chayes, Lovasz, Sos, Szegedy, Vesztergombi) Graph limits approach … (Borgs, Chayes, Lovasz, Sos, Szegedy, Vesztergombi) Algebraic Property Testing: Algebraic Property Testing: Many stronger results Many stronger results Fewer new properties Fewer new properties [Alon-Kaufman-Krivelevich-Litsyn-Ron, Kaufman- Ron, Jutla-Patthak-Rudra-Zuckerman] [Alon-Kaufman-Krivelevich-Litsyn-Ron, Kaufman- Ron, Jutla-Patthak-Rudra-Zuckerman] Low-degree testing over small fields (F 2 ) Low-degree testing over small fields (F 2 ) December 2, 2009 IPAM: Invariance in Property Testing 16

My concerns … Why is the understanding of Algebraic Property Testing so far behind? Why is the understanding of Algebraic Property Testing so far behind? Why can’t we get “rich” class of properties that are all testable? Why can’t we get “rich” class of properties that are all testable? Why are proofs so specific to property being tested. Why are proofs so specific to property being tested. What made Graph Property Testing so well- understood? What made Graph Property Testing so well- understood? What is “novel” about Property Testing, when compared to “polling”? What is “novel” about Property Testing, when compared to “polling”? December 2, 2009 IPAM: Invariance in Property Testing 17

Example Conjecture (AKKLR ‘96): Conjecture (AKKLR ‘96): Suppose property P is a vector space over F 2 ; Suppose property P is a vector space over F 2 ; Suppose its invariant group is 2-transitive. Suppose its invariant group is 2-transitive. Suppose P satisfies a k-ary constraint Suppose P satisfies a k-ary constraint 8 f 2 P, f( ® 1 ) +  + f( ® k ) = 0. 8 f 2 P, f( ® 1 ) +  + f( ® k ) = 0. Then f is (q(k), ² (k,δ),δ(k))-locally testable. Then f is (q(k), ² (k,δ),δ(k))-locally testable. Inspired by “low-degree” test over F 2. Implied all previous algebraic tests (at least in weak forms). Inspired by “low-degree” test over F 2. Implied all previous algebraic tests (at least in weak forms). December 2, 2009 IPAM: Invariance in Property Testing 18

Invariances Property P invariant under permutation (function) ¼ : D  D, if Property P invariant under permutation (function) ¼ : D  D, if f 2 P ) f ο ¼ 2 P Property P invariant under group G if for all ¼ 2 G, P is invariant under ¼. Property P invariant under group G if for all ¼ 2 G, P is invariant under ¼. December 2, 2009 IPAM: Invariance in Property Testing 19

Invariances are the key? “Polling” works well when (because) invariant group of property is the full symmetric group. “Polling” works well when (because) invariant group of property is the full symmetric group. Modern property tests work with much smaller group of invariances. Modern property tests work with much smaller group of invariances. Graph property ~ Invariant under vertex renaming. Graph property ~ Invariant under vertex renaming. Algebraic Properties & Invariances? Algebraic Properties & Invariances? December 2, 2009 IPAM: Invariance in Property Testing 20

Abstracting Algebraic Properties [Kaufman & S.] [Kaufman & S.] Range is a field F and P is F-linear. Range is a field F and P is F-linear. Domain is a vector space over F (or some field K extending F). Domain is a vector space over F (or some field K extending F). Property is invariant under affine (sometimes only linear) transformations of domain. Property is invariant under affine (sometimes only linear) transformations of domain. “Property characterized by single constraint, and its orbit under affine (or linear) transformations.” “Property characterized by single constraint, and its orbit under affine (or linear) transformations.” December 2, 2009 IPAM: Invariance in Property Testing 21

Example: Degree d polynomials Constraint: When restricted to a small dimensional affine subspace, function is polynomial of degree d (or less). Constraint: When restricted to a small dimensional affine subspace, function is polynomial of degree d (or less). #dimensions · d/(K - 1) #dimensions · d/(K - 1) Characterization: If a function satisfies above for every small dim. subspace, then it is a degree d polynomial. Characterization: If a function satisfies above for every small dim. subspace, then it is a degree d polynomial. Single orbit: Take constraint on any one subspace of dimension d/(K-1); and rotate over all affine transformations. Single orbit: Take constraint on any one subspace of dimension d/(K-1); and rotate over all affine transformations. December 2, 2009 IPAM: Invariance in Property Testing 22

Some results If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. Unifies previous algebraic tests (in weak form) with single proof. Unifies previous algebraic tests (in weak form) with single proof. December 2, 2009 IPAM: Invariance in Property Testing 23

Analysis of Invariance-based test Property P given by ® 1,…, ® k ; V 2 F k Property P given by ® 1,…, ® k ; V 2 F k P = {f | f(A( ® 1 )) … f(A( ® k )) 2 V, 8 affine A:K n K n } P = {f | f(A( ® 1 )) … f(A( ® k )) 2 V, 8 affine A:K n K n } Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® k )) not in V ] Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® k )) not in V ] Wish to show: If Rej(f) < 1/k 3, Wish to show: If Rej(f) < 1/k 3, then δ(f,P) = O(Rej(f)). then δ(f,P) = O(Rej(f)). December 2, 2009 IPAM: Invariance in Property Testing 24

BLR Analog Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Define g(x) = majority y {Vote x (y)}, Define g(x) = majority y {Vote x (y)}, where Vote x (y) = f(x+y) – f(y). where Vote x (y) = f(x+y) – f(y). Step 0: Show δ(f,g) small Step 0: Show δ(f,g) small Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 2: Use above to show g is well-defined and a homomorphism. Step 2: Use above to show g is well-defined and a homomorphism. December 2, 2009 IPAM: Invariance in Property Testing 25

BLR Analysis of Step 1 Why is f(x+y) – f(y) = f(x+z) – f(z), usually? Why is f(x+y) – f(y) = f(x+z) – f(z), usually? December 2, 2009 IPAM: Invariance in Property Testing 26 - f(x+z) f(y) - f(x+y) f(z) -f(y) f(x+y+z) -f(z) 0 ?

Generalization g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, Pr A [ ¯,f(A( ® 2 ),…,f(A( ® k )) 2 V] Pr A [ ¯,f(A( ® 2 ),…,f(A( ® k )) 2 V] Step 0: δ(f,g) small. Step 0: δ(f,g) small. Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® k )) 2 V Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® k )) 2 V (if such ¯ exists) (if such ¯ exists) Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 2: Use above to show g 2 P. Step 2: Use above to show g 2 P. December 2, 2009 IPAM: Invariance in Property Testing 27

Matrix Magic? December 2, 2009 IPAM: Invariance in Property Testing 28 A( ® 2 ) B( ® k ) B( ® 2 ) A( ® k ) x t Say A( ® 1 ) … A( ® t ) independent; rest dependent t Random No Choice Doesn’t Matter!

Some results If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. Unifies previous algebraic tests with single proof. Unifies previous algebraic tests with single proof. If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) (explains the AKKLR optimism) (explains the AKKLR optimism) December 2, 2009 IPAM: Invariance in Property Testing 29

Some results If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) (explains the AKKLR optimism) (explains the AKKLR optimism) Unfortunately, q depends inherently on K, not just F … giving counterexample to AKKLR conjecture [joint with Grigorescu & Kaufman] Unfortunately, q depends inherently on K, not just F … giving counterexample to AKKLR conjecture [joint with Grigorescu & Kaufman] Linear invariance when P is not F-linear: Linear invariance when P is not F-linear: Abstraction of some aspects of Green’s regularity lemma … [ Bhattacharyya, Chen, S., Xie ] Abstraction of some aspects of Green’s regularity lemma … [ Bhattacharyya, Chen, S., Xie ] Nice results due to [Shapira] Nice results due to [Shapira] December 2, 2009 IPAM: Invariance in Property Testing 30

More results Invariance of some standard codes (BCH etc.): Invariance of some standard codes (BCH etc.): Have k-single orbit property! So duals are testable. [Grigorescu, Kaufman, S.] Have k-single orbit property! So duals are testable. [Grigorescu, Kaufman, S.] Side effect: New (essentially tight) relationships between Rej AKKLR (f) (=½ + Gowers norm 2 d ) and δ(f,Degree-d). [with Bhattacharyya, Kopparty, Schoenebeck, Zuckerman] Side effect: New (essentially tight) relationships between Rej AKKLR (f) (=½ + Gowers norm 2 d ) and δ(f,Degree-d). [with Bhattacharyya, Kopparty, Schoenebeck, Zuckerman] One hope: Could lead to “simple, good locally testable code”? One hope: Could lead to “simple, good locally testable code”? (Sadly, not with affine-inv. [Ben-Sasson, S.]) (Sadly, not with affine-inv. [Ben-Sasson, S.]) Still … other groups could be used? [Kaufman+Wigderson] Still … other groups could be used? [Kaufman+Wigderson] December 2, 2009 IPAM: Invariance in Property Testing 31

Conclusions Invariance seems to be a very nice perspective on “property testing” … Invariance seems to be a very nice perspective on “property testing” … (Needs Harmonic Analysis ) (Needs Harmonic Analysis ) Hope: Can lead to interesting, new results? Hope: Can lead to interesting, new results? December 2, 2009 IPAM: Invariance in Property Testing 32

December 2, 2009 IPAM: Invariance in Property Testing 33 Thanks