1. of 31 March 2, 2011Local List Decoding @ IPAM 1 Local List Decoding Madhu Sudan Microsoft Research TexPoint fonts used in EMF. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A Read the TexPoint manual before you delete this box.: A A A A

2. of 31 Overview Last 20 years: Last 20 years: Lots of work on List Decoding Lots of work on List Decoding Lots of work on Local Decoding Lots of work on Local Decoding Today: Today: A look at the intersection: Local List Decoding A look at the intersection: Local List Decoding Part I: The accidental beginnings Part I: The accidental beginnings Part II: Some applications Part II: Some applications Part III: Some LLD codes Part III: Some LLD codes Part IV: Current works Part IV: Current works March 2, 2011Local List Decoding @ IPAM2

3. of 31 March 2, 2011Local List Decoding @ IPAM 3 Part I: History

4. of 31 List Decodable Code Encoding function: E: Σ k Σ n Encoding function: E: Σ k Σ n Code: C = Image(E) Code: C = Image(E) (ρ,L) -List-Decodable Code: 8 r 2 Σ n, (ρ,L) -List-Decodable Code: 8 r 2 Σ n, #{w 2 C | Δ(r,w) · ρ.n} · L. #{w 2 C | Δ(r,w) · ρ.n} · L. List-decoder: Outputs list, given r. List-decoder: Outputs list, given r. [Elias 57, Wozencraft 58] [Elias 57, Wozencraft 58] March 2, 2011Local List Decoding @ IPAM4

5. of 31 Local (Unique) Decoding ρ-decoder: ρ-decoder: Has access to r s.t. Δ(r, E(m)) · ρ.n Has access to r s.t. Δ(r, E(m)) · ρ.n Outputs m. Outputs m. ρ-local decoder: ρ-local decoder: Has query access to r: [n] Σ. Has query access to r: [n] Σ. Input: i 2 [k] Input: i 2 [k] Outputs: m i Outputs: m i (ρ,t)-LDC: makes · t queries for every r,i. (ρ,t)-LDC: makes · t queries for every r,i. March 2, 2011Local List Decoding @ IPAM5

6. of 31 Local List Decoding ρ-List decoder: ρ-List decoder: Access to r 2 Σ n Access to r 2 Σ n Outputs {m 1,…,m L } = {m | Δ(r,E(m)) · ρ.n} Outputs {m 1,…,m L } = {m | Δ(r,E(m)) · ρ.n} (ρ,t)-list decoder: (ρ,t)-list decoder: Query access to r:[n] Σ Query access to r:[n] Σ Inputs: i 2 [k], j 2 [L] Inputs: i 2 [k], j 2 [L] Outputs: (m j ) i Outputs: (m j ) i Note: numbering m 1,…,m L may be arbitrary; Note: numbering m 1,…,m L may be arbitrary; but consistent as we vary i. but consistent as we vary i. March 2, 2011Local List Decoding @ IPAM6

7. of 31 (Convoluted) History 1950 [Reed+Muller]: 1950 [Reed+Muller]: Code due to Muller; Decoder due to Reed. Code due to Muller; Decoder due to Reed. Majority-logic decoder: Essentially a local decoder for ρ < distance/2, Majority-logic decoder: Essentially a local decoder for ρ < distance/2, Not stated/analyzed in local terms. Not stated/analyzed in local terms. 1957 [Elias] 1957 [Elias] Defined List Decoding. Defined List Decoding. Analyzed in random-error setting only. Analyzed in random-error setting only. [1980s] Many works on random-self-reducibility [1980s] Many works on random-self-reducibility Essentially: Local decoders ( for un/natural codes ). Essentially: Local decoders ( for un/natural codes ). March 2, 2011Local List Decoding @ IPAM7

8. of 31 (Convoluted) History 1986 [Goldreich-Levin]: 1986 [Goldreich-Levin]: Local List-decoder for Hadamard code. Local List-decoder for Hadamard code. No mention of any of the words in paper. No mention of any of the words in paper. List-decoding in acknowledgments. List-decoding in acknowledgments. But idea certainly there – also in [Levin 85] But idea certainly there – also in [Levin 85] (many variations since: KM, GRS). (many variations since: KM, GRS). 90-92 [BeaverFeigenbaum, Lipton, GemmellLiptonRubinfeldSWigderson,GemmellS.]: 90-92 [BeaverFeigenbaum, Lipton, GemmellLiptonRubinfeldSWigderson,GemmellS.]: Local decoder for generalized RM codes. Local decoder for generalized RM codes. 96,98 [Guruswami+S]: 96,98 [Guruswami+S]: List-decoder for Reed-Solomon codes. List-decoder for Reed-Solomon codes. March 2, 2011Local List Decoding @ IPAM8

9. of 31 (Convoluted) History 1999 [S.TrevisanVadhan]: 1999 [S.TrevisanVadhan]: Local List-Decoding defined Local List-Decoding defined LLD for Generalized RM code. LLD for Generalized RM code. 2000 [KatzTrevisan]: 2000 [KatzTrevisan]: Local Decoding defined. Local Decoding defined. Lower bounds for LDCs. Lower bounds for LDCs. March 2, 2011Local List Decoding @ IPAM9

10. of 31 Why Convoluted? What is convoluted? What is convoluted? Big gap (positive/negative) between definitions and algorithms Big gap (positive/negative) between definitions and algorithms Why? Why? Motivations/Applications changing. Motivations/Applications changing. Algorithms not crucial to early applications Algorithms not crucial to early applications Some applications needed specific codes Some applications needed specific codes Different communities involved Different communities involved Information theory/Coding theory Information theory/Coding theory CS: Complexity/Crypto/Learning CS: Complexity/Crypto/Learning March 2, 2011Local List Decoding @ IPAM10

11. of 31 March 2, 2011Local List Decoding @ IPAM 11 Part II: Applications

12. of 31 Hardcore Predicates f:{0,1} n {0,1} n is a owf if f:{0,1} n {0,1} n is a owf if f easy to compute f easy to compute f -1 hard on random inputs: f -1 hard on random inputs: random: given y = f(x) for uniform x, output x in f -1 (y). random: given y = f(x) for uniform x, output x in f -1 (y). hard: every polytime alg. succeeds with negligible probabilty. hard: every polytime alg. succeeds with negligible probabilty. b:{0,1} n {0,1} is hardcore predicate for f, if f remains hard to invert given b(x) and f(x) b:{0,1} n {0,1} is hardcore predicate for f, if f remains hard to invert given b(x) and f(x) March 2, 2011Local List Decoding @ IPAM12

13. of 31 Hardcore Predicates b:{0,1} n £ [M] {0,1} is a (randomized) hardcore predicate for f, if b(x,s) hard to predict w.p. ½ + є, given f(x) and s. b:{0,1} n £ [M] {0,1} is a (randomized) hardcore predicate for f, if b(x,s) hard to predict w.p. ½ + є, given f(x) and s. [BlumMicali,Yao,GoldreichLevin]: [BlumMicali,Yao,GoldreichLevin]: 1-1 owf f + hardcore b ) pseudorandom generator. 1-1 owf f + hardcore b ) pseudorandom generator. [GoldreichLevin,Impagliazzo]: [GoldreichLevin,Impagliazzo]: If E:{0,1} k {0,1} m is a (½ - є,poly(n))-LLDC, then b(x,s) = E(x) s is a hardcore predicate for every owf f. If E:{0,1} k {0,1} m is a (½ - є,poly(n))-LLDC, then b(x,s) = E(x) s is a hardcore predicate for every owf f. March 2, 2011Local List Decoding @ IPAM13

14. of 31 Proof of [GL,I] Suppose A predicts b(x,s) given f(x), s Suppose A predicts b(x,s) given f(x), s Fix f(x); let r(s) = A(f(x),s). Fix f(x); let r(s) = A(f(x),s). Run Decoder(r,i,j) for all i,j to recover {x 1,…,x L }. Run Decoder(r,i,j) for all i,j to recover {x 1,…,x L }. Check if f(x j ) = f(x)! Check if f(x j ) = f(x)! (Easy) Claim: This recovers f -1 (f(x)) w.h.p. (Easy) Claim: This recovers f -1 (f(x)) w.h.p. March 2, 2011Local List Decoding @ IPAM14

15. of 31 Thoughts Did [GL] really need Local List-Decoding? Did [GL] really need Local List-Decoding? No. Simple LDC mapping k to poly(k) bits would do. No. Simple LDC mapping k to poly(k) bits would do. Unfortunately, none was known with poly(k) time list-decoder. Unfortunately, none was known with poly(k) time list-decoder. GL: Designed (½ - epsilon,poly(k))-LLDC for Hadamard code (which maps k bits to 2 k bits). GL: Designed (½ - epsilon,poly(k))-LLDC for Hadamard code (which maps k bits to 2 k bits). March 2, 2011Local List Decoding @ IPAM15

16. of 31 Hardness amplification Classical quest in complexity: Classical quest in complexity: Find hard functions (for some class). E.g., Find hard functions (for some class). E.g., f 2 NP – P f 2 NP – P f 2 PSPACE – P f 2 PSPACE – P Story so far: Cant find such. Story so far: Cant find such. Modern question: Modern question: Find functions that are really hard. Find functions that are really hard. Boolean f 2 NP that is hard to distinguish from random function in P. Boolean f 2 NP that is hard to distinguish from random function in P. March 2, 2011Local List Decoding @ IPAM16

17. of 31 Hardness amplification Thm [Lipton, …, S. Trevisan Vadhan]: Thm [Lipton, …, S. Trevisan Vadhan]: Let f:{0,1} k {0,1} be a hard to compute in time poly(k). Let f:{0,1} k {0,1} be a hard to compute in time poly(k). Let E:{0,1} K {0,1} N be (½-є,poly(k)) locally-list-decodable with K = 2 k, N = 2 n. Let E:{0,1} K {0,1} N be (½-є,poly(k)) locally-list-decodable with K = 2 k, N = 2 n. Then g:{0,1} n {0,1} given by g = E(f) is hard to distinguish from random for poly(k) time algorithms. Then g:{0,1} n {0,1} given by g = E(f) is hard to distinguish from random for poly(k) time algorithms. Proof: Obvious from definitions. Proof: Obvious from definitions. March 2, 2011Local List Decoding @ IPAM17

18. of 31 Agnostic Learning General goal of learning theory: General goal of learning theory: Given a class of functions F; Given a class of functions F; query/sample access to f 2 F; query/sample access to f 2 F; Learn f (or circuit (approx.) computing it). Learn f (or circuit (approx.) computing it). Learning with Noise: Learning with Noise: f not in F, but well-approximated by some function in F f not in F, but well-approximated by some function in F Agnostic Learning: Agnostic Learning: No relationship between f and F; No relationship between f and F; learn some approximation of f in F (if it exists). learn some approximation of f in F (if it exists). Useful in applications, as well as theory. Useful in applications, as well as theory. March 2, 2011Local List Decoding @ IPAM18

19. of 31 Agnostic Learning (contd.) GL result (Kushilevitz-Mansour interpretation): GL result (Kushilevitz-Mansour interpretation): Can agnostically learn linear approximations to Boolean functions, with queries. Can agnostically learn linear approximations to Boolean functions, with queries. Kushilevitz-Mansour: Kushilevitz-Mansour: List-decoding helps even more: Can learn decision trees. List-decoding helps even more: Can learn decision trees. Jackson: Jackson: Also CNF/DNF formulae … Also CNF/DNF formulae … March 2, 2011Local List Decoding @ IPAM19

20. of 31 March 2, 2011Local List Decoding @ IPAM 20 Part III: Some LLD Codes

21. of 31 Hadamard Code Code: Maps {0,1} k {0,1} 2 k. Code: Maps {0,1} k {0,1} 2 k. Codewords: Codewords: functions from {0,1} k {0,1}. functions from {0,1} k {0,1}. Encoding of m = is the function E m (y 1 … y k ) = Σ i=1 k m i y i (mod 2). Encoding of m = is the function E m (y 1 … y k ) = Σ i=1 k m i y i (mod 2). I.e., codewords are homogenous, k-variate, degree 1 polynomials over GF(2). I.e., codewords are homogenous, k-variate, degree 1 polynomials over GF(2). GL/KM version. GL/KM version. (Rackoff version). (Rackoff version). March 2, 2011Local List Decoding @ IPAM21

22. of 31 Decoding Hadamard Code (GL/KM) Preliminaries: Preliminaries: View words as functions mapping to {+1,-1}. View words as functions mapping to {+1,-1}. = Exp_y [f(y).g(y)]. = Exp_y [f(y).g(y)]. = 0 if a b and 1 o.w. = 0 if a b and 1 o.w. Let f_a =. Let f_a =. Then f[x] = a f a E(a)[x] Then f[x] = a f a E(a)[x] For all f, a f a 2 = 1. For all f, a f a 2 = 1. (½ - є)-List decoding: Given f, find all a such that f a > 2є. (½ - є)-List decoding: Given f, find all a such that f a > 2є. March 2, 2011Local List Decoding @ IPAM22

23. of 31 Decoding Hadamard Code [GL/KM] Consider 2 n sized binary tree. Consider 2 n sized binary tree. Node labelled by path to root. Node labelled by path to root. Value of leaf a = f a 2 Value of leaf a = f a 2 Value of node Value of node = sum of children values = sum of children values Main idea: Can approximate value of any node Main idea: Can approximate value of any node b f ab 2 = Exp x,y,z [f(xz).f(yz).E a (x).E a (y)] b f ab 2 = Exp x,y,z [f(xz).f(yz).E a (x).E a (y)] Algorithm: Algorithm: Explore tree root downwards. Explore tree root downwards. Stop if node value less than є 2 Stop if node value less than є 2 Report all leaves found. Report all leaves found. March 2, 2011Local List Decoding @ IPAM23Φ1 0100 0

24. of 31 (Generalized) Reed-Muller Code Message space = m-variate, degree r polynomials over GF(q). Message space = m-variate, degree r polynomials over GF(q). Encoding: Values over all points. Encoding: Values over all points. k = ( ) k = ( ) n = q m n = q m distance = 1 – r/q (if r < q). distance = 1 – r/q (if r < q). ¼ q -r/(q-1) if r > q. ¼ q -r/(q-1) if r > q. Decoding problem: Given query access to function that is close to polynomial, find all nearby polynomials, locally. Decoding problem: Given query access to function that is close to polynomial, find all nearby polynomials, locally. March 2, 2011Local List Decoding @ IPAM24 m+rr

25. of 31 Decoding (contd.) Specifically: Specifically: Given query access to f, and x 2 GF(q) m Given query access to f, and x 2 GF(q) m Output p 1 (x),…, p L (x) consistently, where p_js are polynomials within distance ρ of f. Output p 1 (x),…, p L (x) consistently, where p_js are polynomials within distance ρ of f. How to index the codewords? How to index the codewords? By values at a few (random) points in GF(q) m. By values at a few (random) points in GF(q) m. Claim: Specifying value of p at (roughly) log q L points specifies it uniquely (given f). Claim: Specifying value of p at (roughly) log q L points specifies it uniquely (given f). March 2, 2011Local List Decoding @ IPAM_j25

26. of 31 Decoding (contd.) Refined question: Refined question: Given query access to f, and values p j (y 1 ),…,p j (y t ), and x; Given query access to f, and values p j (y 1 ),…,p j (y t ), and x; Compute p j (x) Compute p j (x) Alg [Rackoff, STV, GKZ] Alg [Rackoff, STV, GKZ] Pick random (low-dim) subspace containing y 1,…,y t and x. Pick random (low-dim) subspace containing y 1,…,y t and x. Brute force decode f restricted to this subspace. Brute force decode f restricted to this subspace. March 2, 2011Local List Decoding @ IPAM_j26

27. of 31 March 2, 2011Local List Decoding @ IPAM 27 Part IV: Current Directions

28. of 31 Many interpretations of GL List-decoder for group homomorphisms [Dinur Grigorescu Kopparty S.] List-decoder for group homomorphisms [Dinur Grigorescu Kopparty S.] Set of homomorphisms from G to H form an error-correcting code. Set of homomorphisms from G to H form an error-correcting code. Decode upto minimum distance? Decode upto minimum distance? List-decoder for sparse high-distance linear codes [Kopparty Saraf] List-decoder for sparse high-distance linear codes [Kopparty Saraf] List-decoder for Reed-Muller codes [Gopalan Klivans Zuckerman] List-decoder for Reed-Muller codes [Gopalan Klivans Zuckerman] March 2, 2011Local List Decoding @ IPAM28

29. of 31 Approximate List-Decoding Given r, approximately compute w in C that is somewhat close to r. Given r, approximately compute w in C that is somewhat close to r. Easier problem, so should be solvable for broader class of codes C (C need not have good distance). Easier problem, so should be solvable for broader class of codes C (C need not have good distance). [ODonnell, Trevisan, IJK]: If encoder for C is monotone and local, then get hardness amplification for NP. [ODonnell, Trevisan, IJK]: If encoder for C is monotone and local, then get hardness amplification for NP. [IJK] Give approximate-LLD for truncated Hadamard code. [IJK] Give approximate-LLD for truncated Hadamard code. March 2, 2011Local List Decoding @ IPAM29

30. of 31 Conclusions Intersection of Locality and List-decoding interesting and challenging. Intersection of Locality and List-decoding interesting and challenging. Ought to be explored more? Ought to be explored more? March 2, 2011Local List Decoding @ IPAM30

31. of 31 March 2, 2011Local List Decoding @ IPAM 31 Thank You!

32. of 31 Decoding Hadamard Code [GL/KM] March 2, 2011Local List Decoding @ IPAM32Φ1 0100 0 Value of node labelled p = sum_a f_a^2 Value of node labelled p = sum_a f_a^2 summing over all a with prefix p Walk down from root. Stop if some node has value < epsilon^2 Walk down from root. Stop if some node has value < epsilon^2 Value(p) = easily estimated.