Locally Decodable Codes from Nice Subsets of Finite Fields and Prime Factors of Mersenne Numbers Kiran Kedlaya Sergey Yekhanin MIT Microsoft Research

An Inequality 

Error Correcting Codes In classical error correcting codes decoder needs to process the whole (corrupted) codeword to recover even a single bit of the original message! 0010…011 01… …001 n bit message N bit codeword Adversarial noise Decoder processes the (corrupted) codeword

Locally Decodable Codes Definition: Definition: A code C encoding n bits to N bits is called k-LDC if given a (linearly) corrupted codeword one can recover any particular bit of the message (w.h.p.) by reading only k randomly chosen bits. 0010…011 01… …001 n bit message N bit codeword Adversarial noise Decoder reads only k bits Codes with sub-linear decoding complexity!

Locally Decodable Codes Example: There is a 2-query LDC of length Exp(n). Example: There is a 2-query LDC of length Exp(n). Major question: Major question: What is the length of optimal k-query LDCs? Applications: Applications: – Cryptography (private information retrieval). – Worst-case to average case reductions. – Fault tolerant computation. – Data transmission / storage.

LDCs: progress in bounds 2-query: Tight bound - Exp(n) [KdW]. 3-query: Ω(n 2 / log log n) [W]. Lower bound: - Ω(n 2 / log log n) [W]. Upper bounds: Exp(n 1/2 ) [BIK]. (P olynomial interpolation.) - Exp(n 1/2 ) [BIK]. (P olynomial interpolation.) - Exp(n 1/t ), where 2 t -1 is prime [Y]. ( Point removal method.)  Exp(n 1/32,582,657 ) - unconditionally.  Exp(n o(1) ) - if there exist infinitely many Mersenne primes. Exp(n o(1) ) unconditionally. Goal: Obtain constant-query LDCs of length Exp(n o(1) ) unconditionally. Mersenne primes Primes

This work We undertake an in-depth study of the point removal method of [Y] to answer two questions: We undertake an in-depth study of the point removal method of [Y] to answer two questions: Are Mersenne primes essential to the method? Are Mersenne primes essential to the method? Has the method been pushed to its limit? Has the method been pushed to its limit?

Heart of the point removal method Definition: A set S  F q is t - combinatorially nice if …. Definition: A set S  F q is t - combinatorially nice if …. Definition: A set S  F q is k - algebraically nice if …. Definition: A set S  F q is k - algebraically nice if …. Theorem: If for some F q there exists S  F q such that: Theorem: If for some F q there exists S  F q such that: t-combinatorially nice and - S is t-combinatorially nice and - S is k-algebraically nice; then there exist k-query LDCs of length Exp(n 1/t ). Lemma: Let p = 2 t -1 be a Mersenne prime; then S = {1,2,4,…,2 t-1 } in F p is t-combinatorially nice and 3-algebraically nice.

Are Mersenne primes essential? Answer: No. Mersenne numbers with large prime factors are good enough! Theorem: Let  > 0. If P(2 t -1) > (2 t -1)  = p; then {1,2,…,2 t-1 }  F p is t-comb. nice and k(  )-algebr. nice; thus {1,2,…,2 t-1 }  F p is t-comb. nice and k(  )-algebr. nice; thus exist k(  ) – query LDCs of length Exp(n 1/t ). exist k(  ) – query LDCs of length Exp(n 1/t ). Notation: P(m) = the largest prime factor of m. Primes Large prime factors of Mersenne numbers Mersenne primes

Has the method been pushed to its limit? Answer: Yes. Unless we progress on some old number theory questions. Primes that are somewhat large factors of Mersenne numbers are necessary! Theorem: If for infinitely many t there is an F q and S  F q that is k- algebraically nice and t-combinatorially nice; then infinitely often: P(2 t -1) > ( t / 2 ) 1+1 / (k-2). The largest function f(t) for that P(2 t -1) > f(t) unconditionally infinitely often is: f(t) = t log 2 t / log log t. [Stewart]

LDCs and factors of Mersenne numbers P(2 t -1) = 2 t -1 (2 t -1)  P(2 t -1) > (2 t -1)  ( t / 2 ) 1+1 / (k-2) P(2 t -1) > ( t / 2 ) 1+1 / (k-2) g 2 P(2 t -1) > t log 2 t / log log t Sufficient Necessary Known Goal: Obtain constant-query codes of subexponential length.

About the proof Mersenne numbers with large prime factors yield nice subsets. Mersenne numbers with large prime factors yield nice subsets. Nice subsets of finite fields yield Mersenne numbers with somewhat large prime factors. Nice subsets of finite fields yield Mersenne numbers with somewhat large prime factors. (We will see a piece of the second proof.)

Nice subsets to large factors of Mersenne numbers Claim: 3-algebraically nice subsets of prime fields yield large prime factors of Mersenne numbers. Theorem: Suppose S  F p is 3-algebraically nice; then 2 t -1; - p | 2 t -1; - p > 0.75 t 2.

Proof: two steps S  F p is 3-algebraically nice; S  F p is 3-algebraically nice; then there exist  1  2  3 in C p such that:  1 +  2 +  3 = 0. There exist  1  2  3 in C p such that:  1 +  2 +  3 = 0; There exist  1  2  3 in C p such that:  1 +  2 +  3 = 0; then 2 t -1 and p > 0.25 t 2. then p | 2 t -1 and p > 0.25 t 2. Notation: C p - the set of p-th roots of unity in F 2. (We will go over the second step.)

Proof of the second step - I Lemma: There exist  1  2  3 in C p such that:  1 +  2 +  3 = 0; then 2 t -1 and p > 0.25 t 2. then p | 2 t -1 and p > 0.25 t 2.Proof: Let t be the smallest such that C p  F 2. Let t be the smallest such that C p  F 2. p | 2 t -1; p | 2 t -1; Elements of C p \ {1} are proper elements of F 2 i.e., Elements of C p \ {1} are proper elements of F 2 i.e., for  in C p \ {1}, and f(x) in F 2 [x], deg f < t: f(  ) = 0. for  in C p \ {1}, and f(x) in F 2 [x], deg f < t: f(  ) = 0. t t F 2 CpCp t

Proof of the second step - II Proof (continued): Let  i denote elements of C p. Let  i denote elements of C p.  1 +  2 +  3 = 0; yields  4 = 1 +  5.  1 +  2 +  3 = 0; yields  4 = 1 +  5. –  4 =   1 ;  5 =   3 Fix  in C p such that (1+  ) is in C p. Fix  in C p such that (1+  ) is in C p. Consider the set Z={  a (1 +  ) b | a,b in [0,…, t/2-1]}. Consider the set Z={  a (1 +  ) b | a,b in [0,…, t/2-1]}.  a (1 +  ) b   c (1 +  ) d else we would have: f(  ) = 0, where deg f < t.  a (1 +  ) b   c (1 +  ) d else we would have: f(  ) = 0, where deg f < t. Thus, |Z| = (t / 2) 2 and hence p > (t / 2) 2.

Conclusions: Summary: Summary: Further progress on upper bounds for LDCs via point removal method is tied to progress on lower bounds for prime factors of Mersenne numbers. Hopes: Hopes: – Progress in number theory problems. – Broader generalizations of the method. (finite rings?)