1. Local Error-Detection and Error-correction
Madhu Sudan
MIT
December 17, 2007
Coding & Sublinear time

2. Algorithmic Problems in Coding Theory
Code:
Encoding:
Error-detection ( Testing):
Error-correction (Decoding): E : k ! n ; I m a g e ( ) = C R , o r l i z d s t c . F i x C o d e a n s c t E : k ! . G v m 2 , p u ( ) - G i v e n x 2 , d c f 9 m k s . t = E ( ) G i v e n x 2 , d c f 9 m k s . t ( E ) ; G i v e n x 2 , c o m p u t k h a z s ( E ) ; r d .
December 17, 2007
Coding & Sublinear time

3. Sublinear time algorithmics
Given can it be “computed” in time?
Answer 1: Clearly NO, since that is the time it takes to even read the input/write the output f : ; 1 g k ! n o ( k ; n ) x - o r a c l e j f f ( x ) i w h e r f ( x ) i f ( x ) x i
Answer 2: YES, if we are willing to
Present input implicitly (by an oracle).
Represent output implicitly
Compute function on approximation to input.
Extends to computing relations as well.
December 17, 2007
Coding & Sublinear time

4. Sub-linear time algorithms
Initiated in late eighties in context of
Program checking
Interactive Proofs/PCPs
Now successful in many more contexts
Property testing/Graph-theoretic algorithms
Sorting/Searching
Statistics/Entropy computations
(High-dim.) Computational geometry
Many initial results are coding-theoretic!
December 17, 2007
Coding & Sublinear time

5. Sub-linear time algorithms & Coding
Encoding: Not reasonable to expect in sub-linear time.
Testing? Decoding? – Can be done in sublinear time.
In fact many initial results do so!
Codes that admit efficient …
… testing: Locally Testable Codes (LTCs)
… decoding: Locally Decodable Codes (LDCs).
December 17, 2007
Coding & Sublinear time

6. Coding & Sublinear time
Rest of this talk
Definitions of LDCs and LTCs
Quick description of known results
Some basic constructions
(Time permitting) Yekhanin’s construction of LDCs.
December 17, 2007
Coding & Sublinear time

7. Coding & Sublinear time
Definitions
December 17, 2007
Coding & Sublinear time

8. Coding & Sublinear time
Locally Decodable Code
Code: C : k ! n i s ( q ; ) - L o c a l y D e d b f 9 r . t g v 2 [ ] w m = , n w D ( i ) r e a d s q n o m p t f w u . l 2 = 3 W h a t i f
> ( C ) = 2 ? M g n e d o r p l s u ` c w .
December 17, 2007
Coding & Sublinear time

9. Coding & Sublinear time
Locally List-Decodable Code
Code: C i s ( ; ` ) - l t d e c o a b f 8 w 2 n , # r . m C i s ( q ; ` ) - l o c a y t d e b f 9 D r . g v n 2 [ k ] j w m 1 : n w D ( i ; j ) r e a d s q n o m p t f w u . l 2 = 3
December 17, 2007
Coding & Sublinear time

10. History of definitions
Constructions predate formal definitions
[Goldreich-Levin ’89].
[Beaver-Feigenbaum ’90, Lipton ’91].
[Blum-Luby-Rubinfeld ’90].
Hints at definition (in particular, interpretation in the context of error-correcting codes): [Babai-Fortnow-Levin-Szegedy ’91].
Formal definitions
[S.-Trevisan-Vadhan ’99] (local list-decoding).
[Katz-Trevisan ’00]
December 17, 2007
Coding & Sublinear time

11. Coding & Sublinear time

Locally Testable Codes
Code: C n i s ( q ; ) - L o c a l y T e t b f 9 r . n w T r e a d s q ( n ) o m p i t : I f w 2 C c . 1 - , h j =
“Weak” definition: hinted at in [BFLS], explicit in
[RS’96, Arora’94, Spielman’94, FS’95].
December 17, 2007
Coding & Sublinear time

12. Coding & Sublinear time
Strong Locally Testable Codes
Code: C n i s ( q ; ) - L o c a l y T e t b f 9 r . n w T r e a d s q ( n ) o m p i t : I f w 2 C c . 1 F v y , j ;
“Strong” Definition: [Goldreich-S. ’02]
December 17, 2007
Coding & Sublinear time

13. Coding & Sublinear time
Motivations
December 17, 2007
Coding & Sublinear time

14. Motivations for Local decodingS u p o s e C N i l c a y - d b f r = 2 n . ( F t h m w , j g ) c 2 C a n b e v i w d s f u t o : ; 1 g ! . L o c a l d e i n g ) m p u t ( x f r v y , s . R - w [ S T V ] A l t e r n a i p o : C m u c ( x ) w h - v g . L d s I H [ B F ] , P f R G K S
December 17, 2007
Coding & Sublinear time

15. Motivation for Local-testing
No generic applications known.
However,
Interesting phenomenon on its own.
Intangible connection to Probabilistically Checkable Proofs (PCPs).
Potentially good approach to understanding limitations of PCPs (though all resulting work has led to improvements).
December 17, 2007
Coding & Sublinear time

16. Contrast between decoding and testing
Decoding: Property of words near codewords.
Testing: Property of words far from code.
Decoding:
Motivations happy with n = quasi-poly(k), and q = poly log n.
Lower bounds show q = O(1) and n = nearly-linear(k) impossible.
Testing: Better tradeoffs possible! Likely more useful in practice.
Even conceivable: n = O(k) with q = O(1)?
December 17, 2007
Coding & Sublinear time

17. Coding & Sublinear time
Some LDCs and LTCs
December 17, 2007
Coding & Sublinear time

18. Coding & Sublinear time
Codes via Multivariate Polynomials
Message:
Encoding:
Parameters: c o e i n t s f d g , m - v a r p l y P F P F m
(Reed Muller code) e v a l u t i o n s f P F m . k ( t = m ) , n j F .
December 17, 2007
Coding & Sublinear time

19. Basic insight to locality
Local Decoding:
Local Testing: m - v a r i t e p o l y n f d g s c
< . ( ) u b P i c k s u b p a e t h r o g n x f , d . Q y m l q = j F ; T ( ) ! V e r i f y s t c d o p a g . S m l x
December 17, 2007
Coding & Sublinear time

20. Summary of Constructions
Polynomial Codes: (Locally decodable and testable)
Polynomial Codes + Composition/Concatenation:
Codes based on “Algebraic Designs” [Yekhanin] L o c a l i t y q w h n = e x p ( k 1 ) L o c a l T e s t b i y w h q = O ( 1 ) n d ~ k = k ( l o g ) c . L o c a l D e d b i t y w h n = x p ( k 1 q ) L o c a l D e d b i t y w h q = 3 n x p ( k )
December 17, 2007
Coding & Sublinear time

21. Coding & Sublinear time
[Yekhanin ’07]’s LDCs
December 17, 2007
Coding & Sublinear time

22. Recall: Combinatorial Designs
Families of Sets:
Restrictions on Intersections:
E.g.,
Basic Question: S 1 ; : k , T . i f m g
i vs. i: j S i \ T e v n . ( L a r g e ) ( S m a l )
i vs. j: j S i \ T o d . ( S m a l ) ( L a r g e ) H o w l a r g e c n k b ? ( A s a f u n c t i o m ? ) T y p i c a l n s w e r k = ( m )
December 17, 2007
Coding & Sublinear time

23. [Yekhanin]’s Algebraic Designs
Families of Vectors:
Restrictions on Inner Products:
Basic Question: u 1 ; : k , v . i 2 F m p p s m a l r i e h u i ; v = h u i ; v = h u i ; v j 6 = h u i ; v j 2 S 6 3 ( p ; S ) - d e s i g n B a s i c p - d e g n H o w l a r g e c n k b ? A t m o s j S ! C a n w e c h i v ? m p 1
December 17, 2007
Coding & Sublinear time

24. [Yekhanin]’s Algebraic Designs
Families of Vectors:
Restrictions on Inner Products:
Basic Question: u 1 ; : k , v . i 2 F m p p s m a l r i e h u i ; v = h u i ; v = h u i ; v j 6 = h u i ; v j 2 S 6 3 ( p ; S ) - d e s i g n B a s i c p - d e g n H o w l a r g e c n k b ? A t m o s j S ! C a n w e c h i v ? m p 1
December 17, 2007
Coding & Sublinear time

25. [Y’07] Algebraic designs and LDCsm a 1 : B s i c p - d g n w t h k v o r F ) q u y ( b D C k = m p 1 ) n e x (
(Matches some of the early constructions) L e m a 2 : 9 q = ( p ; S ) s . t - d i g n w h k v c o r F u y D C b q ( p ; S ) - A l g e b r a i c n s o f F .
December 17, 2007
Coding & Sublinear time

26. [Y’07] Algebraic designs and LDCsm a 2 : 9 q = ( p ; S ) s . t - d i g n w h k v c o r F u y D C b q ( p ; S ) - a l g e b r i c n s o f F . D e n i t o : S s q - a l g b r c y f 9 p m h ( x ) 2 F [ ] = 1 . d j v ( O n e o f t w q u i v a l d s )
December 17, 2007
Coding & Sublinear time

27. [Y’07] Algebraic designs and LDCsm a 2 : 9 q = ( p ; S ) s . t - d i g n w h k v c o r F u y D C b q ( p ; S ) - a l g e b r i c n s o f F . E x a m p l e : = 1 2 7 ; S f 4 8 6 3 g S i s 3 - a l g e b r c y n m 7 l o n g ( p ; S ) - d e s i x t ! ) 3 - q u e r y L D C m a p i n g k b t s o x ( 1 = 7
December 17, 2007
Coding & Sublinear time

28. [Y’07] Algebraic designs and LDCsm a 2 : 9 q = ( p ; S ) s . t - d i g n w h k v c o r F u y D C b q ( p ; S ) - a l g e b r i c n s o f F . L e m a 3 : p = 2 t 1 ) S f ; 4 g i s - l b r c y n . L e m a 4 : S u l t i p c v s b g r o f F ) 9 ( ; - d n h j . T h e o r m : 9 3 - q u y L D C a p i n g k b t s x ( 1 ) .
December 17, 2007
Coding & Sublinear time

29. Coding & Sublinear time
Proofs?
Disclaimer: Proof of Lemma 2, Lemma 3 too long to fit here. (Many context switches, but elementary.)
Will only attempt to show Lemmas 1 and 4.
December 17, 2007
Coding & Sublinear time

30. Coding & Sublinear time
Basic designs and LDCs G i v e n u 1 ; : k G = 2 6 4 . 1 h u i ; x 3 7 5 u i x
December 17, 2007
Coding & Sublinear time

31. Coding & Sublinear time
Basic designs and LDCs 2 6 4 . 1 h u i ; x 3 7 5 m 1 ; : k
message …
codeword … y s . t h u i ; 6 = y + 2 v i y + ( p 1 ) v i y + v i
Report parity of locations
December 17, 2007
Coding & Sublinear time

32. Coding & Sublinear time
Basic designs and LDCs 2 6 4 . 1 h u i ; x 3 7 5 = 2 6 4 . ? 3 7 5 i t h r o w - a l n e s . + + + o t h e r w s - p 1 n . …
codeword
December 17, 2007
Coding & Sublinear time

33. Coding & Sublinear time
Proof of Lemma 4
Construction of Basic p-designs:
Construction of (p,S)-designs for S multiplicative: i $ s e t o f z x a c l y p 1 u i = c h a r t e s v o f . v i = c h a r t e s o f m p l n . h u i ; v = j \ 2 f 1 : p g T a k e u i ; v s b o U s e ~ u i ; v = p j S t h n o r w f .
December 17, 2007
Coding & Sublinear time

34. Coding & Sublinear time
Conclusions
Local algorithms in error-detection/correction lead to interesting new questions.
Non-trivial progress so far.
Limits largely unknown
O(1)-query LDCs must have R(C) = 0 [Katz-Trevisan]
December 17, 2007
Coding & Sublinear time