1. 1 List decoding of binary codes: New concatenation-based results Venkatesan Guruswami U. Washington Joint work with Atri Rudra

2. 2 The basic coding trade-off 001…10 m “message” E(m) “codeword” 01101…0011 E(m)+noise 01001…1111 Noisy channe l Decode message m’ (= m, hopefully ) encoding E: {0,1} k {0,1} n Code C = { E(x) : x  {0,1} k } Rate = k/n, measure of redundancy - To correct more errors, need to add more redundancy during encoding - Our focus: worst-case errors Fundamental trade-off: Rate vs. fraction of errors

3. 3 The central questions What is this trade-off? (between rate and noise level)  aka “channel capacity” Can we find explicit codes achieving (or coming close to) this trade-off? Can they be decoded efficiently?

4. 4 Capacity 2 H(p)n possible received words for which c has to be decoded as a candidate codeword  In other words, nearly-disjoint spheres of volume 2 H(p)n around the 2 Rn codewords So R ≤ 1 - H(p)  Holds also when each bit is flipped independently with probability p and decoding has to succeed with good prob. (converse to Shannon theorem) c pn Region of possible received words when codeword c is transmitted Hamming ball Goal: correct fraction p of worst-case errors

5. 5 List decoding capacity In fact, rate of 1-H(p) can be approached! Thm: [Zyablov-Pinsker’81; Elias’91; G.-Hastad-Sudan-Zuckerman’02] There exist binary codes C of rate R  1 - H(p) - 1/L s.t.  y  {0,1} n at most L codewords of C lie within Hamming distance pn of y. Note: Using such code, can correct arbitrary fraction p of errors with worst-case list ambiguity L  (p,L)-list-decodable  The list ambiguity cannot be avoided For unambiguous decoding, R = 1 - H(2p) is best known (even existentially)

6. 6 The big challenge Achieving list decoding capacity: Explicit construction of binary (p,poly(n))-list- decodable codes of rate approaching 1-H(p) ? Plus, an efficient list decoding algorithm to correct fraction p of errors Comment: Analogous problem for probabilistic noise (BSC p ) is solved [Forney’66], albeit complexity is exponential in 1/  when R = 1- H(p) - 

7. 7 This talk 1.Explicit binary codes with best known trade- off between list decoding radius and rate Decoding up to Blokh-Zyablov radius 2.  binary concatenated codes that achieve list decoding capacity  Most good binary code constructions use concatenation 3.Binary linear code for correcting (1/2-  ) fraction of errors (for  0, subconstant)  block length n  k 3 /  3

8. 8 Codes over large alphabet List decoding capacity for q-ary codes is R = 1 - H q (p) Trade-off is p  1 - R -  for large enough q  Need at least fraction R correct symbols, so fraction 1-R of errors is best one can hope to correct Thm [G.-Rudra’06] :  R, 0 < R < 1, explicit rate R codes to list decode fraction 1 - R -  of errors  q  exp(1/  4 ) (not far from best possible)  List size bound  n 1/  (bad, can be as small as O(1/  ))  Builds on [Parvaresh-Vardy’05]

9. 9 Folded Reed-Solomon codes f(X) f(  ) f (  ) f(  2 ).... f(  n-2 ) f(  n-1 ) Reed-Solomon encoding Fold f(  ) f(  3 ) f(  n-3 ) f(  ) f(  4 ) … f(  n-2 ) Message (deg k polynomial) f(  2 ) f(  5 ) f(  n-1 ) This is 3-way folding m-way folding for large m approaches capacity List decode error fraction m m –s+1 R s/(s+1) for any s, 1  s  m 1 –

10. 10 List recovering The folded RS decoder provides stronger guarantees  soft decoding: handle weights w(i,  ), an estimate of odds/confidence that i’th symbol of codeword is   list recovering: set of possibilities for each position S1S1 S2S2 SNSN Each |S i |  l Can find all codewords c s.t. c i  S i for at least fraction  R s/(s+1) l 1/(s+1) of positions - can mitigate effect of any constant l by choosing s large enough! (Reed-Solomon bottlenecks at l = 1/R [G.-Rudra’05])

11. 11 This talk 1.Explicit binary codes with best known trade-off between list decoding radius and rate Decoding up to Blokh-Zyablov radius 2.  binary concatenated codes that achieve list decoding capacity  Most good binary code constructions use concatenation 3.Binary linear code for correcting (1/2-  ) fraction of errors (for  0, subconstant)  block length n  k 3 /  3

12. 12 Result 1: Achieving BZ radius # Errors Zyablov radius Blokh-Zyablov radius Previous best Optimal Tradeoff Rate Optimal error-frac. H -1 (1-R) Zyablov: (1-R o ) H -1 (1-r) maximized over R o r = R Blokh-Zyablov: ….

13. 13 Code concatenation Simple, yet very useful technique [Forney 66] C 1 :  K !  N (“Outer” code, over large , say GF(2 k ) ) C 2 :  ! {0,1} n (“Inner” binary code)  Concatenated code C 1 ± C 2 (a length Nn binary code)  Inner code has polynomial size; so decodable by brute force m1m1 m2m2 wNwN w1w1 w2w2 mKmK m C 1 (m) C 2 (w 1 )C 2 (w 2 ) C 2 (w N ) C 1 ± C 2 (m)

14. 14 List decoding concatenation C 1 ± C 2 Natural strategy: Divide up received word into N blocks of n bits each y1y1 y2y2 yNyN C 2 (w 1 )C 2 (w 2 ) C 2 (w N ) transmitted received 11 22 NN candidate recd. word for outer code Find closest inner codeword to each block List decoding algorithm for C 1 Loses Information !

15. 15 List Decoding C 1 ± C 2 y1y1 y2y2 yNyN S1S1 S2S2 SNSN List decode inner code C 2 received inner blocks List of l possible symbols for each position For folded RS code of rate R, for any l, only need  R fraction of lists S i to contain correct symbol w i - Previously (eg. RS), rate degraded badly with l List recovery algorithm for C 1

16. 16 Decoding up to Zyablov radius Lemma : C 2 is (  2, l )- list-decodable & C 1 list-recoverable from fraction  1 errors and input sets of size l  C 1 ± C 2 list decodable from fraction  1  2 errors Suitable folded RS of rate R o is list recoverable from fraction 1 - R o -  of errors (regardless of l !)  inner codes of rate r list decodable from fraction H -1 (1- r -  ) errors (with l  1/  2 )  Can find one by “brute-force” search C 1 ± C 2 list decodable from  (1-R o ) H -1 (1-r) errors  Optimize over R o r = R, aka Zyablov radius

17. 17 Multilevel Concatenation [BZ] C 1 : (GF(2 k )) K ! (GF(2 k )) N (“Outer” code 1) C 2 : (GF(2 k )) L ! (GF(2 k )) N (“Outer” code 2) C in : {0,1} 2k ! {0,1} n (“Inner” code) m1m1 m2m2 mKmK m vNvN v1v1 v2v2 C 1 (m) M1M1 M2M2 MLML M wNwN w1w1 w2w2 C 2 (M) C in (v 1,w 1 )C in (v 2,w 2 )C in (v N,w N ) C 1 and C 2 are folded RS

18. 18 Beating rate R o r concat. codes C 1, C 2,C in have rates R 1, R 2 and r  Final rate rR o where R o = (R 1 +R 2 )/2. Choose R 1 < R o Step 1: Just recover m  List decode C in up to H -1 (1-r) errors  List recover C 1 up to 1-R 1 errors m1m1 m2m2 mKmK m vNvN v1v1 v2v2 C 1 (m) M1M1 M2M2 MLML M wNwN w1w1 w2w2 C 2 (M) C in (v 1,w 1 )C in (v 2,w 2 )C in (v N,w N ) Can handle (1-R 1 )H -1 (1-r) > (1-R o )H -1 (1-r) errors

19. 19 Advantage over Concatenation Step 2: Recover M, given m  Subcode of C in of rate r/2 acts on M  List decode subcode up to H -1 (1-r/2) errors  List recover C 2 up to 1-R 2 errors  Can handle (1-R 2 ) H -1 (1-r/2) errors m1m1 m2m2 mKmK m vNvN v1v1 v2v2 C 1 (m) M1M1 M2M2 MLML M wNwN w1w1 w2w2 C 2 (M) C in (v 1,w 1 )C in (v 2,w 2 )C in (v N,w N )

20. 20 Blokh-Zyablov beats Zyablov Fraction of errors that can be handled  min {(1-R 1 )H -1 (1-r), (1-R 2 ) H -1 (1-r/2) } Bigger than (1-R o )H -1 (1-r)  (R 1 +R 2 )/2=R o ( recall that R 1 <R)  H -1 (1-r/2) > H -1 (1-r) so choose R 2 a bit > R o Optimize over choices of r, R 1 and R 2 & beat Zyablov Need nested list decodability of inner code  Achieved by random linear code  Careful derandomization (via conditional expectation) to construct in 2 O(n) time Blokh-Zyablov radius approached by taking multiple (more than 2) outer codes

21. 21 Summing up # Errors Rate Blokh-Zyablov radius List decoding capacity Caveat: Construction time and decoding complexity grow as N poly(1/  )

22. 22 This talk 1.Explicit binary codes with best known trade-off between list decoding radius and rate Decoding up to designed distance (Blokh-Zyablov bound), not half-the-distance 2.  binary concatenated codes that achieve list decoding capacity  Most good binary code constructions use concatenation 3.Binary linear code for correcting (1/2-  ) fraction of errors (for  0)  block length n  k 3 /  3

23. 23 Concatenation is da King All the known explicit binary code constructions for list decoding (with rate > 0) use concatenation  Zyablov & Blokh-Zyablov, for instance Also true for champion constructions for rate vs. distance Could the structural restriction of concatenation preclude them from achieving list-decoding capacity?

24. 24 Fortunately … We prove that concatenated codes can achieve list-decoding capacity  p = H -1 (1-R) Still an existential result  Random code from certain ensemble of concatenated codes achieves capacity w.h.p  Can focus our search on a class of codes with more structure

25. 25 A slight generalization C 1 : (GF(2 k )) K  (GF(2 k )) N Multiple inner codes  C in, C in, … C in : GF(2) k  (GF(2)) n  Justesen codes are a famous example m1m1 m2m2 wNwN w1w1 w2w2 mKmK m C 1 (m) C in (w 1 )C in (w 2 ) C in (w N ) (1)(2)(N) (1)(2) (N)

26. 26 Our result C 1 : (GF(2 rn )) RN  (GF(2 k )) N Multiple inner codes  C in, C in, … C in : GF(2) rn  (GF(2)) n  Conditions on r and R (r=1 OK) List decodable up to H -1 (1-rR) errors, w.h.p. m1m1 m2m2 wNwN w1w1 w2w2 mKmK m C 1 (m) C in (w 1 )C in (w 2 ) C in (w N ) (1) (2) (N) Folded RS code Random and independent linear codes (2)(N)

27. 27 Proof method Extends a proof of [Thommesen’83]  Random concatenated codes lie on the Gilbert-Varshamov bound (for rate vs. distance) w.h.p  Outer Reed-Solomon codes  Random, independent linear inner codes Ingredients in list decoding extension  Large enough set of outer codewords has L codewords whose symbols are component-wise linearly independent over GF(2) (for many positions)  Key insight: Implied by list recoverability of folded RS code Philosophy: Inner codes encode a structured collection of messages, namely symbols that arise in outer codeword

28. 28 Proof sketch Suppose we target final list size of L’ (  L N (log L)/  ) List recoverability of Folded RS code   every L’ tuple has “good” subset of L codewords {c 1,c 2,…, c L } s.t. each c j has set T j of  (1-R)N symbols that are linearly independent from corresponding symbols of {c 1,c 2,…, c j-1 } BFor fixed “good” subset & fixed ball B of radius p,  probability that symbols in T j get mapped into B is small (since the inner encodings are independent and T j is large)  and this event is independent of similar events for T i,, i < j (due to linear independence guarantee) Union bound over good tuples & balls, plus a careful calculation yields good list-decodability for p  H -1 (1-rR)

29. 29 This talk 1.Explicit binary codes with best known trade-off between list decoding radius and rate Decoding up to designed distance (Blokh-Zyablov bound), not half-the-distance 2.  binary concatenated codes that achieve list decoding capacity  Most good binary code constructions use concatenation 3.Binary linear code for correcting (1/2-  ) fraction of errors (for  0)  block length n  k 3 /  3

30. 30 High noise regime Desire code C : {0,1} k  {0,1} n such that (1/2-  ) fraction of errors can be corrected (think  = o(1) )  Want small n  Efficient construction, list decoding (poly(k/  ) time) Non-constructive optimal bound: n  k/  2 Zyablov radius result achieves n  k/  3, but construction time is exponential in 1/  Such codes have many complexity theory applications  Hardcore predicates, extractors & pseudorandom generators, worst-case to average-case reduction, approximating NP witnesses, hardness of approximation.

31. 31 Prior work Hadamard code: n = 2 k, Goldreich-Levin local list decoder  -biased codes (with list decoding radius (1/2 -  ) ) :  3 constructions with n  k 2 /  4 (but without efficient list- decoding) [Alon-Goldreich-Hastad-Peralta’92] [G.-Sudan’00]: n  k 2 /  4 with efficient list decoding  Reed-Solomon concatenated with Hadamard (one of the AGHP constructions)  crucial use of soft decoding of Reed-Solomon codes Without soft decoding, can get k 2 poly(1/  ) length Our result: n  k 3 /  3+ , construction is folded Reed-Solomon concatenated with dual BCH

32. 32 RS+Hadamard soft decoding f0f0 f1f1 aNaN a1a1 a2a2 f k-1 f(X) =  f j X j RS(f) N = 2 m H(a 1 )H(a 2 ) H(a N ) y1y1 y2y2 yNyN a i = f(  i )  GF(2 m ) len(H(a i ))= N Received word y i decoded to a  GF(2 m ) with weight w i,a =1/2 -  (y i,H(a)) Parseval: For each i,  a 2 GF(2 m ) w 2 i,a  O(1)  RS soft decoder succeeds when RS rate k/N  O(  2 ). Final block length = N 2  k 2 /  4

33. 33 Folded RS soft decoding Order s folded RS code provides similar soft decoding guarantee only assuming (s+1)-moment  a w i,a  O(1)  Further, rate k/N is better:   1+1/s  But alphabet size is Q=(2 m ) s, so Hadamard encoding has length N s & final block length = N s+1 (too big) Can use less redundant inner code  Dual of BCH code with distance 2t+1 has small 2t-moment Hadamard is t=1 case  Maps log Q (= ms = 2mt) bits to 2 2m (= N 2 ) bits  Final block length = N 3  ( k/  1+1/s ) 3 S+1

34. 34 Dual BCH code dBCH : (GF(2 b )) t  {0,1} (  1,  2,…,  t )  [Tr(  1 x+  2 x 3 +  3 x 5 +…+  t x 2t-1 )] x  GF(2 )  Hadamard encoding of a  GF(2 b ) is {Tr(ax) for x  GF(2 b )} Proving the moment bound:  Dual of dBCH has distance at least 2t+1  [Kaufman-Litsyn’05] If dual of C has distance > d, weight distribution of C looks binomial to degree d polynomials: E c [ f(wt[c]) ] = 2 -n  i C n i f(i) if deg(f)  d  Use this with f(i) = (1/2- i/n) 2t to bound, for any y, the sum  c (1/2-dist(y,c)/n) 2t 2b2b b

35. 35 Comments Needed to generalized both  outer code (folded RS in place of RS), and  inner code (dual BCH in place of Hadamard) Soft decoding guarantee of folded RS code meshes perfectly with appropriate moment bound of dual BCH “coset weight distribution” Works also with Parvaresh-Vardy codes, the precursor of folded RS code (unlike results 1 and 2)

36. 36 Summary Can make good progress on binary list decoding using recent list recovery & soft list decoding algorithms for folded RS codes The algorithmic results 1 & 3 (Blokh-Zyablov & list- decodable  -biased codes) seem best achievable via techniques which decode inner blocks independently  Need global way to reason about decoding the inner codes, taking into account outer code structure

37. 37 Some open questions Combinatorics of list decoding  Rate of convergence to capacity (R  1 - H(p) - 1/L ?) “De-concatenation”: List-decodable expander codes ?  [G.-Indyk’03] expander-based construction over large alphabet Achieve binary list decoding capacity for erasures  rate R  1 - p where p = fraction of erasures The biggie: Approach binary list decoding capacity  rate  (  2 ) for decoding radius (1/2-  ) ?