1. Locality in Coding Theory
Madhu Sudan
Harvard
April 9, 2016
Skoltech: Locality in Coding Theory

2. Error-Correcting Codes
(Linear) Code 𝐶⊆ 𝔽 𝑞 𝑛 .
𝔽 𝑞 : Finite field with 𝑞 elements.
𝑛 ≝ block length
𝑘=dim(𝐶)≝ message length
𝑅(𝐶)≝ 𝑘/𝑛: Rate of 𝐶
𝛿 𝐶 ≝ min 𝑥≠𝑦∈𝐶 {𝛿 𝑢,𝑣 ≝ Pr 𝑖 [ 𝑢 𝑖 ≠ 𝑣 𝑖 ]}.
Basic Algorithmic Tasks
Encoding: map message in 𝔽 𝑞 𝑘 to codeword.
Testing: Decide if 𝑢∈𝐶
Correcting: If 𝑢∉𝐶, find nearest 𝑣∈𝐶 to 𝑢.
April 9, 2016
Skoltech: Locality in Coding Theory

3. Locality in Algorithms
“Sublinear” time algorithms:
Algorithms that run in time o(input), o(output).
Assume random access to input
Provide random access to output
Typically probabilistic; allowed to compute output on approximation to input.
(Property testing ≜ Sublinear time for Boolean functions)
LTCs: Codes that have sublinear time testers.
Decide if 𝑢∈𝐶 probabilistically.
Allowed to accept 𝑢 if 𝛿(𝑢,𝐶) small.
LCCs: Codes that have sublinear time correctors.
If 𝛿(𝑢,𝐶) is small, compute 𝑣 𝑖 , for 𝑣∈𝐶 closest to 𝑢.
April 9, 2016
Skoltech: Locality in Coding Theory

4. LTCs and LCCs: Formally
𝐶 is a (ℓ, 𝜖)-LTC if there exists a tester that
Makes ℓ(𝑛) queries to 𝑢.
Accept 𝑢∈𝐶 w.p. 1
Reject 𝑢 w.p. at least 𝜖⋅𝛿(𝑢,𝐶).
C is a (ℓ,𝜖)-LCC if exists decoder 𝐷 s.t.
Given oracle access 𝑢 close to 𝑣∈𝐶, and 𝑖
Decoder makes ℓ(𝑛) queries to 𝑢.
Decoder 𝐷 𝑢 (𝑖) usually outputs 𝑣 𝑖 .
∀𝑖, Pr 𝐷 [ 𝐷 𝑢 𝑖 ≠ 𝑣 𝑖 ]≤𝛿(𝑢,𝑣)/𝜖
Often: ignore 𝜖 (fix to 1 3 ) and focus on ℓ
LDC if only coordinates of message can be recovered.
April 9, 2016
Skoltech: Locality in Coding Theory

5. Skoltech: Locality in Coding Theory
Outline of this talk
Part 0: Definitions of LTC, LCC
Part 1: Elementary construction
Part 2: Motivation (historical, current)
Part 3: State-of-the-art constructions of LCCs
Part 4: State-of-the-art constructions of LTCs
April 9, 2016
Skoltech: Locality in Coding Theory

6. Part 1: Elementary Construction
April 9, 2016
Skoltech: Locality in Coding Theory

7. Main Example: Reed-Muller Codes
Message = multivariate polynomial;
Encoding = evaluations everywhere.
RM 𝑚,𝑟,𝑞 ≝
{ 𝑓 𝛼 𝛼∈ 𝔽 𝑞 𝑚 | 𝑓∈ 𝔽 q 𝑥 1 ,…, 𝑥 𝑚 , deg 𝑓 ≤𝑟}
Locality?
Restrictions of low-degree
polynomials to lines yield
low-degree (univ.) polys.
Random lines sample 𝔽 𝑞 𝑚
uniformly (pairwise ind’ly)
Say 𝑟<𝑞
April 9, 2016
Skoltech: Locality in Coding Theory

8. LCCs and LTCs from Polynomials
Decoding (𝑟<𝑞):
Problem: Given 𝑓≈𝑝, 𝛼∈ 𝔽 𝑞 𝑚 , compute 𝑝 𝛼 .
Pick random 𝛽 and consider 𝑓 ​ 𝐿
where 𝐿= {𝛼+𝑡 𝛽 | 𝑡∈ 𝔽 𝑞 } is a random line ∋𝛼.
Find univ. poly ℎ≈𝑓 ​ 𝐿 and output ℎ(𝛼)
Testing (𝑟<𝑞/2):
Verify deg 𝑓 ​ 𝐿 ≤𝑟 for random line 𝐿
Parameters:
𝑛= 𝑞 𝑚 ;ℓ=𝑞= 𝑛 1 𝑚 ; 𝑅 𝐶 ≈ 1 𝑚!
Analysis non-trivial
Ideas can be extended to 𝑟>𝑞.
Locality ≈poly( 𝛿 −1 )
April 9, 2016
Skoltech: Locality in Coding Theory

9. Skoltech: Locality in Coding Theory
Part 2: Motivations
April 9, 2016
Skoltech: Locality in Coding Theory

10. Motivation – 1 (“Practical”)
How to encode massive data?
Solution I
Encode all data in one big chunk
Pro: Pr[failure] = exp(-|big chunk|)
Con: Recovery time ~ |big chunk|
Solution II
Break data into small pieces; encode separately.
Pro: Recovery time ~ |small|
Con: Pr[failure] = #pieces X Pr[failure of a piece]
Locality (if possible): Best of both Solutions!!
April 9, 2016
Skoltech: Locality in Coding Theory

11. Aside: LCCs vs. other Localities
Local Reconstruction Codes (LRC):
Recover from few (one? two?) erasures locally.
AND Recover from many errors globally.
Regenerating Codes (RgC):
Restricted access pattern for recovery: Partition coordinates and access few symbols per partition.
Main Differences:
#errors: LCCs high vs LRC/RgC low
Asymptotic (LCC) vs. Concrete parameters (LRC/RgC)
April 9, 2016
Skoltech: Locality in Coding Theory

12. Motivation – 2 (“Theoretical”)
(Many?) mathematical consequences:
Probabilistically checkable proofs:
Use specific LCCs and LTCs
Hardness amplification:
Constructing functions that are very hard on average from functions that are hard on worst-case.
Any (sufficiently good) LCC ⇒ Hardness amplification
Small set expanders (SSE):
Usually have mostly small eigenvalues.
LTCs ⇒ SSEs with many big eigenvalues [Barak et al., Gopalan et al.]
April 9, 2016
Skoltech: Locality in Coding Theory

13. Skoltech: Locality in Coding Theory
Aside: PCPs (1 of 4)
Familiar task: Protect massive data 𝑚∈ 0,1 𝑘
PCP task: Protect 𝑚 + analysis 𝜙 𝑚 ∈{0,1}.
𝜙(𝑚) is just one bit long – would like to read & trust 𝜙(𝑚) with few probes.
Can we do it? Yes! PCPs!
“Functional Error-correction” 𝑚
𝐸(𝑚)
𝐸 𝜙 (𝑚)
𝜙 𝑚
April 9, 2016
Skoltech: Locality in Coding Theory

14. Skoltech: Locality in Coding Theory
PCPs (2 of 4) - Definition W 1 V
PCP Verifer: Given 𝑊∈ 0,1 𝑁
HTHHTH
Tosses random coins
Determines query locations
Reads locations. Accepts/Rejects
𝑊≈ 𝐸 𝜙 𝑚 with 𝜙 𝑚 =1⇒ V accepts w.h.p.
𝑊 far from every 𝐸 𝜙 𝑚 with 𝜙 𝑚 =1⇒ rejects w.h.p.
Distinguishes 𝜙 −1 1 ≠∅ from 𝜙 −1 1 =∅
April 9, 2016
Skoltech: Locality in Coding Theory

15. PCPs (3 of 4): “Polynomial-speak”
𝑚→𝑀(𝑥) low-degree (multiv.) polynomial
𝜙→Φ : local map from poly’s to poly’s
𝜙 𝑚 =1⇔∃ 𝐴,𝐵,𝐶 s.t. Φ 𝑀,𝐴,𝐵,𝐶 ≡0
𝐸 𝜙 𝑚 =( 𝑀 , 𝐴 , 𝐵 , 𝐶 ) (evaluations)
Local testability of RM codes ⇒ can verify 𝐸 𝜙 (𝑚) syntactically correct. ( 〈𝑀〉, 〈𝐴〉, 〈𝐵〉, 𝐶 ≈ polynomials )
Distance of RM codes + Φ 𝑀,𝐴,𝐵,𝐶 𝑎 =0 for random 𝑎 ⇒ Semantically correct (𝜙 𝑚 =1)).
April 9, 2016
Skoltech: Locality in Coding Theory

16. Skoltech: Locality in Coding Theory
PCP (4 of 4)
𝑚≡𝐸:𝑉×𝑉→ 0,1 (edges of graph).
≡ 𝐸 : 𝔽 𝑞 2 → 𝔽 𝑞 with deg 𝐸 ≤𝑛≝|𝑉|
𝜙 𝐸 =1⇔ graph 3-colorable.
∃ 𝜒:𝑉→{−1,0,1} s.t. ∀𝑦,𝑧∈𝑉, 𝐸 𝑦,𝑧 =1⇒𝜒 𝑦 ≠𝜒 𝑧
Φ 𝜒 ,𝐴, 𝐴 1 , 𝐵, 𝐵 1 , 𝐵 2 , 𝑡 =
𝐴(𝑦) − 𝜒 𝑦 +1 ⋅ 𝜒 (𝑦)⋅ 𝜒 (𝑦)+1
+ 𝑡 𝐴 𝑦 − 𝐴 1 𝑦 ⋅ 𝑎∈V (𝑦−𝑎)
+ 𝑡 2 𝐵 𝑦,𝑧 −𝐸 𝑦,𝑧 ⋅ 𝑗∈{−2,−1,1,2} 𝜒 𝑦 − 𝜒 𝑧 −𝑗
+ 𝑡 3 𝐵 𝑦,𝑧 − 𝐵 1 𝑦,𝑧 𝑎∈𝑉 𝑦−𝑎 − 𝐵 2 𝑦,𝑧 𝑏∈𝑉 (𝑧−𝑏)
April 9, 2016
Skoltech: Locality in Coding Theory

17. Part 3: Recent Progress on LCCs + LTCs
April 9, 2016
Skoltech: Locality in Coding Theory

18. Summary of Recent Progress
Till 2010: locality 𝑛 =𝑜 𝑛 ⇒𝑅𝑎𝑡𝑒< 1 2 .
2016:
LCC locality 𝑛 = 2 ~ log 𝑛 & 𝑅𝑎𝑡𝑒→1
⇒ℓ 𝑛 = 2 ~ log 𝑛 meeting Singleton Bound
⇒ℓ 𝑛 = 2 ~ log 𝑛 binary, Zyablov bound.
(locally correcting half-the-distance!)
LTC locality 𝑛 =~poly log 𝑛 & 𝑅𝑎𝑡𝑒→1
⇒ … Singleton Bound, Zyablov bound …
April 9, 2016
Skoltech: Locality in Coding Theory

19. Skoltech: Locality in Coding Theory
Main References 2
Multiplicity codes [KoppartySarafYekhanin’10]
See also
Lifted Codes [GuoKoppartySudan’13]
Expander codes [HemenwayOstrovskyWootters’13]
Tensor codes [Viderman ‘11] (see also [GKS’13] )
Above + Alon-Luby composition: [KoppartyMeirRon-ZewiSaraf’16a]
A “Zig-Zag” construction with above ingredients: [KMR-ZS’16b] 1 3 4
April 9, 2016
Skoltech: Locality in Coding Theory

20. Skoltech: Locality in Coding Theory
Lifted Codes
Codes obtained by inverting decoder:
Recall decoder for RM codes.
What code does it decode?
𝐶 𝑚,𝑟,𝑞 = 𝑓: 𝔽 𝑞 𝑚 → 𝔽 𝑞 deg 𝑓 ​ 𝐿 ≤𝑟 ∀ 𝐿}
What we know: RM 𝑚,𝑟,𝑞 ⊆ 𝐶 𝑚,𝑟,𝑞
Theorem [GKS’13]: 𝛿(𝐶 𝑚,𝑟,𝑞 )≈𝛿 RM 𝑚,𝑟,𝑞
𝑅𝑎𝑡𝑒 𝐶 𝑚,𝑟,𝑞 →1 if 𝑞= 2 𝑡 and 𝑡→∞
Local decodability by construction.
Local testability [KaufmanS’07,GuoHaramatyS’15].
April 9, 2016
Skoltech: Locality in Coding Theory

21. Skoltech: Locality in Coding Theory
Why does Rate 𝐶 𝑚,𝑟,𝑞 →1?
Example: 𝑚=2
Some monomials OK!
E.g. 𝑞= 2 𝑡 ; 𝑟= 𝑞 2 ; 𝑓 𝑥,𝑦 = 𝑥 𝑟 𝑦 𝑟 satisfies deg 𝑓 ​ 𝑙𝑖𝑛𝑒 ≤𝑟 for every line!
𝑥 𝑟 𝑎𝑥+𝑏 𝑟 = 𝑏 𝑟 𝑥 𝑟 + 𝑎 𝑟 𝑥 (mod 𝑥 𝑞 −𝑥)
As 𝑟→𝑞 many more monomials
Proof: Lucas’s lemma + simple combinatorics
April 9, 2016
Skoltech: Locality in Coding Theory

22. Skoltech: Locality in Coding Theory
Multiplicity Codes
Basic example
Message = (coeffs. of) poly 𝑝∈ 𝔽 𝑞 𝑥,𝑦 .
Encoding = Evaluations
Local-decoding via lines. Locality = O 𝑛
More multiplicities ⇒ Rate →1
More derivatives ⇒ Locality → 𝑛 𝜖
of 𝑝, 𝜕𝑝 𝜕𝑥 , 𝜕𝑝 𝜕𝑦 over 𝔽 𝑞 2 .
Length = 𝑛= 𝑞 2 ; Alphabet = 𝔽 𝑞 3 ; Rate → 2 3
April 9, 2016
Skoltech: Locality in Coding Theory

23. Skoltech: Locality in Coding Theory
Multiplicity Codes - 2
Why does Rate → 2 3 ?
Every zero of 𝑝, 𝜕𝑝 𝜕𝑥 , 𝜕𝑝 𝜕𝑦 ≡ two zeroes of 𝑝
Can afford to use 𝑝 of degree →2𝑞.
Dimension ↑ ×4 ; But encoding length ↑×3 (Same reason that multiplicity improves radius of list-decoding in [Guruswami,S.])
April 9, 2016
Skoltech: Locality in Coding Theory

24. State-of-the-art as of 2014
∀𝜖,𝛼>0 ∃𝛿= 𝛿 𝜖,𝛼 >0 s.t. ∃ codes w.
Rate ≥1−𝛼
Distance ≥𝛿
Locality = 𝑛 𝜖
Promised:
Locality 𝑛 𝑜(1)
Singleton bound [What if you need higher distance?]
Zyablov bound [What if you want a binary code?]
April 9, 2016
Skoltech: Locality in Coding Theory

25. Alon-Luby Transformation
Key ingredient in [Meir14], [Kopparty et al.’15]
Expander-based
Interleaving
Short MDS
Rate →1
Code
Message
Encoding
April 9, 2016
Skoltech: Locality in Coding Theory

26. Alon-Luby Transformation
Key ingredient in [Meir14], [Kopparty et al.’15]
Rate →1
Code
Short MDS
Expander-based
Interleaving
Message
Encoding
Rate/Distance of final code ~ Rate of MDS
[Meir] Locality ~ Locality of Rate 1 code
Proof = Picture
April 9, 2016
Skoltech: Locality in Coding Theory

27. Subpolynomial Locality
Apply previous transform, with initial code of Rate 1−𝑜(1) and locality 𝑛 𝑜(1) !
[e.g., multiplicity codes with 𝑚=𝜔(1)]
Singleton bound
Zyablov bound?
Concatenation [Forney’66]
April 9, 2016
Skoltech: Locality in Coding Theory

28. Part 4: Locally Testable Codes (after break)
April 9, 2016
Skoltech: Locality in Coding Theory