1. 1 Verification Codes Michael Luby, Digital Fountain, Inc. Michael Mitzenmacher Harvard University and Digital Fountain, Inc.

2. 2 Motivation LDPC (Low-Density Parity-Check) codes –Linear time and near optimal performance. –For erasures: Packet-level code, where basic op is XORing packets. Very fast. –For errors: Bit-level code, where basic op is probabilistic calculation or XORing bits. Slow. Bit level unsuitable for some applications. Packet-level codes useful as “outer codes.” Can we design packet-level LDPC codes for channels with errors?

3. 3 Design Steps LDPC codes for symmetric q-ary channels, large q –Use that errors are random in a large space. –Tree analysis. Adversarial errors to random errors. –Code scrambling, independently discovered by [L94,GLD01]. Implementation details. –Packet numbers.

4. 4 Setup View code as one layer bipartite graph. Variable nodes on left. Check nodes on right. Codewords satisfy: sum of all neighbors of a check node is 0. Note: sum allows general q. For bits/ packets, sum is XOR. D A B C H F E I J G K

5. 5 Verification A message node is verified if it is believed to have the right value, with high probability. For symmetric q-ary channel, a check node verifies its neighbors if they sum to 0. A B C H If A + B + C = 0, then H verifies A, B, C.

6. 6 Verification Argument Claim: If errors are random, probability of a false verification at each step is at most 1/(q-1). Proof: Consider the last erroneous message node. It must take on the precise value to cause a false verification. For large q, verifications are correct with high probability. A B C H

7. 7 Correction Mechanism If B and C are verified, H can correct and verify value for A. Corrections possible when all but one neighbor of a check node has been verified. If verifications correct with high probability, so are corrections. A B C H

8. 8 Message Passing Algorithm 1 Message nodes may be verified or unverified. –Initially all unverified. Message nodes have a value. –Initially received value. In parallel message nodes send value, state. If all neighbors of a check node sum to 0, check node verifies all neighbors. If all but one neighbor of a check node is verifed, the check node can correct and verify the value of the remaining node. In parallel check nodes send appropriate messages. Repeat.

9. 9 Analysis Error probability from incorrect verification bounded by (#verification steps)/(q-1). Small for large q. –Ignored hereon in the analysis. End goal: all message nodes are verified. LDPC tree analysis yields equations.

10. 10 Degree Sequence Functions Left Side – fraction of edges of degree i on the left. Right Side – fraction of edges of degree i on the right.

11. 11 Tree Based Analysis Left Right Left Assume verified = correct. Pr[not verified, correct] Pr[not verified, incorrect] Pr[verified] After j rounds:

12. 12 Tree Based Analysis Defining equations. English: to be correct and unverified after round j+1, must be: –Correct initially. –Each check node under you must have one incorrect node under it after round j. Probability, z = Pr[a check node has no incorrect neighbor after round j].

13. 13 Tree Based Analysis Defining equations. To be incorrect after round j+1, must be: –Incorrect initially. –Each check node under you must have one unverified node under it after round j. Equivalently: Want

14. 14 Bounds Finding good polynomials and  is difficult non-linear optimization problem. From previous work on LDPCs, for code rate R have and  such that Plugging in this and  yield codes that will handle fraction of errors up to: Good for low rate codes.

15. 15 Proof Let x = b j. Then for Now use Suffices that

16. 16 Improved algorithm D A B C H I E F Incorrect Verified

17. 17 Improved Algorithm Each check node also sends a proposed value to each message node. –Equal to 0 - sum of other neighbors. If a message node receives two of the same proposed values, assumes it is the correct value and sets itself to verified. –Important: no cycles of length 4. Again, small probability of incorrect verification. Similar limiting equations derivable.

18. 18 Reed-Solomon + Verification D A B C RS1 RS2 Tradeoff: RS check nodes yield more redundancy, for more powerful check nodes.

19. 19 Reed-Solomon + Verification Check nodes consist of a Reed-Solomon code over the message nodes. Example: check nodes consists of two points. –So check node of degree d defines a degree d - 1 polynomial. Now check nodes can –Correct any single error among neighbors. –Correct two errors if all other neighbors are verified.

20. 20 Bounds Equations in terms of and  derivable. Finding good polynomials and  is a difficult non-linear optimization problem. Consider case where check nodes can correct if at most 1 neighbor in error. LDPC erasure codes correct if 1 at most 1 neighbor erased. Equivalency implies can correct fraction of errors. Better using additional power of check nodes.

21. 21 Comparison Point Standard Reed-Solomon codes can handle error rate of up to (1-R)/2. –Different model: worst case errors, not random. –Also additional techniques: List Decoding. Verification codes can provably handle higher error rates, with high probability. –Low rates using simple verification. –High rates using verification + 2 RS nodes.

22. 22 Code Scrambling Using shared (pseudo)-randomness, errors can appear random instead of worst case. –Assume a suitably oblivious adversary. –Permute which data in which message packets, so adversary does not know where errors are introduced. –Replace jth packet data x with c j x + d j for random values c j and d j ; inversion causes adversarial error to look random. Symmetric q-ary code sufficient against a suitably oblivious channel.

23. 23 Code Scrambling Example Consider modulo 61. If adversary does not know c, d the inverted values looks random. 7 Data c d Sent Error Inverted 11 23 39 38 18 1155 5 0 13 8

24. 24 Implementation Details Packets must have ID bits. If data can be corrupted so might ID bits. These errors can be absorbed in data errors. –Bad ID# will either be an erased packet (equivalent to an error) or will cause another data packet to be in error.

25. 25 Additional Work Verification codes based on LT (Luby Transform) codes. Extended verification codes: unverified nodes send speculative list of values. To appear in the full paper.

26. 26 Related Work Gallager mentions LDPC codes for symmetric q-ary channels back in 1960’s. –But did not see the possibilities of verification for large q. LDPC codes over GF(q), Davey and MacKay. –Meant for small q. Code scrambling. –Applied primarily to Reed-Solomon codes.

27. 27 Conclusions Verification paradigm leads to simple codes for random errors over large alphabets. Potentially useful as an outer code. –Inner code say an LDPC code on a packet of bits. –Faulty decoding leads to arbitrary packet errors. Useful when avoiding bit-level calculations. –Codes in software vs. hardware.