Error-Correcting Code Message 0 1 0 0 1 0 1 Encoding Algorithm cn Encoding 0 1 0 0 1 0 1 0 1 0 0 1 0 1 Transmission Received 0 1 0 1 1 0 0 0 0 0 0 0 0 1 Decoding Algorithm Message 0 1 0 0 1 0 1

Performance Measures Encoding and decoding times Decoding guarantee How many errors can you recover from?

Encoding Process Encoding time is linear in number of edges.

Decoding Process Correct one layer at a time. WLOG assume previous layer corrected. Bipartite graph message bits check bits = correct bits = erroneous bits

Decoding Hard decision decoding [Gallager 62] No probabilities in the state, votes. All votes are 1 bit, 0/1. Goal: votes converge to correct values. 1 From my other check bits, I vote that I am a 0. From my other message bits, I vote you are a 1.

Mental Picture: A Regular Tree

Example: Regular Graphs Strategy: message bit m will send received bit to c unless all other check bits say otherwise. Assume initial message all zeroes. m 1 c

Round 0, initial received bits m 1 1 Initial vote = Bit received

Round 1, message to check c 1 m 1 1 1

Round 1, check to message c m 1 1 1

Round 2, message to check c 1 m 1 1

Round 2, check to message c m 1 1

Threshold Generalization Strategy: message bit m will send initial vote to c unless at least b other check bits say otherwise. Optimize the b value for each round. Initially, b begins high. Strategy eventually becomes: message bit passes on decision of majority of other check bits (or initial vote if tied).

Irregular Graphs Same algorithm Strategy: message bit m with degree j will send initial vote to c unless the discrepancy (|# 0 votes - #1 votes|) is at least b. Value b is independent of j! Vary b according to round as before.

Degree of Right Nodes The higher the degree on the right, the less information it yields to the nodes on the left.

High Degree Nodes on Left Left nodes of high degree get information from many right nodes; quickly their beliefs converge to their correct value.

Low Degree Nodes on Left = high degree Left nodes of low degree get less information; however, as high degree nodes settle on a value, their info. grows more accurate, and their beliefs converge correctly.

Irregular Decoding High degree nodes get corrected FIRST. Low degree nodes get corrected LATER.

Analysis Assume graph looks like a tree (for constant number of levels). Derive recursive formula for probability pr after r rounds. Show pr goes to 0. c 1 m

Finding Irregular Graphs Our analysis yields equations Given graph description, can find percentage of errors that can be corrected Goal: find graphs with good performance. Methodology: use equations to design a linear program Given right hand side, find the best left hand side Experimented with just one degree on right

Experimental Results Best regular graph gets 5.17% asymptotically Analysis gives Code 10: 5.78%, Code 14: 6.27% Shannon bound is 11.1%

A Better Algorithm Belief propagation. Send probabilities, instead of 0/1 votes. Nodes keep track of their conditional probabilities of being a 0/1. 0.78 0.83 From my other check bits, I believe my probability of being 0 is 0.78 From my other message bits, I believe your probability of being 0 is 0.83

One Level of Belief Propagation message bit check bits must be correct! message bits A node R looks at neighboring check bits, and their neighboring message bits. From this it determines its conditional probability of being correct.

Belief Propagation R If R knows the values of the complete tree down message bit check bits If R knows the values of the complete tree down L levels, it can determine its conditional probability of being correct. message bits check bits message bits check bits

Belief Propagation : Implementation These conditional probabilities are easy to compute. Nodes update themselves in parallel over rounds. Each round equivalent to exploring one level further in graph. Process stops when current solution is consistent, or fails after preset number of rounds.

Results: Left/Right = 2/1

Time to Decode Time is proportional to product of number of edges and number of rounds Irregular graphs have more edges, but often succeed in fewer rounds. Current times (for 2/1 ratio) at peak about: 1 second for 2K left nodes 10-25 seconds for 20K left nodes 80-130 seconds for 100K left nodes.