1. Soft Decision Decoding of RS Codes Using Adaptive Parity Check Matrices
Jing Jiang and Krishna R. Narayanan
Wireless Communication Group
Department of Electrical Engineering
Texas A&M University

2. Reed Solomon Codes Consider an (n,k) RS code over GF(2m), n = 2m-1
Linear block code – e.g. (7,5) RS code over GF(8)
 be a primitive element in GF(8)
Cyclic shift of any codeword is also a valid codeword
RS codes are MDS (dmin = n-k+1)
The dual code is also MDS

3. Introduction Advantages Drawback source RS Encoder interleaving
PR Encoder
sink
hard decision +
AWGN
RS Decoder
BCJR Equalizer
de-interleaving
RS Coded Turbo Equalization System - +
a priori
extrinsic
interleaving
Advantages
Guaranteed minimum distance
Efficient bounded distance hard decision decoder (HDD)
Decoder can handle errors and erasures
Drawback
Performance loss due to bounded distance decoding
Soft input soft output (SISO) decoding is not easy!

4. Presentation Outline Existing soft decision decoding techniques
Iterative decoding based on adaptive parity check matrices
Variations of the generic algorithm
Applications over various channels
Conclusion and future work
Existing Reed Solomon (RS) codes soft decision decoding techniques
Iterative decoding of RS codes based on adaptive parity check matrices
Variations of the generic algorithm
Simulation results over different channel models
Conclusion and future works

5. Existing Soft Decoding Techniques

6. Enhanced Algebraic Hard Decision Decoding
Generalized Minimum Distance (GMD) Decoding (Forney 1966):
Basic Idea:
Erase some of the least reliable symbols
Run algebraic hard decision decoding several times
Drawback: GMD has a limited performance gain
Chase decoding (Chase 1972):
Exhaustively flip some of the least reliable symbols
Running algebraic hard decision decoding several times
Drawback: Has an exponentially increasing complexity
Enhanced algebraic hard decision decoding algorithms:
Generalized Minimum Distance (GMD) Decoding (Forney 1966)
Chase decoding (Chase 1972)
Combined Chase & GMD(Tang et al. 2001)
Reliability based decoding:
Ordered Statistic Decoding (OSD) algorithm (Fossorier & Lin 1995)
Variations, e.g., Box and Match (BMA) decoding (Valembois & Fossorier 2003)
Algebraic SIHO decoding:
Algebraic interpolation based decoding (Koetter & Vardy 2003)
Reduced complexity KV algorithm (Gross et al. submitted 2003)
Combined Chase & GMD(Tang et al. 2001).

7. Algebraic Soft Input Hard Output Decoding
Algebraic SIHO decoding:
Algebraic interpolation based decoding (Koetter & Vardy 2003)
Reduced complexity KV algorithm (Gross et al. submitted 2003)
Basic ideas:
Based on Guruswami and Sudan’s algebraic list decoding
Convert the reliability information into a set of interpolation points
Generate a list of candidate codewords
Pick up the most likely codeword from the codeword list
Drawback:
The complexity increases with , maximum number of multiplicity.

8. Reliability based Ordered Statistic Decoding
Reliability based decoding:
Ordered Statistic Decoding (OSD) (Fossorier & Lin 1995)
Box & Match Algorithm(BMA) (Valembois & Fossorier to appear 2004)
Basic ideas:
Order the received bits according to their reliabilities
Make hard decisions on a set of independent reliable bits (MR Basis)
Re encode to obtain a list of candidate codewords
Drawback:
The complexity increases exponentially with the reprocessing order
BMA must trade memory for complexity

9. Trellis based Decoding using the Binary Image Expansion
Maximum-likelihood decoding and variations
Trellis based decoding using binary image expansion (Vardy & Be’ery ‘91)
Reduced complexity version (Ponnampalam & Vucetic 2002)
Basic ideas:
Binary image expansion of RS
Trellis structure construction using the binary image expansion
Drawback:
Exponentially increasing complexity
Work only for very short codes or codes with very small distance
Maximum-likelihood SISO decoding and variations
Trellis based decoding using the binary image expansions of RS codes over GF(2m) (Vardy & Be’ery 1991)
Reduced complexity version (Ponnampalam & Vucetic 2002)
Iterative SISO algorithms
Sub-trellis structure self-concatenation and iterative decoding of RS codes using their binary image (Ungerboeck 2003)
Stochastic shifting based iterative decoding of RS codes using their binary image (Jing & Narayanan, submitted 2003)
Iterative decoding via sparse factor graph representations of RS codes based on a fast Fourier transform (FFT) (Yedidia, MERL report 2003)

10. Binary Image Expansion of RS Codes

11. Consider the (7,5) RS code
Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)

12. Recent Iterative Techniques
Sub-trellis based iterative decoding (Ungerboeck 2003)
Self-concatenation structure based on sub-trellis constructed from the parity check matrix
Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)
Drawbacks:
Performance deteriorates due to large number of short cycles
Work for short codes with small minimum distances
Potential error floor problem in high SNR region

13. Recent Iterative Techniques (cont’d)
Stochastic shifting based iterative decoding (Jing & Narayanan, to appear 2004)
Due to the irregularity in the H matrix, iterative decoding favors some bits
Taking advantage of the cyclic structure of RS codes
Shift by 2
Stochastic shift prevent iterative procedure from getting stuck
Best result: RS(63,55) about 0.5dB gain from HDD
However, for long codes, this algorithm still doesnt provide good improvement

14. Remarks on Existing Techniques
Most SIHO algorithms are either too complex to implement or having only marginal gain
Moreover, SIHO decoders cannot generate soft output directly
Trellis-based decoders have exponentially increasing complexity
Iterative decoding algorithms do not work for long codes, since the parity check matrices of RS codes are not sparse
Most SIHO algorithms are either too complex to implement or have only marginal gain.
Moreover, SIHO decoders cannot generate soft output efficiently.
Trellis-based ML decoder has exponentially increasing complexity.
Iterative decoding algorithms do not work for long codes. Since the parity check matrices of RS codes are not sparse. There’s no known technique to generate a sparse parity check matrix for long RS codes.
“Soft decoding of large RS codes as employed in many standard transmission systems, e.g., RS(255,239), with affordable complexity remains an open problem” (Ungerboeck, ISTC2003)

15. Questions
Q: Why doesn’t iterative decoding work for codes with non-sparse parity check matrices?
Q: Can we get some idea from the failure of iterative decoder?
Most SIHO algorithms are either too complex to implement or have only marginal gain.
Moreover, SIHO decoders cannot generate soft output efficiently.
Trellis-based ML decoder has exponentially increasing complexity.
Iterative decoding algorithms do not work for long codes. Since the parity check matrices of RS codes are not sparse. There’s no known technique to generate a sparse parity check matrix for long RS codes.

16. How does standard message passing algorithm work?
erased bits
bit nodes
………….
……….. ?
…………….
check nodes
If two or more of the incoming messages are erasures the check is erased
Otherwise, check to bit message is the value of the bit that will satisfy the check

17. How does standard message passing algorithm work?
bit nodes
………….
………..
…………….
check nodes
Small values of vj can be thought of as erasures and hence more than two
edges with small vj’s saturate the check

18. A Few Unreliable Bits “Saturate” the Non-sparse Parity Check Matrix
Consider RS(7, 5) over GF(23)
Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)
Unfortunately, RS codes usually possess no sparse parity check matrices as the codeword length increases. There are no known operation to make the whole parity check matrix sparse.
Thus, as the codeword length goes to be large, Consequently, iterative decoding is stuck at some “pseudo-codes” due to only a few unreliable bits.
Iterative decoding is stuck due to only a few unreliable bits “saturating” the whole non-sparse parity check matrix

19. Sparse Parity Check Matrices for RS Codes
Can we find an equivalent binary parity check matrix that is sparse?
For RS codes, this is not possible!
The H matrix is the G matrix of the dual code
The dual of an RS code is also an MDS Code
Every row has weight at least (N-K)!

20. Iterative Decoding Based on Adaptive Parity Check Matrix
Idea: reduce the sub-matrix corresponding to the unreliable positions to a sparse nature.
For example, consider (7,4) Hamming code:
transmitted codeword
received vector
parity check matrix
Need animation of stopping pattern of iterative decoding.
After the adaptive update, iterative decoding can proceed.

21. Adaptive Decoding Procedure
unreliable bits
bit nodes
………….
………..
…………….
check nodes

22. More Details about the Matrix Adaptive Scheme
Consider the previous example: (7,4)Hamming code
transmitted codeword
received vector
parity check matrix
We can guaranteed reduce some (n-k)m columns to degree 1
We attempt to chose these to be the least reliable independent bits
Least Reliable Basis

23. Interpretation as an Optimization Procedure
Standard iterative decoding procedure is interpreted as gradient descent optimization (Lucas et al. 1998).
Proposed algorithm is a generalization, two-stage optimization procedure:
Bit reliabilities updating stage (gradient descent)
Iterative decoding is applied to generate extrinsic information
Extrinsic information is scaled by a damping coefficient and fed to update the bit-level reliabilities
Parity check matrix update (change direction)
All bit-level reliabilities are sorted by their absolute values
Systemize the sub-matrix corresponding to LRB in the parity check matrix
The damping coefficient serves to control the convergent dynamics.

24. A Hypothesis Adaption help gradient descent to converge
Stuck at pseudo-equilibrium point
Adaption help gradient descent to converge

25. Complexity Analysis Binary Floating Point Operation
Check Node Update
Overall Complexity
Variable Node Update
Matrix Adaption
Reliability Ordering
Binary
Floating Point
Operation
The complexity is in polynomial time with or
Complexity can be even reduced when implemented in parallel

26. Complexity Comparison
Method
Dominant Complexity
GMD
Chase KV
OSD
Trellis
ADP

27. Variation1: Symbol-level Adaptive Scheme
Systemizing the sub-matrix involves undesirable Gaussian elimination.
This problem can be detoured via utilizing the structure of RS codes.
We implement Symbol-level adaptive scheme.
least reliable symbols
This step can be efficiently realized using Forney’s algorithm (Forney 1965)
binary mapping

28. Variation2: Degree-2 sub-graph in the unreliable part
Reduce the “unreliable” sub-matrix to a sparse sub-graph rather than an identity to improve the asymptotic performance.
bit nodes
………….
………..
…………….
check nodes
unreliable bits
weakly connected

29. Variation2: Degree-2 sub-graph in the unreliable part (cont’d)
Q: How to adapt the parity check matrix?

30. Variation3: Different grouping of unreliable bits (cont’d)
Some bits at the boundary part may also have the wrong sign.
Run the proposed algorithm several times, each time with an exchange of some “reliable” and “unreliable” bits at the boundary.
Consider the received LLR of an RS(7,5) code:
Group1
Group2
…….
A list of candidate codewords are generated using different groups. Pick up the most likely from the list.

31. Variation4: Partial updating scheme (cont’d)
The main complexity comes from updating the bits in the high density part, however, only few bits at the boundary part will be affected.
In variable node updating stage: update only the “unreliable” bits in the sparse sub-matrix and a few “reliable” bits at the boundary part.
In check node updating stage: make an approximation of the check sum via taking advantage of the ordered reliabilities.
Complexity in floating point operation part is reduced to be
ascending reliability
Maximum-likelihood SISO decoding and variations
Trellis based decoding using the binary image expansions of RS codes over GF(2m) (Vardy & Be’ery 1991)
Reduced complexity version (Ponnampalam & Vucetic 2002)
Iterative SISO algorithms
Sub-trellis structure self-concatenation and iterative decoding of RS codes using their binary image (Ungerboeck 2003)
Stochastic shifting based iterative decoding of RS codes using their binary image (Jing & Narayanan, submitted 2003)
Iterative decoding via sparse factor graph representations of RS codes based on a fast Fourier transform (FFT) (Yedidia, MERL report 2003)

32. Applications
Q: How do the proposed algorithm and its variations perform?
Simulation results:
Proposed algorithm and variations over AWGN channel
Performance over symbol level fully interleaved slow fading channel
RS coded turbo equalization (TE) system over EPR4 channel
RS coded modulation over fast fading channel
Simulation setups:
A “genie aided” HDD is assumed for AWGN and fading channel.
In the TE system, all coded bits are interleaved at random. A “genie aided” stopping rule is applied.
The performance of the proposed algorithm and its variations are verified via computer simulation.
Proposed algorithm is implemented in conjunction with a “genie aided” hard decision decoding over AWGN and fading channels to speed up simulation.
In the turbo equalization system, no hard decision is assumed, however, when all bits converges to the correct value, iteration stops.

33. Additive White Gaussian Noise (AWGN) Channel

34. AWGN Channels

35. AWGN Channels (cont’d)
Asymptotic performance is consistent with the ML upper-bound.

36. AWGN Channels (cont’d)

37. AWGN Channels (cont’d)

38. Remarks
Proposed scheme performs near ML for medium length codes.
Symbol-level adaptive updating scheme provides non-trivial gain.
Partial updating incurs little penalty with great reduction in complexity.
For long codes, proposed scheme is still away from ML decoding.
Q: How does it work over other channels?
Generic algorithm provides near
Symbol level adaptive updating scheme provides comparable performance with Chase and KV algorithms.
Partial updating scheme incurs little penalty while greatly reduces the complexity.
For long codes, e.g., RS(255,239), the proposed algorithms, though provides significant gain over hard decision decoding, are still quite away ML performance.

39. Interleaved Slow Fading Channel

40. Fully Interleaved Slow Fading Channels

41. Fully Interleaved Slow Fading Channels (cont.)

42. Turbo Equalization Systems

43. Embed the Proposed Algorithm in the Turbo Equalization System
source
RS Encoder
interleaving
PR Encoder
sink
hard decision +
AWGN
RS Decoder
BCJR Equalizer
de-interleaving
RS Coded Turbo Equalization System - +
a priori
extrinsic
interleaving

44. Turbo Equalization over EPR4 Channels

45. Turbo Equalization over EPR4 Channels

46. RS Coded Modulation

47. RS Coded Modulation over Fast Rayleigh Fading Channels

48. RS Coded Modulation over Fast Rayleigh Fading Channels (cont’d)

49. Remarks
More noticeable gain is observed for fading channels, especially for symbol-level adaptive scheme.
In RS coded modulation scheme, utilizing bit-level soft information seems provide more gain.
The proposed TE scheme can combat ISI and performs almost identically as the performance over AWGN channels.
The proposed algorithm has a potential “error floor” problem.
However, simulation down to even lower FER is impossible.
Asymptotic performance analysis is still under investigation.
For EPR4 channels, the proposed turbo equalization scheme can mitigate the effect ISI and performs almost identically as the performance over AWGN channels, while other SIHO algorithms are not efficient to generate soft information.
For symbol-level fully interleaved slow fading channels, more noticeable gain is observed for RS codes of practical length, especially for symbol-level adaptive scheme.
For RS coded modulation over fast Rayleigh fading channels, proposed algorithm also outperforms hard decision decoding via more efficiently utilizing bit level soft information as long as Gray mapping is applied.
For long codes, an “error floor” phenomenon of the proposed algorithm is observed. Simulation down to even lower FER is impossible. Asymptotic performance analysis is under investigation.

50. Conclusion and Future work
Iterative decoding of RS codes based on adaptive parity check matrix works favorably for practical codes over various channels.
The proposed algorithm and its variations provide a wide range of complexity-performance tradeoff for different applications.
More works under investigation:
Asymptotic performance bound.
Understanding how this algorithm works from an information theoretic perspective, e.g., entropy of ordered statistics.
Improving the generic algorithm using more sophisticated optimization schemes, e.g., conjugate gradient method.
Iterative decoding of RS codes based on adaptive parity check matrix works favorably for practical codes both over AWGN channels and in TE systems.
The proposed algorithm and its variations provide a wide range of complexity-performance tradeoff for different applications.
More works under investigation:
Asymptotic performance bound.
Understanding how this algorithm works from an information theoretic perspective (mutual information of ordered statistics).
Improving the generic algorithm using more sophisticated optimization schemes, e.g., conjugate gradient method.

51. Thank you!