1. Soft Decision Decoding Algorithms of Reed-Solomon Codes
Jing Jiang and Krishna R. Narayanan
Department of Electrical Engineering
Texas A&M University
Title

2. Historical Review of Reed Solomon Codes
Date of birth: 40 years ago (Reed and Solomon 1960)
Related to non-binary BCH codes (Gorenstein and Zierler 1961)
Efficient decoder: not until 6 years later (Berlekamp 1967)
Linear feedback shift register (LFSR) interpretation (Massey 1969)
Other algebraic hard decision decoder:
Euclid’s Algorithm (Sugiyama et al. 1975)
Frequency-domain decoding (Gore 1973 and Blahut 1979)
Most of the algebraic hard decision decoders are of similar complexity, which is about o(d^2)

3. Wide Range of Applications of Reed Solomon Codes
NASA Deep Space: CC + RS(255, 223, 32)
Multimedia Storage:
CD: RS(32, 28, 4), RS(28, 24, 4) with interleaving
DVD: RS(208, 192, 16), RS(182, 172, 10) product code
Digitial Video Broadcasting: DVB-T CC + RS(204, 188)
Magnetic Recording: RS(255,239) etc. (nested RS code)
Recording channels are the most important application, perhaps!

4. Basic Properties of Reed Solomon Codes
Two forms of representations are equivalent!

5. Basic Properties of Reed Solomon Codes (cont’d)
Properties of RS code:
Symbol level cyclic (nonbinary BCH codes)
Maximum distance separable (symbol level):
Properties of BM algorithm:
Decoding region:
Decoding complexity: Usually
Basic properties

6. Motivation for RS Soft Decision Decoder
source
RS Encoder
interleaving
PR Encoder
sink
hard decision +
AWGN
RS Decoder
Channel Equalizer
de-interleaving
RS Coded Turbo Equalization System - +
a priori
extrinsic
interleaving
Hard decision decoder does not fully exploit the decoding capability
Efficient soft decision decoding of RS codes remains an open problem
The motivation of an efficient SISO decoder for RS codes
Soft input soft output (SISO) algorithm is favorable

7. Presentation Outline Symbol-level algebraic soft decision decoding
Binary expansion of RS codes and soft decoding algorithms
Iterative decoding for RS codes
Simulation results
outline
Applications and future works

8. Symbol-level Algebraic Soft Decision Decoding

9. Reliability Assisted Hard Decision Decoding
Generalized Minimum Distance (GMD) Decoding (Forney 1966):
New distance measure: generalized minimum distance
Successively erase the least reliable symbols and run the hard decision decoder
GMD is shown to be asymptotically optimal
Chase Type-II decoding (Chase 1972):
Exhaustively flip the least reliable symbols and run the hard decision decoder
Chase algorithm is also shown to be asymptotically optimal
Related works:
Fast GMD (Koetter 1996)
Efficient Chase (Kamiya 2001)
Combined Chase and GMD for RS codes (Tang et al. 2001)
Performance analysis of these algorithms for RS codes seems still open
Generalized Minimum Distance (GMD) Decoding (Forney 1966)
Chase decoding (Chase 1972)
Related works

10. Algebraic Beyond Half dmin List Decoding
Bounded distance + 1 decoding (Berlekamp 1996)
Beyond decoding for low rate RS codes (Sudan 1997)
Decoding up errors (Guruswami and Sudan 1999)
A good tutorial paper (JPL Report, McEliece 2003)
Algebraic beyond half dmin decoding, which is the kernel of KV algorithm

11. Outline of Algebraic Beyond Half Distance Decoding
Basic idea: find f(x), which fits as many points in pairs
Interpolation Step:
Construct a bivariate polynomial of minimum (1,K-1) degree, which has a zero of order at , i.e.:
Factorization Step:
generate a list of y-roots, i.e.:
Pick up the most likely codeword from the list L
Outline: 1) curve fitting!
2) multiplicity assignment, let (x,y) pass through (alpha, beta) m times
3) the larger the m, the larger the decoding radius
4) when beta fails in the decoding radius, (y-f(x)) can factorize Q(x,y)
5) smart way to construct Q(x,y) and factorize it
6) when soft information is available, can do weighted multiplicity assignment
Complexity:
Interpolation (Koetter’s fast algorithm):
Factorization (Roth and Ruckenstein’s algorithm):

12. Algebraic Soft Interpolation Based List Decoding
Koetter and Vardy algorithm (Koetter & Vardy 2003)
Based on the Guruswami and Sudan’s algebraic list decoding
Use the reliability information to assign multiplicities
KV is optimal in multiplicity assignment for long RS codes
Reduced complexity KV (Gross et al. submitted 2003)
Re-encoding technique: largely reduce the cost for high rate codes
VLSI architecture (Ahmed et al. submitted 2003)
KV 1) proportional assignment
2) asymptotically optimal

13. Soft Interpolation Based Decoding
Definition:
Reliability matrix:
Multiplicity matrix:
Score:
Cost:
Basic idea: interpolating more symbols using the soft information
The interpolation and factorization is the same as GS algorithm
Sufficient condition for successful decoding:
The complexity increases with , maximum number of multiplicity
KV 1) performance analysis becomes interesting
2) no longer a fixed radius
3) sufficient condition

14. Recent Works and Remarks
Recent works on performance analysis and multiplicity assignment:
Gaussian approximation (Parvaresh and Vardy 2003)
Exponential bound (Ratnakar and Koetter 2004)
Chernoff bound (El-Khamy and McEliece 2004)
Performance analysis over BEC and BSC (Jiang and Narayanan 2005)
The ultimate gain of algebraic soft decoding (ASD) over AWGN channel is about 1dB
Complexity is scalable but prohibitively huge for large multiplicity
The failure pattern of ASD algorithm and optimal multiplicity assignment scheme is of interest
KV 1) failure is interesting, can we do better?

15. Performance Analysis of ASD over Discrete Alphabet Channels
Performance Analysis over BEC and BSC (Jiang and Narayanan, accepted by ISIT2005)
The analysis gives some intuition about the decoding radius of ASD
We investigate the bit-level decoding radius for high rate codes
For BEC, bit-level radius is twice as large as that of the BM algorithm
For BSC, bit-level radius is slightly larger than that of the BM algorithm
In conclusion, ASD is limited by its algebraic engine
KV 1) failure is interesting, can we do better?

16. Binary Image Expansion of RS Codes and Soft Decision Decoding

17. Binary Image Expansion of RS Codes over GF(2m)
Life becomes much easier when we go to bit level

18. Bit-level Weight Enumerator
“The major drawback with RS codes (for satellite use) is that the present generation of decoders do not make full use of bit-based soft decision information” (Berlekamp)
How does the binary expansion of RS codes perform under ML decoding?
Performance analysis using its weight enumerator
Averaged ensemble weight enumerator of RS codes (Retter 1991)
It gives some idea about how RS codes perform under ML decoding
When turbo code comes, people realize convolutional codes are inherently bad codes, but how about algebraic codes?

19. Performance Comparison of RS(255,239)
Asymptotically be optimal, but practically suffer a loss!

20. Performance Comparison of RS(255,127)
More significant difference

21. Remarks RS codes themselves are good code
However, ML decoding is NP-hard (Guruswami and Vardy 2004)
Are there sub-optimal decoding algorithms using the binary expansions?
Motivation for bit level decomposition

22. Trellis based Decoding using BCH Subcode Expansion
Maximum-likelihood decoding and variations:
Partition RS codes into BCH subcodes and glue vectors (Vardy and Be’ery 1991)
Reduced complexity version (Ponnampalam and Vucetic 2002)
Soft input soft output version (Ponnampalam and Grant 2003)
Maximum-likelihood SISO decoding and variations
Trellis based decoding using the binary image expansions of RS codes over GF(2m) (Vardy & Be’ery 1991)
Reduced complexity version (Ponnampalam & Vucetic 2002)
SISO (Ponnampalam & Grant 2003)

23. Subfield Subcode Decomposition
BCH subcodes
Glue vector
Remarks:
Decomposition greatly reduces the trellis size for short codes
Impractical for long codes, since the size of the glue vectors is very large
Related work:
Construct sparse representation for iterative decoding (Milenkovic and Vasic 2004)
Subspace subcode of Reed Solomon codes (Hattori et al. 1998)
1) BCH subcode and glue vectors
2) Useful to construct sparse parity check matrices for short codes

24. Reliability based Ordered Statistic Decoding
Reliability based decoding:
Ordered Statistic Decoding (OSD) (Fossorier and Lin 1995)
Box and Match Algorithm (BMA) (Valembois and Fossorier 2004)
Ordered Statistic Decoding using preprocessing (Wu et al. 2004)
Basic ideas:
Order the received bits according to their reliabilities
Propose hard decision reprocessing based on the most reliable basis (MRB)
Remarks:
The reliability based scheme is efficient for short to medium length codes
The complexity increases exponentially with the reprocessing order
BMA algorithm trade memory for time complexity
Efficient for general linear block codes

25. Iterative Decoding Algorithms for RS Codes

26. A Quick Question
How does the panacea of modern communication, iterative decoding algorithm work for RS codes?
Note that all the codes in the literature, for which we can use soft decoding algorithms are sparse graph codes with small constraint length.

27. How does standard message passing algorithm work?
erased bits
bit nodes
………….
……….. ?
…………….
check nodes
The nice property of ldpc codes is that the parity check matrix is sparse, thus, the probability that two erased bits participate in another check diminishes.
If two or more of the incoming messages are erasures the check is erased
For the AWGN channel, two or more unreliable messages invalidate the check

28. A Few Unreliable Bits “Saturate” the Non-sparse Parity Check Matrix
Consider RS(7, 5) over GF(23) :
Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)
Even in the optimistic case, erasure channel, it won’t give good results.
Iterative decoding is stuck due to only a few unreliable bits “saturating” the whole non-sparse parity check matrix

29. Sparse Parity Check Matrices for RS Codes
Can we find an equivalent binary parity check matrix that is sparse?
For RS codes, this is not possible!
The H matrix is the G matrix of the dual code
The dual of an RS code is also an MDS Code
Each row has weight at least (K+1)
Typically, the row weight is much higher
It is impossible to have a sparse representation of RS codes

30. Iterative Decoding for RS Codes
Iterative decoding for general linear block codes:
Iterative decoding for general linear block codes (Hagenauer et al. 1996)
APP decoding using minimum weight parity checks (Lucas et al. 1998)
Generalized belief propagation (Yedidia et al. 2000)
Recent progress on RS codes:
Sub-trellis based iterative decoding (Ungerboeck 2003)
Stochastic shifting based iterative decoding (Jiang and Narayanan, 2004)
Sparse representation of RS codes using GFFT (Yedidia, 2004)
Iterative decoding for general linear block codes (tough problem!)

31. Recent Iterative Techniques
Sub-trellis based iterative decoding (Ungerboeck 2003)
Self concatenation using sub-trellis constructed from the parity check matrix:
Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)
Remarks:
Performance deteriorates due to large number of short cycles
Work for short codes with small minimum distances

32. Recent Iterative Techniques (cont’d)
Stochastic shifting based iterative decoding (Jiang and Narayanan, 2004)
Due to the irregularity in the H matrix, iterative decoding favors some bits
Taking advantage of the cyclic structure of RS codes
Shift by 2
Stochastic shift prevent iterative procedure from getting stuck
Best result: RS(63,55) about 0.5dB gain from HDD
However, for long codes, the performance deteriorates

33. Proposed Iterative Decoding for RS Codes

34. Iterative Decoding Based on Adaptive Parity Check Matrix
Idea: reduce the sub-matrix corresponding to unreliable bits to a sparse nature using Gaussian elimination
For example, consider (7,4) Hamming code:
transmitted codeword
received vector
parity check matrix
Since the parity check matrix is of full rank, we are guaranteed to get (n-k) LRB
We can make the (n-k) less reliable positions sparse!

35. Adaptive Decoding Procedure
unreliable bits
bit nodes
………….
………..
…………….
check nodes
After the adaptive update, iterative decoding can proceed

36. Gradient Descent and Adaptive Potential Function
Geometric interpretation (suggested by Ralf Koetter)
Define the tanh domain transform as:
The syndrome of a parity check can be expressed as:
Define the soft syndrome as:
Define the cost function as:
The decoding problem is relaxed as minimizing J using gradient descent with the initial value T observed from the channel
Define syndrome as binary summation of all participating bits
Define the soft syndrome product of the channel reliabilities
J is minimized iff the decoding converges to a valid codeword
J is also a function of H. It is adapted such that unreliable bits are separated in order to avoid getting stuck at zero gradient points:

37. Two Stage Optimization Procedure
Proposed algorithm is a generalization of the iterative decoding scheme proposed by Lucas et al. (1998), two-stage optimization procedure:
A variation of standard gradient descent
The damping coefficient serves to control the convergent dynamics

38. Avoid Zero Gradient Point
Adaptive scheme changes the gradient and prevents it getting stuck at zero gradient points
Zero gradient point
Think about erasure channel. This does happen! J is not minimized but we get zero gradient!

39. Variations of the Generic Algorithm
Connect unreliable bits as deg-2
Incorporate this algorithm with hard decision decoder
Adapting the parity check matrix at symbol level
Exchange bits in reliable and unreliable part. Run the decoder multiple times
Reduced complexity partial updating scheme
Not going deep to details!

40. Simulation Results

41. AWGN Channels

42. AWGN Channels (cont’d)
Asymptotic performance is consistent with the ML upper-bound.

43. AWGN Channels (cont’d)

44. AWGN Channels (cont’d)
KV 0.65dB
BMA 1dB
With extreme huge complexity, we gain 1.6dB gain at FER = 10^-4.

45. Interleaved Slow Fading Channel

46. Fully Interleaved Slow Fading Channels

47. Fully Interleaved Slow Fading Channels (cont.)

48. Turbo Equalization Systems

49. Embed the Proposed Algorithm in the Turbo Equalization System
source
RS Encoder
interleaving
PR Encoder
sink
hard decision +
AWGN
RS Decoder
BCJR Equalizer
de-interleaving
RS Coded Turbo Equalization System - +
a priori
extrinsic
interleaving

50. Turbo Equalization over EPR4 Channels

51. Turbo Equalization over EPR4 Channels

52. RS Coded Modulation

53. RS Coded Modulation over Fast Rayleigh Fading Channels

54. Applications and Future Works

55. Potential Problems in Applications
“In theory, there is no difference between theory and practice.
But, in practice, there is…” (Jan L.A. van de Snepscheut)
General Problems:
Coding gain may shrink down in practical systems
Concatenated with CC: difficult to generate the soft information
Performance in the practical SNRs should be analyzed
Respective problems for various decoding schemes:
Reliability assisted HDD: Gain is marginal in practical SNRs
Algebraic soft decoding: performance is limited by the algebraic nature
Reliability based decoding: huge memory, not scalable with SNR
Sub-code decomposition: only possible for very short codes
Iterative decoding: adapting Hb at each iteration is a huge cost
In practice, there is a huge difference!

56. A Case Study (System Setups)
Forward Error Control of a Digital Television Transmission Standard:
Modulation format: 64 or 16 QAM modulation (semi-set partitioning mapping)
Inner code: convolutional code rate=2/3 or 8/9
Bit-interleaved coded modulation (BICM)
Iterative demodulation and decoding (BICM-ID)
The decoded bytes from inner decoder are interleaved and fed to outer decoder
Outer code: RS(208,188) using hard decision decoding
Will soft decoding algorithm significantly improve the overall performance?
Case study!

57. A Case Study (Simulation Results)
Coding gain may shrink in practical systems

58. Future Works
How to incorporate the proposed ADP with other soft decoding schemes?
Taking advantage of the inherent structure of RS codes at bit level
More powerful decoding tool, e.g., trellis
Extend the idea of adaptive algorithms to demodulation and equalization
Apply the ADP algorithm to quantization or to solve K-SAT problems
No really successful bit level soft decoder which can handle long constraint length codes

59. Thank you!
Acknowledgements