Information Theory & Coding for Digital Communications Prof JA Ritcey EE 417 Source; Anderson Digital Transmission Engineering 2005

Some Definitions R_b bit rate ( to be Information Bit Rate) E_b energy per (information )bit N_o Noise spectral density Watts/Hz E_b/N_o Energy per bit to Noise at Detector

Coding Digital operation --. Bits to bits Goal - increase minimum distance between constellation points, decrease reqd Eb/No Method - addition of parity or check bits Cost – bandwidth increase (Rb constant) Cost - Rate decrease (bandwidth constant)

Some Types of Coding for Forward Error Correction (FEC) Block codes – Hamming, Golay, Reed Solomon Can be binary or q-ary (byte based) Input k info bits maps to n channel bits Generally bandwidth must increase by (n/k) to hold the information rate constant Performance can increase with block size Decoding can be difficult for large block size Modern Block Codes are LDPC – low density parity check codes

Coding Taxonomy Block Codes with algebraic structure Convolutional Codes with Viterbi Decoding Coded Modulation (solves the BW growth problem) TCM, BICM Turbo Codes (1993) with iterative decoding LDPC (rediscovered circa 2000) Codes on Graphs and Message Passing Decoding

Memoryless Channel Models bits mapping to output bits (symbols) BSC(p) Crossover Probability p

Gilbert-Elliot Channel with Memory shows Good and Bad States Gilbert Elliot can models bursts of errors BSC (p) would take P_GB =p, P_BB =p, else 1-p

Gilbert-Elliot Model Markov Chain model 2 states G/B Stationary distribution of State Occupancy Let State 1 be G, State 2 be B, Stationary Distribution is P_1 = P_BG/(P_BG + P_GB) P_2 = P_GB/( P_BG + P_GB) For the BSC(p), P_1 = 1-p, P_2 = p

BSC (p) and BPSK Crossover p = Q( \sqrt [ 2 E_b/N_o] ) NO Not quite, should be Crossover p = Q( \sqrt [2 (k/n) E_b/N_o] ) Code rate = k/n < 1 and a dB loss!

Information Theory Shannon 1948 Ultimate limit to Digital Communications Depends on the Channel Abstracts the key features of physical channel Highly successful in predicting the trajectory of modern digital comm Key Quantity – Channel Capacity C

World View before Shannon Digital Comms will experience Errors measured at the BER – bit error rate To reduce the BER … Gather more receive energy Gather less receive noise Reduce the data Rate Should be a smooth transistion

IQ Modulation (Xmt & Rcv)

QAM – Quadrature Amplitude Modulation Constellations XMT on Left RCV on Right

Typical Error Rate Curves Error Prob vs Eb/No Can increase Data rate by sending more bits per symbol But, Noise immunity suffers

Classical Block Diagram – It is Suboptimal to separate functions!

Shannon’s View Why limit ourselves to just 2 dimensions? Any waveform is a point in N = 2TW dimensions (by sampling Theory) If a long block code is used (latency problems???), we can protect against errors There Exists a Number, C, Channel Capacity Signal at rate R_b < C, zero errors are possible

BSC(p) and BEC (p) Comparison

The Channel Model Matters! Different Channels may require additional system costs Coherent vs non-Coherent Channels Constraints on the Signaling Alphabet Constraints on the input to the decoder Soft (analog) vs Hard (binary) inputs

Comparison of Capacity in bits per channel usage

Shannon AWGN C/W = log2 ( 1+SNR) Notice the Gap to Capaccity

Conclusions Reviewed Memoryless Modulation Introduced Abstract Channels Information Theory & Channel Capacity Showed Typical Gains