Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Cutoff Rate and Other Limits: Passing the Impassable Richard E. Blahut University of Illinois UIUC 5/4/20151.

Similar presentations


Presentation on theme: "The Cutoff Rate and Other Limits: Passing the Impassable Richard E. Blahut University of Illinois UIUC 5/4/20151."— Presentation transcript:

1 The Cutoff Rate and Other Limits: Passing the Impassable Richard E. Blahut University of Illinois UIUC 5/4/20151

2 Shannon’s Ideal Channel Example: Binary Memoryless Channel Stationary Discrete Memoryless 5/4/20152

3 … … … … … … …. 1 1 … … … … … … A Large Code 5/4/20153

4 + + + A convolutional encoder 5/4/20154

5 Information theory asserts existence of good codes Coding theory wants practical codes and decoders There are binary codes 5/4/20155

6 Brief History of Codes Algebraic Block Codes 1948 Reed-Solomon codes (1960) Convolutional Codes 1954 Sequential decoding (1951) Viterbi algorithm (1967) Euclidean Trellis Codes 1982 Turbo Codes 1993 Gallager (LDPC) codes (1960) 5/4/20156

7 Decoders Maximum Likelihood Maximum Block Posterior Maximum Symbol Posterior Typical Sequence Iterative Posterior Minimum Distance Bounded Distance 5/4/20157

8 My View 1)Channel Capacity 2)Cutoff Rate 3)Critical Rate Distance -based codes Likelihood -based codes Posterior -based codes Polar codes 5/4/20158

9 For any fixed there is a sequence of codes for which exponentially in blocklength. This sequence does not approach Channel Error Exponent Fact #2 Every code satisfies Fact #1 Codes exist such that 5/4/20159

10 E(R) 5/4/201510

11 A sequence of codes drawn from a set of ensembles 5/4/201511

12 Channel Capacity Channel Critical Rate Channel Cutoff Rate 5/4/201512

13 Binary Hypotheses Testing Type 1 Error Type 2 Error 5/4/201513

14 Binary Hypotheses Testing Change Notation 5/4/201514

15 Bounds onand Upper Bounds on Sphere Packing Bound Minimum Distance Bound Lower Bounds on Random Coding Bound Expurgated Bound 5/4/201515

16 Bhattacharyya Distance 5/4/201516

17 175/4/2015

18 is quadratic near Let be a sequence with Then with so A Code Sequence Approaching Capacity with if 5/4/201518

19 Capacity: C Shannon(1948) Cutoff Rate: Jacobs & Berlekamp(1968) Massey(1981) Arikan(1985/1988) Error Exponent: Gallager(1965) Forney(1968) Blahut(1972) 5/4/201519

20 Gallager (1965) Forney (1968) Blahut (1972) where is the Kullback divergence 5/4/201520

21 Forney’s List Decoding Likelihood Function Likelihood Ratio 5/4/201521

22 Sequential Decoding Exponential waiting time Work exponential in time Pareto Distribution with Work unbounded if Sequential decoding fails if Is maximum likelihood decoding sequential decoding? 5/4/201522

23 Two Pareto parameters and Pareto Distribution 5/4/201523

24 Start with an exponential distribution If is exponential, then is a Pareto distribution The Origin of a Pareto Distribution 5/4/201524

25 The Origins of Graph-Based Codes Brillouin deBrogle Shannon Battail (1987) Hagenauer (1989) Berrou et al (1993) 5/4/201525

26 Coding Beyond the Cutoff Rate Parallel – Pinsker Hybrid – Jelinek Turbo – Berrou/Glavieux LDPC – Gallager/Tanner/Wiberg Polar - Arikan 5/4/201526

27 The Massey Distraction (1981) QECBEC QEC 2 BEC 5/4/201527

28 Performance Measures Bit Error Rate vs. Message Error Rate 5/4/201528

29 The Arikan Retraction 5/4/201529

30 5/4/201530

31 5/4/201531

32 The Arkan Redistraction* *Rhetorical 5/4/201532


Download ppt "The Cutoff Rate and Other Limits: Passing the Impassable Richard E. Blahut University of Illinois UIUC 5/4/20151."

Similar presentations


Ads by Google