Notes II Jay Weitzen University of Massachusetts Lowell

2

3

4

5

6

7

8

9

10

11

12

13

14

15 Information and Communication

16 Key Slide

17

18 Hard decision vs Soft Decision

19

20

21

22

23

24

25

26

27 Applications of Information Theory: Compression

28 Shannon’s First Theorem: A.K.A Source Coding Theorem

29 Shannon’s source coding theorem

30

31

32

33 Rate of a Source Code

34 2 nd order Block Codes and Huffman Encoding

35

36

37

38

39

40 Some Definitions

41 Higher Order Codes Converge

42

43

44

45