Huffman Coding

Main properties ： –Use variable-length code for encoding a source symbol. –Shorter codes are assigned to the most frequently used symbols, and longer codes to the symbols which appear less frequently. –Unique decodable & Instantaneous code. –It was shown that Huffman coding cannot be improved or with any other integral bit-width coding stream.

Example Number of pixels Gray level a.Step 1:Histogram

d.Step 4:Reorder and add until only two values remain b.Step 2:Order c.Step 3:Add

a.Assign 0 and 1 to the rightmost probabilities b.Bring 0 and 1 back along the tree c.Append 0 and 1 previously added branches d.Repeat the process until the original branch is labeled

1.846 bits/pixel =1.9 bits/pixel

Arithmetic Coding

Huffman coding has been proven the best fixed length coding method available. Yet, since Huffman codes have to be an integral number of bits long, while the entropy value of a symbol may (as a matter of fact, almost always so) be a faction number, theoretical possible compressed message cannot be achieved.

A rithmetic Coding For example, if a statistical method assign 90% probability to a given character, the optimal code size would be 0.15 bits. The Huffman coding system would probably assign a 1-bit code to the symbol, which is six times longer than necessary.

A rithmetic Coding Arithmetic coding bypasses the idea of replacing an input symbol with a specific code. It replaces a stream of input symbols with a single floating point output number.

Character probability Range ^(space) 1/10 A 1/10 B 1/10 E 1/10 G 1/10 I 1/10 L 2/10 S 1/10 T 1/10 BILL GATES” Suppose that we want to encode the message “ BILL GATES”

A rithmetic Coding Encoding algorithm for arithmetic coding ： low = 0.0 ; high =1.0 ; while not EOF do range = high - low ; read(c) ; high = low + range  high_range(c) ; low = low + range  low_range(c) ; end do output(low);

A rithmetic Coding To encode the first character B properly, the final coded message has to be a number greater than or equal to 0.20 and less than –range = 1.0 – 0.0 = 1.0 –high = × 0.3 = 0.3 –low = × 0.2 = 0.2 After the first character is encoded, the low end for the range is changed from 0.00 to 0.20 and the high end for the range is changed from 1.00 to 0.30.

A rithmetic Coding The next character to be encoded, the letter I, owns the range 0.50 to 0.60 in the new subrange of 0.20 to So, the new encoded number will fall somewhere in the 50th to 60th percentile of the currently established. Thus, this number is further restricted to 0.25 to 0.26.

A rithmetic Coding Note that any number between 0.25 and 0.26 is a legal encoding number of ‘BI’. Thus, a number that is best suited for binary representation is selected. (Condition : the length of the encoded message is known or EOF is used.)

( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T ( ) A B E G I L S T

A rithmetic Coding CharacterLowHigh B I L L ^(space) G A T E S

A rithmetic Coding So, the final value (or, any value between and , if the length of the encoded message is known at the decode end), will uniquely encode the message ‘BILL GATES’.

A rithmetic Coding Decoding is the inverse process. Since falls between 0.2 and 0.3, the first character must be ‘B’. Removing the effect of ‘B’from by first subtracting the low value of B, 0.2, giving Then divided by the width of the range of ‘B’, 0.1. This gives a value of

A rithmetic Coding Then calculate where that lands, which is in the range of the next letter, ‘I’. The process repeats until 0 or the known length of the message is reached.

A rithmetic Coding Decoding algorithm ： r = input_number repeat search c such that r falls in its range output(c) ; r = r - low_range(c); r = r ÷ (high_range(c) - low_range(c)); until EOF or the length of the message is reached

r cLow High range B I L L ^(space) G A T E S

A rithmetic Coding In summary, the encoding process is simply one of narrowing the range of possible numbers with every new symbol. The new range is proportional to the predefined probability attached to that symbol. Decoding is the inverse procedure, in which the range is expanded in proportion to the probability of each symbol as it is extracted.

A rithmetic Coding Coding rate approaches high-order entropy theoretically. Not so popular as Huffman coding because ×, ÷ are needed.