CSI-447: Multimedia Systems Chapter 8: Data Compression (b)

Outline Entropy Encoding Predictive Coding Arithmetic Coding Lossless Predictive Coding Differential Coding Lossy Predictive Coding Differential Pulse Code Modulation Coding (DPCM) Delta Modulation (DM)

Entropy Encoding: Arithmetic Coding Initial idea introduced in 1948 by Shannon Many researchers worked on this idea Modern arithmetic coding can be attributed to Pasco (1976) and Rissanen and Langdon (1979) Arithmetic coding treats the whole message as one unit In practice, the input data is usually broken up into chunks to avoid error propagation

Entropy Encoding: Arithmetic Coding A message is represented by a half-open interval [a,b), where a,b. General idea of encoding Map the message into an open interval [a,b) Find a binary fractional number with minimum length that belongs to the above interval. This will be the encoded message Initially, [a,b) = [0,1) When the message becomes longer, the length of the interval shortens and the # of bits needed to represent the interval increases

Entropy Encoding: Arithmetic Coding Coding Algorithm Algorithm // Input: ArithmeticCoding symbol: Input stream of the message terminator: terminator symbol // : Low[] and High[]: all symbols’ // ranges the message Output: binary fractional code of low = 0; high = 1; range = 1; do { get (symbol); high = low + low = low + range * High(symbol); range * Low(symbol); – low; range = high } while (symbol != terminator) return CodeWord(low,high);

Entropy Encoding: Arithmetic Coding Binary code generation Algorithm CodeWord // Input: low and high // Output: binary fractional code = 0; k = 1; while (value(code) < low) { assign 1 to the kth binary if (value(code) > high) code fraction bit; replace the kth bit by 0; k++; }

Entropy Encoding: Arithmetic Coding Example: Assume S = {A,B,C,D,E,F,$}, where $ is the terminator symbol. In addition, assume the following probabilities for each character: – Pr (A) = 0.2 – Pr(B) = 0.1 – Pr(C) = 0.2 – Pr(D) = 0.05 – Pr(E) = 0.3 – Pr(F) = 0.05 – Pr($) = 0.1 Generate the fractional binary code of the message CAEE

Entropy Encoding: Arithmetic Coding It can be proven that ⎡log2 (1/ Pi)⎤ is the upper bound on the number of bits needed to encode a message In our case, the maximum is equal to 12. When the length of the message increases, the range decreases and the upper bound value ...... Generally, arithmetic coding outperforms Huffman coding Treats the whole message as one unit vs. an integral number of bits to code each character in Huffman coding Redo the previous example CAEE$ using Huffman coding and notice how many bits are required to code this message.

Entropy Encoding: Arithmetic Coding Decoding Algorithm ArithmeticDecoding // Input: code: binary code // : Low[] and High[]: all // Algorithm symbols’ ranges Output: The decoded message value = Do { find a convert2decimal(code); symbol s so that Low(s) <= value < High(s); s; output low = Low(s); high = High(s); range = high – low; value = (value – low) / range; } while s is not the terminator symbol;

Entropy Encoding: Arithmetic Coding Example

Predictive Coding Predictive coding simply means transmitting differences Predict the next sample as being equal to the current sample More complex prediction schemes can be used Instead of sending the current sample, send the error involved in the previous assumption

Predictive Coding: Why? The idea of forming differences is to make the histogram of sample values more peaked. In this case, what happens to the entropy? As a result, which is better to compress?

Predictive Coding: Why?

Lossless Predictive Coding Formally, define the integer signal as the set of values fn. Then, we predict values f^n and compute the error en as follows: t ˆ   ank fnk fn k 1 ˆ en  fn  fn when t = 1, we get ... Usually, t is between 2 and 4 (in this case it is called a linear predictor) We might need to have a truncating or rounding operation following the prediction computation

Lossless Predictive Coding

Lossless Predictive Coding: Example Consider the following predictor: ⎢ 1 ⎥ ˆ fn  ⎢  fn1  fn2 ⎥ ⎣ 2 ⎦ ˆ en  fn  fn Show how to code the following sequence f1 , f 2 , f3 , f 4 , f5  21, 22 , 27 , 25, 22 .

Lossless Predictive Coding Examples in the Image Compression Domain Differential Coding Lossless JPEG

Lossy Predictive Coding: DPCM DPCM = Differential Pulse Code Modulation Form the prediction f ^ n Form an error en Quantize the error

Lossy Predictive Coding: DPCM The distortion is the average squared error To illustrate the quality of a compression scheme, diagrams of distortion vs. the number of bit levels used are usually shown Quantization used Uniform Lloyd-Max – Does better than “Uniform”

Lossy Predictive Coding: DPCM

Lossy Predictive Coding: DPCM Example ⎢ 1 ~ f  f  ~ ⎥ ˆ f  ⎢ 2 ⎥ n n1 n2 ⎣ ⎦ ˆ en  fn  fn e~  Q[e ]  16 * ⎢ 255  en ⎥  256  8 ⎢ ⎥ n n ⎣ 16 ⎦ ~ ˆ ~ fn  fn  en Show how to code the following sequence f1 , f 2 , f 3 , f 4 , f 5  130 , 150 , 140 , 200 , 230

Lossy Predictive Coding DM (Delta Modulation) is a simplified version of DPCM that is used as a quick analog-to-digital converter. – Note that the prediction simply involves a delay