By Dr. Hadi AL Saadi Lossy Compression

Source coding is based on changing of the original image content. Also called semantic-based coding High compression rates may be high but a price is a loss of information. Lossy Compression (Source Coding )

Block Based Transform Coding Transform coding is performed by taking an image and breaking it down into sub-image (block) of size nxn. The transform is then applied to each sub-image (block) and the resulting transform coefficients are quantized and entropy coded. ~ (N/n) 2 subimages

Block Based Transform Coding o Consider the following block of data o For NxN a 2-Dimensional transform can be carried out in a separable way (i.e. first down the columns and then along the rows).  The 1-Dimensional transform is calculated according to Where [T] is the transform matrix 4x4 block image

4x4 block

A 2-dimentional transform can be extended according to

JPEG is an image compression standard which was accepted as an international standard in New standard called JPEG 2000 is being developed and target for 2001 Developed by the Joint Photographic Expert Group of the ISO/IEC For coding and compression of color/gray scale images Yields acceptable compression in the 10:1 range JPEG is a lossy compression technique Based on the DCT ( Discrete Cosine Transform ) JPEG is a general image compression technique independent of  Image resolution  Image and pixel aspect ratio  Color system  Image complexity A scheme for video compression based on JPEG called Motion JPEG (MJPEG) exists JPEG : Joint Photographers Experts Group ( Standards)

 JPEG is effective because of the following three observations  Image data usually changes slowly across an image, especially within an 8x8 block  Therefore images contain much redundancy  Experiments indicate that humans are not very sensitive to the high frequency data images  Therefore we can remove much of this data using transform coding  Humans are much more sensitive to brightness (luminance) information than to color (chrominance)  JPEG uses Chroma subsampling (4:2:0) JPEG : Joint Photographers Experts Group

General Overview of JPEG Process

The main steps in JPEG encoding are the following Transform RGB to YUV or YIQ and subsample color DCT on 8x8 image blocks Quantization Zig-Zag ordering and run-length encoding Entropy coding General Overview of JPEG Process

Divide into 8x8 blocks This is an example

RGB vs. YC b C r The precision of colors suffer less (for a human eye) than the precision of contours (based on luminance) Color space conversion makes use of it! Simple color space model: [R,G,B] per pixel JPEG uses [Y, Cb, Cr] Model Y = Brightness Cb = Color blueness Cr = Color redness

Convert RGB to YCC 8x8 pixel 1 pixel = 3 components

Downsampling 4 blocks 16 x16 pixel The three different chrominance downsampling format 4:4:4

4 blocks 16 x16 pixel 4:2:2 4:2:0 Downsampling

Discrete Cosine Transform (DCT) Matrix The M-by-M DCT transform matrix T is given by Where M may be 4, 8 or 16 The two-dimensional DCT of A can be computed as

T(2X2)= T(4X4)= T( 8x8)=

The 4-by-4 DCT basis

The 8-by-8 DCT basis u v

JPEG - image preparation  Single 2D matrix for 8-bit monochrome image  Color look up table – CLUT  3 D matrices (R,G and B) for color images  The alternative form Y, Cb, Cr could be alternatively used, producing a reduced form over the equivalent RGB

JPEG Steps (Gray Image ) 1. Divide the image into 8x8 subimages; For each subimage do: 2. Shift the gray-levels in the range [-128, 127] - DCT requires range be centered around 0 3. Apply DCT (i.e., 64 coefficients) 1 DC coefficient: F(0,0) 63 AC coefficients: F(u,v)

subtract 128 from each pixel DCT Transformation Quantize scan and form descriptors Entropy code 8X8 image Block Predict Encode AC Coefficients DC Coefficients Add 128 from each pixel IDCT Transformation Dequantize Decode descriptors reform Block Entropy code 8X8 image Block Predict Encode File or Transmit compressed data JPEG Steps

The image matrix is divided into a set of smaller 8x8 sub matrices. Each is known as a block, which are fed in a sequence to the DCT (after subtraction 128 from each pixel ) that transforms each block separately JPEG Block Preparation

JPEG – Forward DCT Transform

Shift operations From [0, 255] To [-128, 127] DCT Result Meaning of each position in DCT result- matrix

Examples of DCT Computation

Quantization Quantization in JPEG aims at reducing the total number of bits in the compressed image  Divide each entry in the frequency space block by an integer, then round  Use a quantization matrix Q(u, v)

Quantization Use larger entries in Q for the higher spatial frequencies  These are entries to the lower right part of the matrix  The following slide shows the default Q(u, v) values for luminance and chrominance  Based on psychophysical studies intended to maximize compression ratios while minimizing perceptual distortion  Since after division the entries are smaller, we can use fewer bits to encode them

Quantization

Multiple quantization matrices can be used (perhaps by scaling the defaults), allowing the user to choose how much compression to use  Trades off quality vs. compression ratio  More compression means larger entries in Q  An example of JPEG coding and decoding on one image block is shown next

 Each DCT coefficient is quantized independently using an uniform quantizer Given x=54, and quantization steps q1=10,q2=5 Integer Round(x/q1)=5, reverse process =x *10=50 error=4 Integer Round(x/q2)=11, reverse process =x *5=55 error=1 Smaller quantization step  greater accuracy,less error The number of quantization intervals per each DCT channel is chosen independently for each coefficient After quantization coefficient of a block is encoded as a difference from the DC term of the previous block in the processing order. This is because there is usually strong correlation between the DCT coefficient of adjacent block Rule for choosing the quantization intervals: Low-frequency coefficients usually have more energy than high-frequency coefficients,therefore require more quantization intervals The Human Visual System is more sensitive to Luminance channels than chrominance channels, require more quantization intervals than chrominance channels. Note : the quantization table is not standardized DCT Coefficient Quantization

Original and DCT coded block DCT result

Quantized and Reconstructed Blocks Quantization result

Quantization Example 2:

Preparation for Entropy Coding We have seen two main steps in JPEG coding: DCT and quantization The remaining steps all lead up to entropy coding of the quantized DCT coefficients  These additional data compression steps are lossless  Most of the lossiness is in the quantization step

Run-Length Coding We now do run-length coding  The AC and DC components are treated differently  Since after quantization we have many 0 AC components, RLC is a good idea  Note that most of the zero components are towards the lower right corner (high spatial frequencies)  To take advantage of this, use zigzag scanning to create a 64-vector

Zigzag Scan in JPEG

JPEG Vectoring Transform Coefficients DC coefficient AC coefficients Because there is usually strong correlation between the DC coefficients of adjacent 8×8 blocks, the quantized DC coefficient is encoded as the difference from the DC term of the previous block The other 63 entries are the AC components. They are treated separately from the DC coefficients in the entropy coding process

We set DC 0 = 0. DC of the current block DC i will be equal to DC i-1 + Diff i. Therefore, in the JPEG file, the first coefficient is actually the difference of DCs. Then the difference is encoded with Huffman coding algorithm together with the encoding of AC coefficients JPEG Vectoring Differential Coding :

5. Order the coefficients using zig-zag ordering - Place non-zero coefficients first - Create long runs of zeros (i.e., good for run-length encoding )

Run-Length Coding Now the RLC step replaces values in a 64-vector (previously an 8x8 block) by a pair (RUNLENGTH, VALUE), where RUNLENGTH is the number of zeroes in the run and VALUE is the next non-zero value  The zigzag value (32, 6, -1, -1, 0, -1, 0, 0, 0, -1, 0, 0, 1, 0, 0, …, 0)  This becomes (0,6) (0,-1)(1,-1)(3,-1)(2,1)(0,0) - Note that DC coefficient is ignored

Zigzag reordering / Run Length Coding Quantization result

Coding of DC Coefficients Now we handle the DC coefficients  1 DC per block  DC coefficients may vary greatly over the whole image, but slowly from one block to its neighbor (once again, zigzag order)  So apply Differential Pulse Code Modulation (DPCM) for the DC coefficients  If the first five DC coefficients are 150, 155, 149, 152, 144, we come up with DPCM code- 150, 5, -6, 3, -8

Entropy Coding Now we apply entropy coding to the RLC coded AC coefficients and the DPCM coded DC coefficients  The baseline entropy coding method uses Huffman coding on images with 8-bit components  DPCM-coded DC coefficients are represented by a pair of symbols (SIZE, AMPLITUDE)  SIZE = number of bits to represent coefficient  AMPLITUDE = the actual bits

Entropy Coding The size category for the different possible amplitudes is shown below  DPCM values might require more than 8 bits and might be negative

Entropy Coding  One’s complement is used for negative numbers  Codes 150, 5, -6, 3, -8 become  (8, ), (3, 101), (2, 11), (4, 0111)  Now the SIZE is Huffman coded  Expect lots of small SIZEs  AMPLITUDE is not Huffman coded  Pretty uniform distribution expected, so probably not worth while

Huffman encoding RLC result: [0, -3] [0, 12] [0, 3]......EOB After group number added: [0,2,00 b ] [0,4,1100 b ] [0,2,00 b ] EOB First Huffman coding (i.e. for [0,2,00 b ] ): [0, 2, 00 b ] => [100 b, 00 b ] ( look up e.g. table AC Chron) Total input: 512 bits, Output: 113 bits output ValuesGReal saved values 0 -1, 1 -3, -2, 2, 3 -7,-6,-5,-4,5,6, ,1 00, 01, 10, ,001,010,011,100,101,110,111.

JPEG for Color Images Encoder Decoder