1. Lossy Compression Based on spatial redundancy Measure of spatial redundancy: 2D covariance Cov X (i,j)=  2 e -  (i*i+j*j) Vertical correlation    Horizontal correlation    For images we assume equal correlations Typically e -        Measure of loss (or distortion): MSE between encoded and decoded image E[X(i,j)X(i-1,j)] E[X 2 (i,j)] E[X(i,j)X(i,j-1)] E[X 2 (i,j)]

2. Rate-Distortion Function Tradeoff between bit rate (R) of compressed image and distortion (D) R measured in its per encoder output symbol Compression ratio = encoder input bits/R D normalized by the variance of the encoder input Possible SNR definition = 10 log 10 D -1 For images that can be modeled as uncorrelated Gaussian R(D)=0.5log 2 D -1 More realistic images See graph How do you make these graphs?

3. Sample vs. Block-based Coding Sample-based In spatial or frequency domain Like the JPEG-LS Make a predictor function (often weighted sum) Compute and quantize residual Encode Block-based Spatial: group pixels into blocks, compress blocks Transform: group into blocks, transform, encode

4. Which Transformation? Considerations Packing the most energy in the least number of elements Minimizing the total entropy of the sequence Decorrelating elements in the input blocks maximally Coding complexity DFT, KLT, DCT, DHT? See the effects DCT-based coding High compaction efficiency for correlated data Orthogonal and separable Fast, approximate DCT algorithms available

5. The JPEG standard Operating modes Lossless Sequential DCT-based Progressive DCT-based Hierarchical

6. JPEG: The Process Preprocess colors Divide the image into 8 pixel x 8 pixel non- overlapping blocks – why 8? Transform each block into 2D DCT Encode Stage 1: predictive coder for DC or run-length coder for AC coefficients Stage 2: Huffman or arithmetic coding

7. JPEG: Color Processing Maximum number of color components = 256 Each sample may be 8 or 12 bits in precision Conversion of a decorrelated color space Y-Cb-Cr, YUV, CIELAB Subsample chrominance components Interleave components Maximum MCU components=4 Maximum number of data units within MCU = 10

8. JPEG: Quantization Tables 8 X 8 quantization table for each image component Q(i,j): quantization step for corresponding DCT element 1  Q(i,j)  255 Psycho-visual experiments Bit-rate control Total number of block = B y k [i,j] : (i,j) output of k-th block target av. bit-rate bits per DCT coeff for 8-bit images

9. JPEG: Entropy Coding Baseline processing Total of 4 code tables allowed Different code tables for luminance and chrominance DC coefficients DC differentials are computed and have the range [-2047, 2047] The range is divided into 12 size categories Category i needs i bits to represent the value DC residuals are represented as [size, amplitude] pairs Size is Huffman encoded

10. JPEG: Entropy Coding AC coefficients Take value in the range [-1023, 1023] 10 size categories Only non-zero coefficients need to be encoded Processed in zig-zag order More efficient run-length encoding of AC coefficients Represented as (run/size, amplitude) If run > 15, possibly several (15/0) symbols are used

11. JPEG: Progressive Coding Coding is performed sequentially but in multiple scans The first scan should produce the full image without all details, and details are provided in successive scans Spectral selection Each block divided into frequency bands Each band transmitted during a different scan Successive approximation Given a frequency band, DCT coefficients are divided by 2 k MSBs are transmitted first

12. Subband Coding Pass an image through an n-band filter bank Possibly subsample each filtered output Encode each subband separately Compression may be achieved by discarding unimportant bands Advantages Fewer artifacts than block-coded compression More robust under transmission errors Selective encoding/decoding possible More expensive

13. Wavelet Compression Special case of subband compression Space and frequency-limited mother function  t) Function family:  mn  t) = a 0 -m/2  a 0 -m t - nb 0 ) If a 0 =2 and b 0 =1, the  mn  t) function is an orthonormal basis function f(t) = The a 0 -m term scales the signal Scaling function  mn  t) = 2 -m/2  2 -m t - n) Unscaled high-pass filter downscaled low-pass filter