Download presentation

Presentation is loading. Please wait.

1
**Lossy Image Compression**

From lossless to lossy Subjective and Objective Image Quality Quantization basics Uniform Quantization and 6dB/bit Rule Lossy predictive coding Differential Pulse Coded Modulation (DPCM) Lossy transform coding Image Transforms (Discrete Cosine Transform) Joint Photographic Expert Group (JPEG) EE465: Introduction to Digital Image Processing

2
**EE465: Introduction to Digital Image Processing**

Why Lossy? In most applications related to consumer electronics, lossless compression is not necessary What we care is the subjective quality of the decoded image, not those intensity values With the relaxation, it is possible to achieve a higher compression ratio (CR) For photographic images, CR is usually below 2 for lossless, but can reach over 10 for lossy EE465: Introduction to Digital Image Processing

3
**EE465: Introduction to Digital Image Processing**

A Simple Experiment Bit-plane representation A=a0+a12+a222+ … … +a727 Least Significant Bit (LSB) Most Significant Bit (MSB) Example A=129 a0a1a2 …a7= a0a1a2 …a7= A= =204 EE465: Introduction to Digital Image Processing

4
**EE465: Introduction to Digital Image Processing**

A Simple Experiment (Con’t) How will the reduction of gray-level resolution affect the image quality? Test 1: make all pixels even numbers (i.e., knock down a0 to be zero) Test 2: make all pixels multiples of 4 (i.e., knock down a0,a1 to be zeros) Test 3: make all pixels multiples of 4 (i.e., knock down a0,a1,a2 to be zeros) EE465: Introduction to Digital Image Processing

5
**EE465: Introduction to Digital Image Processing**

Experiment Results original Test 1 Test 3 Test 2 EE465: Introduction to Digital Image Processing

6
**How to Measure Image Quality?**

Subjective Evaluated by human observers Do not require the original copy as a reference Reliable, accurate yet impractical Objective Easy to operate (automatic) Often requires the original copy as the reference (measures fidelity rather than quality) Works better if taking HVS model into account EE465: Introduction to Digital Image Processing

7
**Objective Quality Measures**

Mean Square Error (MSE) Peak Signal-to-Noise-Ratio (PSNR) original decoded Question: Can you think of a counter-example to prove objective measure is not consistent with subjective evaluation? EE465: Introduction to Digital Image Processing

8
**EE465: Introduction to Digital Image Processing**

Answer Shifted (MSE=337.8) Original cameraman image By shifting the last row of the image to become the first row, we affect little on subjective quality but the measured MSE is large EE465: Introduction to Digital Image Processing

9
**EE465: Introduction to Digital Image Processing**

Lossy Image Compression From lossless to lossy Subjective and Objective Image Quality Quantization basics Uniform Quantization and 6dB/bit Rule Lossy predictive coding Differential Pulse Coded Modulation (DPCM) Lossy transform coding Image Transforms (Discrete Cosine Transform) Joint Photographic Expert Group (JPEG) EE465: Introduction to Digital Image Processing

10
**EE465: Introduction to Digital Image Processing**

What is Quantization? In Physics To limit the possible values of a magnitude or quantity to a discrete set of values by quantum mechanical rules In image compression To limit the possible values of a pixel value or a transform coefficient to a discrete set of values by information theoretic rules EE465: Introduction to Digital Image Processing

11
**EE465: Introduction to Digital Image Processing**

Examples Unlike entropy, we encounter it everyday (so it is not a monster) Continuous to discrete a quarter of milk, two gallons of gas, normal temperature is 98.6F, my height is 5 foot 9 inches Discrete to discrete Round your tax return to integers The mileage of my car is about 55K. EE465: Introduction to Digital Image Processing

12
**EE465: Introduction to Digital Image Processing**

A Game Played with Bits Precision is finite: the more precise, the more bits you need (to resolve the uncertainty) Keep a card in secret and ask your partner to guess. He/she can only ask Yes/No questions: is it bigger than 7? Is it less than 4? ... However, not every bit has the same impact How much did you pay for your car? (two thousands vs. $ ) EE465: Introduction to Digital Image Processing

13
**Scalar vs. Vector Quantization**

We only consider the scalar quantization (SQ) in this course Even for a sequence of values, we will process (quantize) each sample independently Vector quantization (VQ) is the extension of SQ into high-dimensional space Widely used in speech compression, but not for image compression EE465: Introduction to Digital Image Processing

14
**Definition of (Scalar) Quantization**

original value quantization index quantized value f f -1 x ^ x sS Quantizer Q x ^ x f finds the closest (in terms of Euclidean distance) approximation of x from a codebook C (a collection of codewords) and assign its index to s; f-1 operates like a table look-up to return the corresponding codeword EE465: Introduction to Digital Image Processing

15
**EE465: Introduction to Digital Image Processing**

Numerical Example (I) xR C={-1,0,1} S={1,2,3} 1 2 3 -1 1 red: codewords green: index in C Three codewords x ^ 1 -0.5 x 0.5 -1 EE465: Introduction to Digital Image Processing

16
**EE465: Introduction to Digital Image Processing**

Numerical Example (II) x{0,1,…,255} C={8,24,40,56,72,…,248} S={1,2,3,…,16} 1 2 3 ... 16 8 248 24 40 ... red: codewords green: index in C sixteen codewords x ^ (note that the scales are different) 40 24 8 x floor operation 16 32 EE465: Introduction to Digital Image Processing

17
**EE465: Introduction to Digital Image Processing**

Numerical Example (III) Q x x ^ Notes For scalar quantization, each sample is quantized independently Quantization is irreversible EE465: Introduction to Digital Image Processing

18
**Lossy Data Compression**

discrete source X S entropy coding (e.g., Huffman) binary bit stream f super-channel S entropy decoding ^ X f-1 EE465: Introduction to Digital Image Processing

19
**Quantization Noise (Distortion)**

x ^ x Quantization noise: For a continuous random variable, distortion is defined by probability distribution function For a discrete random variable, distortion is defined by probabilities EE465: Introduction to Digital Image Processing

20
**Revisit the Game with Bits**

Recall the goal of communication is to transmit bits for the purpose of resolving uncertainty Not all bits are equal in the sense of resolving uncertainty (e.g., MSB vs. LSB) The game rule now changes to Given a bit budget, how to most effectively resolve the uncertainty (reduce the distortion)? Given a distortion target, how to use the fewest bits to achieve it? EE465: Introduction to Digital Image Processing

21
**Shannon’s Rate-Distortion Theory**

Q1 Q2 Rate (bps) Question: which quantization scheme wins, 1 or 2? EE465: Introduction to Digital Image Processing

22
**The Game of Quantizer Design**

Assume P(X) or Pi is given, design the mapping scheme such that it achieves the smallest distortion for a given rate or vice versa Usually this game is complicated; however, we do know how to play this game for certain class of source (e.g., uniform distribution) EE465: Introduction to Digital Image Processing

23
**Uniform Quantization for Uniform Distribution**

A scalar quantization is called uniform quantization (UQ) if all its codewords are uniformly distributed (equally-distanced) Example (quantization stepsize =16) 8 40 ... 248 24 Uniform Distribution denoted by U[-A,A] f(x) 1/2A x -A A EE465: Introduction to Digital Image Processing

24
**Quantization Noise of UQ**

A -A f(e) 1/ e - /2 /2 Quantization noise of UQ on uniform distribution is also uniformly distributed Recall Variance of U[- /2, /2] is EE465: Introduction to Digital Image Processing

25
**EE465: Introduction to Digital Image Processing**

6dB/bit Rule Signal: X ~ U[-A,A] Noise: e ~ U[- /2, /2] Choose N=2n (n-bit) codewords for X (quantization stepsize) N=2n EE465: Introduction to Digital Image Processing

26
**EE465: Introduction to Digital Image Processing**

How Much is 6dB? 6dB increases in SNR implies the reduction of MSE by 75% To cut MSE by half, we approximately need 0.5bps EE465: Introduction to Digital Image Processing

27
**EE465: Introduction to Digital Image Processing**

Lossy Image Compression From lossless to lossy Subjective and Objective Image Quality Quantization basics Uniform Quantization and 6dB/bit Rule Lossy predictive coding Differential Pulse Coded Modulation (DPCM) Lossy transform coding Image Transforms (Discrete Cosine Transform) Joint Photographic Expert Group (JPEG) EE465: Introduction to Digital Image Processing

28
**Predictive Coding Revisited**

- Encoding x1 x2 … … xN y1 y2 … … yN y1=x1 yn=xn-xn-1, n=2,…,N - Decoding y1 y2 … … yN x1 x2 … … xN x1=y1 xn=yn+xn-1, n=2,…,N xn yn yn xn _ + xn-1 xn-1 xn D + D xn-1 Decoder DPCM Loop Encoder EE465: Introduction to Digital Image Processing

29
**EE465: Introduction to Digital Image Processing**

Open-loop DPCM ^ ^ xn yn yn xn _ Q + ^ xn-1 xn-1 xn D + D xn-1 Decoder Encoder Notes: Prediction is based on the past unquantized sample Quantization is located outside the DPCM loop EE465: Introduction to Digital Image Processing

30
**EE465: Introduction to Digital Image Processing**

Catastrophic Error Propagation with Open-loop DPCM -1 error caused by quantization original samples 90 92 91 93 93 95 … a b prediction residues a-b 90 2 2 2 a b decoded samples 90 92 92 94 94 96 … a+b EE465: Introduction to Digital Image Processing

31
**EE465: Introduction to Digital Image Processing**

Closed-loop DPCM ^ ^ ^ xn yn yn yn xn _ Q + xn-1 ^ ^ ^ xn-1 xn D + D ^ xn-1 Decoder Encoder Xn,yn: unquantized samples and prediction residues ^ ^ Xn,yn: decoded samples and quantized prediction residues Notes: Prediction is based on the past decoded sample Quantization is located inside the DPCM loop EE465: Introduction to Digital Image Processing

32
**EE465: Introduction to Digital Image Processing**

Numerical Example xn 90 92 91 93 93 95 … a yn 90 2 -2 3 2 a-b Q b 90 3 -3 3 3 yn ^ a b a+b xn ^ 90 93 90 93 93 96 … EE465: Introduction to Digital Image Processing

33
**Closed-loop DPCM Analysis**

^ xn yn yn _ Q A ^ ^ B xn-1 xn D + ^ xn-1 A: B: The distortion introduced to prediction residue yn is identical to that introduced to the original sample xn EE465: Introduction to Digital Image Processing

34
**EE465: Introduction to Digital Image Processing**

DPCM Summary Open-loop DPCM Prediction is based on original unquantized samples Catastrophic error propagation problems (decoder does not have access to the original but quantized samples to do prediction) Closed-loop DPCM Both encoder and decoder employ decoded samples to do prediction Quantization noise affects the accuracy of prediction DPCM is only suitable for lossy image coding at high bit rate (small quantization noise) EE465: Introduction to Digital Image Processing

35
**EE465: Introduction to Digital Image Processing**

Lossy Image Compression From lossless to lossy Subjective and Objective Image Quality Quantization basics Uniform Quantization and 6dB/bit Rule Lossy predictive coding Differential Pulse Coded Modulation (DPCM) Lossy transform coding Image Transforms (Discrete Cosine Transform) Joint Photographic Expert Group (JPEG) EE465: Introduction to Digital Image Processing

36
**An Example of 1D Transform with Two Variables**

y2 y1 (1,1) (1.414,0) x1 Transform matrix EE465: Introduction to Digital Image Processing

37
**Generalization into N Variables**

forward transform basis vectors (column vectors of transform matrix) EE465: Introduction to Digital Image Processing

38
**Decorrelating Property of Transform**

x2 y2 y1 x1 x1 and x2 are highly correlated y1 and y2 are less correlated p(x1x2) p(x1)p(x2) p(y1y2) p(y1)p(y2) EE465: Introduction to Digital Image Processing

39
**Transform=Change of Coordinates**

Intuitively speaking, transform plays the role of facilitating the source modeling Due to the decorrelating property of transform, it is easier to model transform coefficients (Y) instead of pixel values (X) An appropriate choice of transform (transform matrix A) depends on the source statistics P(X) We will only consider the class of transforms corresponding to unitary matrices EE465: Introduction to Digital Image Processing

40
**Unitary Matrix and 1D Unitary Transform**

Definition conjugate transpose A matrix A is called unitary if A-1=A*T When the transform matrix A is unitary, the defined transform is called unitary transform Example For a real matrix A, it is unitary if A-1=AT EE465: Introduction to Digital Image Processing

41
**Inverse of Unitary Transform**

For a unitary transform, its inverse is defined by Inverse Transform basis vectors corresponding to inverse transform EE465: Introduction to Digital Image Processing

42
**Properties of Unitary Transform**

Energy compaction: only few transform coefficients have large magnitude Such property is related to the decorrelating role of unitary transform Energy conservation: unitary transform preserves the 2-norm of input vectors Such property essentially comes from the fact that rotating coordinates does not affect Euclidean distance EE465: Introduction to Digital Image Processing

43
**Energy Compaction Example**

Hadamard matrix significant insignificant EE465: Introduction to Digital Image Processing

44
**EE465: Introduction to Digital Image Processing**

Energy Conservation A is unitary Proof EE465: Introduction to Digital Image Processing

45
**EE465: Introduction to Digital Image Processing**

Numerical Example Check: EE465: Introduction to Digital Image Processing

46
**Implication of Energy Conservation**

Q T T-1 A is unitary EE465: Introduction to Digital Image Processing

47
**Summary of 1D Unitary Transform**

Unitary matrix: A-1=A*T Unitary transform: A unitary Properties of 1D unitary transform Energy compaction: most of transform coefficients yi are small Energy conservation: quantization can be directly performed to transform coefficients EE465: Introduction to Digital Image Processing

48
**EE465: Introduction to Digital Image Processing**

From 1D to 2D Do N 1D transforms in parallel EE465: Introduction to Digital Image Processing

49
**EE465: Introduction to Digital Image Processing**

Definition of 2D Transform 2D forward transform 1D column transform 1D row transform EE465: Introduction to Digital Image Processing

50
**2D Transform=Two Sequential 1D Transforms**

(left matrix multiplication first) column transform row transform row transform (right matrix multiplication first) column transform Conclusion: 2D separable transform can be decomposed into two sequential The ordering of 1D transforms does not matter EE465: Introduction to Digital Image Processing

51
**From Basis Vectors to Basis Images**

1D transform matrix A consists of basis vectors (column vectors) 2D transform corresponds to a collection of N-by-N basis images basis image EE465: Introduction to Digital Image Processing

52
**Example of Basis Images**

Hadamard matrix: j i Bij EE465: Introduction to Digital Image Processing

53
**EE465: Introduction to Digital Image Processing**

2D Unitary Transform Suppose A is a unitary matrix, forward transform inverse transform Proof Since A is a unitary matrix, we have EE465: Introduction to Digital Image Processing

54
**Energy Compaction Property of 2D Unitary Transform**

Example A coefficient is called significant if its magnitude is above a pre-selected threshold th insignificant coefficients (th=64) EE465: Introduction to Digital Image Processing

55
**Energy Conservation Property of 2D Unitary Transform**

2-norm of a matrix X A unitary Example: You are asked to prove such property in your homework EE465: Introduction to Digital Image Processing

56
**EE465: Introduction to Digital Image Processing**

Implication of Energy Conservation Q T T-1 Similar to 1D case, quantization noise in the transform domain has the same energy as that in the spatial domain EE465: Introduction to Digital Image Processing

57
**Important 2D Unitary Transforms**

Discrete Fourier Transform Widely used in non-coding applications (frequency-domain approaches) Discrete Cosine Transform Used in JPEG standard Hadamard Transform All entries are 1 N=2: Haar Transform (simplest wavelet transform for multi-resolution analysis) EE465: Introduction to Digital Image Processing

58
**Discrete Cosine Transform**

EE465: Introduction to Digital Image Processing

59
**EE465: Introduction to Digital Image Processing**

1D DCT forward transform inverse transform Properties Real and orthogonal excellent energy compaction property DCT is NOT the real part of DFT Fact The real and imaginary parts of DFT are generally not orthogonal matrices fast implementation available: O(Nlog2N) EE465: Introduction to Digital Image Processing

60
**2D DCT Notice the excellent energy compaction property of DCT**

Its DCT coefficients Y (2451 significant coefficients, th=64) Original cameraman image X Notice the excellent energy compaction property of DCT EE465: Introduction to Digital Image Processing

61
**EE465: Introduction to Digital Image Processing**

Summary of 2D Unitary Transform Unitary matrix: A-1=A*T Unitary transform: A unitary Properties of 2D unitary transform Energy compaction: most of transform coefficients yi are small Energy conservation: quantization can be directly performed to transform coefficients EE465: Introduction to Digital Image Processing

62
**A Tour of JPEG Coding Algorithm**

Q C Flow-chart diagram of DCT-based coding algorithm specified by Joint Photographic Expert Group (JPEG) EE465: Introduction to Digital Image Processing

63
**Transform Coding of Images**

Why not transform the whole image together? Require a large memory to store transform matrix It is not a good idea for compression due to spatially varying statistics within an image Idea of partitioning an image into blocks Each block is viewed as a smaller-image and processed independently It is not a magic, but a compromise EE465: Introduction to Digital Image Processing

64
**EE465: Introduction to Digital Image Processing**

8-by-8 DCT Basis Images EE465: Introduction to Digital Image Processing

65
**Block Processing under MATLAB**

Type “help blkproc” to learn the usage of this function B = BLKPROC(A,[M N],FUN) processes the image A by applying the function FUN to each distinct M-by-N block of A, padding A with zeros if necessary. Example I = imread('cameraman.tif'); fun J = blkproc(I,[8 8],fun); EE465: Introduction to Digital Image Processing

66
**Block-based DCT Example**

I J note that white lines are artificially added to the border of each 8-by-8 block to denote that each block is processed independently EE465: Introduction to Digital Image Processing

67
**EE465: Introduction to Digital Image Processing**

Boundary Padding Example 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 padded regions When the width/height of an image is not the multiple of 8, the boundary is artificially padded with repeated columns/rows to make them multiple of 8 EE465: Introduction to Digital Image Processing

68
**EE465: Introduction to Digital Image Processing**

Work with a Toy Example Any 8-by-8 block in an image is processed in a similar fashion EE465: Introduction to Digital Image Processing

69
**Encoding Stage I: Transform**

• Step 1: DC level shifting 128 (DC level) _ EE465: Introduction to Digital Image Processing

70
**Encoding Step 1: Transform (Con’t)**

• Step 2: 8-by-8 DCT 88 DCT EE465: Introduction to Digital Image Processing

71
**Encoding Stage II: Quantization**

Q-table : specifies quantization stepsize (see slide #28) Notes: Q-table can be specified by customer Q-table is scaled up/down by a chosen quality factor Quantization stepsize Qij is dependent on the coordinates (i,j) within the 8-by-8 block Quantization stepsize Qij increases from top-left to bottom-right EE465: Introduction to Digital Image Processing

72
**Encoding Stage II: Quantization (Con’t)**

Example xij sij f EE465: Introduction to Digital Image Processing

73
**Encoding Stage III: Entropy Coding**

zigzag scan (20,5,-3,-1,-2,-3,1,1,-1,-1, 0,0,1,2,3,-2,1,1,0,0,0,0,0, 0,1,1,0,1,EOB) End Of the Block: All following coefficients are zero Zigzag Scan EE465: Introduction to Digital Image Processing

74
**EE465: Introduction to Digital Image Processing**

Run-length Coding (20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) DC coefficient AC coefficient - DC coefficient : DPCM coding encoded bit stream - AC coefficient : run-length coding (run, level) (5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) (0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1), (0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB Huffman coding encoded bit stream EE465: Introduction to Digital Image Processing

75
**EE465: Introduction to Digital Image Processing**

JPEG Decoding Stage I: Entropy Decoding encoded bit stream Huffman decoding (0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1), (0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB encoded bit stream DPCM decoding (20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) DC coefficient AC coefficients EE465: Introduction to Digital Image Processing

76
**JPEG Decoding Stage II: Inverse Quantization**

(20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) zigzag f-1 EE465: Introduction to Digital Image Processing

77
**JPEG Decoding Stage III: Inverse Transform**

88 IDCT 128 (DC level) + EE465: Introduction to Digital Image Processing

78
**EE465: Introduction to Digital Image Processing**

Quantization Noise ^ X X ^ Distortion calculation: MSE=||X-X||2 Rate calculation: Rate=length of encoded bit stream/number of pixels (bps) EE465: Introduction to Digital Image Processing

79
**EE465: Introduction to Digital Image Processing**

JPEG Examples 10 (8k bytes) 50 (21k bytes) 90 (58k bytes) worst quality, highest compression best quality, lowest compression 100 EE465: Introduction to Digital Image Processing

80
**JPEG Coding Algorithm Summary**

EE465: Introduction to Digital Image Processing

81
**Lossy Compression of Color Images**

Color-space transformation luminance chrominance forward transform inverse transform Luminance channel can be treated just like a gray-scale image EE465: Introduction to Digital Image Processing

82
**Compression of Chrominance Channels**

Down-sampling Q-Table 2 Cr/Cb Human eye is relatively insensitive to the high-frequency content of the chrominance channels. EE465: Introduction to Digital Image Processing

83
**EE465: Introduction to Digital Image Processing**

Image Example raw image data JPEG compressed data 768×512 ×24=1.18M bytes 26K bytes Compression ratio=44.6 EE465: Introduction to Digital Image Processing

84
**A Glimpse into Wavelet Transform***

The idea of multi-resolution analysis (MRA) Low-resolution (LR) High-resolution (HR) EE465: Introduction to Digital Image Processing

85
**EE465: Introduction to Digital Image Processing**

MRA by Haar Transform LP HP Example EE465: Introduction to Digital Image Processing

86
**Reordering Coefficients into Subbands**

approximation horizontal details Apply Haar transform to each 2-by-2 block vertical details diagonal details é 200 200 ù ê ú 100 100 regrouping Y = ê ú ê ú ê ú ë û EE465: Introduction to Digital Image Processing

87
**EE465: Introduction to Digital Image Processing**

Multi-level Wavelet Transform Apply one-level Haar transform ú û ù ê ë é = 100 200 Y Apply Haar transform to the LL band only EE465: Introduction to Digital Image Processing

88
**EE465: Introduction to Digital Image Processing**

Image Example EE465: Introduction to Digital Image Processing

89
**EE465: Introduction to Digital Image Processing**

Wavelet vs. DCT discrete cosine transform based wavelet transform based JPEG (CR=64) JPEG2000 (CR=64) EE465: Introduction to Digital Image Processing

90
**Lossy Image Compression Summary**

Quantization introduces irreversible information loss Lossy predictive coding: open-loop DPCM vs. closed-loop DPCM Lossy transform coding: energy compaction an preservation properties of unitary transforms Objective measure for image distortion MSE or PSNR are widely used for their convenience Does not necessarily match subjective quality EE465: Introduction to Digital Image Processing

Similar presentations

Presentation is loading. Please wait....

OK

Multimedia Data Compression

Multimedia Data Compression

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google