1. Wavelet – An Introduction
2018/11/17
Wavelet – An Introduction
Multi-resolution decomposition
Wavelet basis
Dilation equations
Wavelet-based image coding
Embedded zero-tree
Wavelet-based video coding
Motion compensated temporal filtering
2018/11/17Wavelet, MC 2009
MC 2007

2. Wavelet Transformation
2018/11/17
Wavelet Transformation
The wavelet transform describes a multi-resolution decomposition process in terms of expansion of an image onto a set of wavelet basis  functions.
Wavelet transform vs. Short-term Fourier transform
Wavelet basis function: well-localized in both space and frequency
Dilation equations  :
Two multi-resolution decomposition schemes: pyramid  and subband.
Wavelet compression process
Take wavelet transform of a digitized image.
Allocate bits among the wavelet coefficients.
2018/11/17Wavelet, MC 2009
MC 2007

3. Wavelet-Based Image Coding
2018/11/17
Wavelet-Based Image Coding
Embedded Zero-tree Wavelet coding (EZW)
by J. M. Shapiro (1993)
Set Partitioning in Hierarchical Trees (SPIHT)
by A. Said and W. A. Pearlman (1996)
Layered Zero Coding (LZC)
by D. Taubman and A. Zakhor
Rate-Distortion optimized Embedded coding (RDE)
by J. Li and S.-M. Lei
Embedded Block Coding with Optimized Truncation (EBCOT)
by D. Taubman
An important part of JPEG2000 Part 1
2018/11/17Wavelet, MC 2009
MC 2007

4. 1-D Pyramidal Decomposition
2018/11/17
1-D Pyramidal Decomposition
2018/11/17Wavelet, MC 2009
MC 2007

5. Multi-resolution Decomposition
2018/11/17
Multi-resolution Decomposition
A multiresolution analysis of the space L2 (R) consists of a sequence of nested subspaces … V0  V1  …  Vn  … that satisfies certain self-similarity relations in spatial and frequency, as well as completeness and regularity relations.
Self-similarity in spatial demands that each subspace Vk is invariant under shifts by integer multiples of 2-k. That is, for each f  Vk, m   there is a g  Vk with
f (x) = g(x+m2-k).
Self-similarity in frequency demands that all subspaces Vk  Vl , k < l, are spatial-scaled versions of each other, with scaling respectively dilation factor 2l-k. That is, for each f  Vk there is a g  Vl with f (x) = g(x2l-k).
LH1
HL1
HH1
HL2
LL1
LL2 
2018/11/17Wavelet, MC 2009
MC 2007

6. Multi-resolution Analysis and the Scaling Functions
2018/11/17
Multi-resolution Analysis and the Scaling Functions
Approximated by scaling functions
Harr (1907) scaling functions
Dilation version of the “mother” scaling function: In fact, we can obtain scaling functions at different resolutions in a manner similar to the procedure used for wavelets,
In Harr
2018/11/17Wavelet, MC 2009
MC 2007

7. Wavelet Approximations by Harr (1)
2018/11/17
Wavelet Approximations by Harr (1)
2018/11/17Wavelet, MC 2009
MC 2007

8. Wavelet Approximations (2)
2018/11/17
Wavelet Approximations (2)
2018/11/17Wavelet, MC 2009
MC 2007

9. Short-term Fourier Transform (STFT)
2018/11/17
Short-term Fourier Transform (STFT)
If we have a very non-stationary signal, we would like to know not only the frequency components but when in time the particular frequency components occurred.
Short-term Fourier Transform
We break the time signal f(t) into pieces of length T and apply Fourier analysis to each pieces. Thus, we obtain an analysis that is a function of both time and frequency.
Boundary effect, (Problem #1)
To reduce the boundary effect, we window each piece before we take the FT. If the window shape is given by g(t), the STFT is given by
If the window function g(t) is Gaussian, the STFT is called the Gabor transform.
The problem with the STFT is the fixed window size. The window size should contain at least one cycle of the low-frequency component, however, it can not accurately localize the high frequency spurt.
Uncertainty principle: If we wish to have finer resolution in time, we end up with a lower resolution in the frequency domain.
2018/11/17Wavelet, MC 2009
MC 2007

10. 2018/11/17
2018/11/17Wavelet, MC 2009
MC 2007

11. STFT basis functions WT basis functions
2018/11/17
STFT basis functions
WT basis functions
Small frequency changes in the FT will produce changes everywhere in the time domain.
Wavelets are local in both frequency/scale (via dilations) and in spatial/time (via translations).
2018/11/17Wavelet, MC 2009
MC 2007

12. Wavelets A wavelet of the mother function (t),
2018/11/17
Wavelets
A wavelet of the mother function (t),
We scale a function f(t) by replacing t with t/a and translate a function to the right or left by an amount b by replacing t by t-b or t+b.
If we want the scaled function to have the norm as the original function, we need to multiply it by
Wavelet coefficients of a function f(t),
We can recover the function f(t) by
Ondelette in French, by Morlet
2018/11/17Wavelet, MC 2009
MC 2007

13. 2018/11/17
Properties of Wavelet
C should be finite, so (0) = 0. Since (0) is the average value of (t); therefore a requirement on the mother wavelet is that it has zero mean.
We would also like the wavelets to have finite energy. Using Parseval’s relationship, we can write this requirement as
For this to happen, | ()|2 have to decay as  goes to infinite. This requirements mean the energy in () is concentrated in a narrow frequency band.
Discrete version:
2018/11/17Wavelet, MC 2009
MC 2007

14. 2018/11/17
Dilation Equations
The scaling function itself can be represented by its dilations at a high resolution:
Multi-resolution analysis (MRA) equations:
by substituting
Harr scaling function:
Orthonormal transforms
2018/11/17Wavelet, MC 2009
MC 2007

15. (x) = ½  (2x) +  (2x-1) + ½  (2x-2)
2018/11/17
(x) = ½  (2x) +  (2x-1) + ½  (2x-2) 2 1
2018/11/17Wavelet, MC 2009
MC 2007

16. Daubechies Wavelet DAUB4
2018/11/17
Daubechies Wavelet DAUB4
4 coefficients
2018/11/17Wavelet, MC 2009
MC 2007

17. Daubechies Wavelet DAUBx (2)
2018/11/17
Daubechies Wavelet DAUBx (2)
Its derivative exists only almost everywhere (fails on points p/2n)
Left differentiable, but not right differential.
The 130th largest wavelet coefficient has an amplitude less than 10-5 of the largest coefficient among 1024 coefficients.
Compact wavelets are better for lower accuracy approximation and for functions with discontinuities (such as edges), while small wavelets are better for achieving high numerical accuracy.
BAUB6, p=3 moment vanishing
2018/11/17Wavelet, MC 2009
MC 2007

18. Daubechies 9-tap/7-tap Filter Cohen-Daubechies-Feauveau (CDF)
2018/11/17
Daubechies 9-tap/7-tap Filter Cohen-Daubechies-Feauveau (CDF)
Biorthogonal wavelet: is a wavelet where the associated wavelet transform is invertible but not necessarily orthogonal.
allows more degrees of freedoms than orthogonal wavelets.
decomposed into a set of low-pass samples and a set of high-pass samples.
Biorthogonal Daubechies 9-tap/7-tap filter (irreversible transform)
Le Gall 5-tap/3-tap filter (reversible transform)
it introduces quantization noise that depends
on the precision of the decoder
analysis filter
5+4
4+3
2018/11/17Wavelet, MC 2009
MC 2007

19. Subband Decomposition of an NM Images
2018/11/17
Subband Decomposition of an NM Images
2018/11/17Wavelet, MC 2009
MC 2007

20. Embedded Zerotree Wavelet (1)
2018/11/17
Embedded Zerotree Wavelet (1)
Embedded coding
The bits are ordered in their importance
The encoding improves as more bits are transmitted.
The codec can terminate at any point thereby allowing a target rate or distortion metric to be met exactly.
Assumption : most energy is compacted into lower bands
The coefficients closer to the root have higher magnitudes than coefficients further from the root.
LH1
HL1
HH1
HL2
LL1
LL2 
2018/11/17Wavelet, MC 2009
MC 2007

21. 2018/11/17
EZW (2)
Embedded Image Coding using Zerotrees of wavelet coefficients: DWT, prediction of the absence of significant information, entropy-coded successive-approximate quantization and lossless data compression.
Zerotree structure
If a coefficient has a magnitude less than a given threshold, all its descendants might have magnitudes less than that threshold.
Significant coefficient at level T
If |x| < T
Symbols
Zerotree root (ZR)
all descenfents (including itself) are insignificant
Isolated zero (IZ)
the coefficient is insignificant but has some
significant descendant
Significant positive (SP)
Significant negative (SN)
2018/11/17Wavelet, MC 2009
MC 2007

22. EZW (3) EZW is a multiple-pass algorithm
2018/11/17
EZW (3)
EZW is a multiple-pass algorithm
Each pass consists of 2 steps
Significance map encoding (dominant pass)
Refinement (subordinate pass)
Two separate lists are maintained
Dominant list
coordinates of the coefficients that have not been found to be significant in the relative order
Subordinate list
magnitudes of the coefficients that have been found to be significant
Successive Approximation
During a subordinate pass
the width of the effective quantizer step size (uncertainty interval) is cut in half
1 : the value falls in the upper half of the uncertainty interval
0 : the value falls in the lower half
The threshold is halved before each dominant pass
Scanning order
2018/11/17Wavelet, MC 2009
MC 2007

23. Flow Chart for Encoding a Coefficient
2018/11/17
Flow Chart for Encoding a Coefficient
2018/11/17Wavelet, MC 2009
MC 2007

24. EZW (4) Example 1 Initial threshold cmax will be in [T0, 2T0)
2018/11/17
EZW (4)
Example 1
Initial threshold
cmax will be in [T0, 2T0)
Dominant list
Subordinate list
Subband Coeff. Symbol
LL SP
HL ZR
LH ZR
HH ZR
“0” : [16, 24)
“1” : [24, 32)
Coef Symbol Recon
2018/11/17Wavelet, MC 2009
MC 2007

25. EZW (5) Example 1 (cont.) Pass 1 Pass 2 T1 = T0/2 = 8 T2 = T1/2 = 4
2018/11/17
EZW (5)
Example 1 (cont.)
Pass 1
T1 = T0/2 = 8
Pass 2
T2 = T1/2 = 4
Dominant list
Subordinate list
Subband Coeff. Symbol
HL IZ
LH ZR
HH ZR
HL SP
HL SP
HL IZ
HL IZ
[8, 16) [16, 24) [24, 32)
“0” : [8, 12) [16,20) [24,28)
“1” : [12, 16) [20,24) [28,32) X
Coef Symbol Recon
26
Subband Coeff. Symbol
HL SP
LH SN
HH SP
HL SP
HL SP
LH SP
LH SN
LH IZ
LH IZ
HH SP
HH IZ
HH IZ
HH IZ
Coef Symbol Recon
25
13 9
X X X
2018/11/17Wavelet, MC 2009
MC 2007

26. EZW (6) Example 2 63 -34 49 10 7 13 -12 7 Initial threshold 63 1 56
2018/11/17
EZW (6)
Example 2
Initial threshold
Dominant list
Subband Coeff. Symbol
LL SP
HL SN
LH IZ
HH ZR
HL SP
HL ZR
HL ZR
HL ZR
LH ZR
LH IZ
LH ZR
LH ZR
HL ZR
HL ZR
HL ZR
HL ZR
LH ZR
LH SP
LH ZR
LH ZR
Subordinate list
“0” : [32, 48)
“1” : [48, 64)
Coef Symbol Recon
2018/11/17Wavelet, MC 2009
MC 2007

27. EZW (7) Example 2 (2nd pass): 63 -34 49 10 7 13 -12 7
2018/11/17
EZW (7)
Example 2 (2nd pass):
Threshold T1 = 16
Dominant list
Subordinate list
[16, 32) [32, 48) [48,64)
“0”: [16, 24) [32,40) [48,56)
“1”: [24, 32) [40,48) [56,64)
Subband Coeff. Symbol
LH SN
HH SP
HL ZR
HL ZR
HL ZR
LH ZR
LH ZR
LH ZR
LH ZR
HH ZR
HH ZR
HH ZR
HH ZR
Coef Symbol Recon
60
36
52
44
2018/11/17Wavelet, MC 2009
MC 2007

28. SPIHT (1) Set Partitioning in Hierarchical Trees Partitioning types
2018/11/17
SPIHT (1)
Set Partitioning in Hierarchical Trees
Use a partitioning of the spatial orientation trees in a manner that tends to keep insignificant coefficients together in a larger subsets
Partitioning decision -> binary decision
Provide a significance map encoding that is more efficient than EZW
Each pass consists of 2 steps
Significance map encoding (set partitioning and ordering step)
Refinement
Partitioning types
O(i, j)
Set of coordinates of the offsprings at (i, j)
D(i, j)
Set of all descendants of the coefficients at (i, j) H
Set of all root nodes
L(i,j)
L(i,j) = D(i, j) - O(i, j)
LIS
Each node can either have 4 or none
2018/11/17Wavelet, MC 2009
MC 2007

29. SPIHT (2) Significant Three lists are maintained At each pass
2018/11/17
SPIHT (2)
Significant
A set D(i, j) or L(i,j) is significant if any coefficient in the set has a magnitude greater than the threshold
Three lists are maintained
LIP (list of insignificant pixels)
Is Initialized with the set H
LSP (list of significant pixels)
Is initially empty
LIS (list of insignificant sets)
Contain the coordinates of the roots of types D or L
At each pass
Significant map encoding step
Process the members of LIP
If the coefficient is significant  transmit (1 + sign), move to LSP;
Otherwise  transmit 0
Then process the members of LIS
Insignificant set  0
Refinement step
Process the elements of LSP
2018/11/17Wavelet, MC 2009
MC 2007

30. SPIHT (3) An example Initialize 1st pass (T = 16) T = 16
2018/11/17
SPIHT (3)
An example
Initialize
T = 16
1st pass (T = 16)
LIP {(0,0)  26, (0,1)  6, (1,0)  -7, (1,1)  7}
LIS {(0,1)D, (1,0)D, (1,1)D}
LSP { }
LIP
26   move to LSP
6  0
-7  0
7  0
LIS
(0,1)D  0
(1,0)D  0
(1,1)D  0
1 + sign
2018/11/17Wavelet, MC 2009
MC 2007

31. SPIHT(4) An example (cont.) 2nd pass (T = 8) 26 6 13 10
2018/11/17
SPIHT(4)
An example (cont.)
2nd pass (T = 8)
n--  n = 3, T = 8
LIP { (0,1)  6, (1,0)  -7, (1,1)  7}
LIS {(0,1)D, (1,0)D, (1,1)D}
LSP { (0,0)  26 }
LIP
6  0
-7  0
7  0
LIS
(0,1)D  1
13   move to LSP
10   move to LSP
6  0  move to LIP
4  0  move to LIP
(1,0)D  0
(1,1)D  0
LSP
26 = (11010)2  1
2018/11/17Wavelet, MC 2009
MC 2007

32. SPIHT (5) An example (cont.) 26 6 13 10 -7 7 6 4 4 -4 4 -3 2 -2 -2 0
2018/11/17
SPIHT (5)
An example (cont.)
3rd pass (T = 4)
n--  n = 2, T = 4
LIP {(0,1)  6, (1,0)  -7, (1,1)  7, (1,2)  6, (1,3)  4}
LIS {(1,0)D, (1,1)D}
LSP {(0,0)  26, (0,2)  13, (0,3)  10}
LIP
6   move to LSP
-7   move to LSP
7   move to LSP
4   move to LSP
LIS
(1,0)D  1
-4   move to LSP
2   move to LIP
-2   move to LIP
(1,1)D  1
-3   move to LIP
0   move to LIP
LSP
26  0, 13  1, 10  0
LIP {(3,0)2, (3,1)-2, (2,3)-3, (3,2)-2, (3,3)0}
LIS { }
LSP { (0,0)  26, (0,2)  13, (0,3)  10, (0,1)  6,
(1,0)  -7, (1,1)  7, (1,2)  6, (1,3)  4,
(2,0)  4, (2,1)  -4, (2,2)  4}
2018/11/17Wavelet, MC 2009
MC 2007

33. Interframe Wavelet Coding
2018/11/17
Interframe Wavelet Coding
Motion-Compensated 3-D Subband Coding of Video
IEEE TRANSACTIONS ON IMAGE PROCESSING
VOL. 8, NO. 2, FEB 1999
Seung-Jong Choi and John W. Woods
Adaptive Motion-Compensated Wavelet Filtering for Image Sequence Coding
VOL. 6, NO. 6, JUNE 1997
Jean-Pierre Leduc, Jean-Marc Odobez, and Claude Labit
Three-dimensional subband coding with motion compensation
VOL. 3, SEPT 1993
J.-R. Ohm
MC 2007

34. Outline Wavelet Transform-based Coding
2018/11/17
Outline
Wavelet Transform-based Coding
Motion Compensated Temporal Filtering (MCTF)
Block Diagram of Interframe Wavelet Coding
Experimental Results
Conclusions
2018/11/17Wavelet, MC 2009
MC 2007

35. Wavelet Transform-based Coding
2018/11/17
Wavelet Transform-based Coding
Discrete wavelet transform (DWT)
decomposes a non-stationary signal into multi-scaled subbands
Wavelet transform
Quantization (Embedded)
Entropy coding (Zero-tree)
2018/11/17Wavelet, MC 2009
MC 2007

36. Motion Compensated Temporal Filtering (MCTF)
2018/11/17
Motion Compensated Temporal Filtering (MCTF)
The spatio-temporal scenes will be segmented according to motion
build the trajectories by a spatio-temporal segmentation algorithm and a related trajectory construction
Temporal filtering will be applied along the assumed motion trajectories
2018/11/17Wavelet, MC 2009
MC 2007

37. Motion Compensated Temporal Filtering (MCTF)
2018/11/17
Motion Compensated Temporal Filtering (MCTF)
(Haar filter [1, 1] [1, -1])
(Haar filter [1, 1] [1, -1]) with motion compensation
2018/11/17Wavelet, MC 2009
MC 2007

38. Motion Compensated 3D Wavelet Coding by Lifting Filter Implementation
2018/11/17
Motion Compensated 3D Wavelet Coding by Lifting Filter Implementation
The “highpass” pictures are very much the same as difference frame in a frame prediction
can be achieved in combination with motion compensation
A “lifting filter” structure is an efficient implementation of wavelet decomposition
Consisting of a “prediction step” (highpass filter) and an “update step” (lowpass filter)
2018/11/17Wavelet, MC 2009
MC 2007

39. An MCTF Four-level Decomposition (GOP=16 frames)
2018/11/17
An MCTF Four-level Decomposition (GOP=16 frames)
Divides consecutive frames into groups (GOP=16)
Perform MC filtering on pairs of frames to produce temporal low and high frames.
Temporal low frames decomposed again with the same MC filtering
2018/11/17Wavelet, MC 2009
MC 2007

40. A Three-level 3D motion Compensated Decomposition
2018/11/17
A Three-level 3D motion Compensated Decomposition
Pictures at leaves of tree are subject to spatial wavelet decomposition
2018/11/17Wavelet, MC 2009
MC 2007

41. Block Diagram of Interframe Wavelet
2018/11/17
Block Diagram of Interframe Wavelet
Motion Compensated Temporal Filtering + Zero-Tree Coding
John Woods, Jens Ohm
2018/11/17Wavelet, MC 2009
MC 2007

42. Examples of comparison between MCTF and AVC (GOP=16 frames)
2018/11/17
Experimental Results
Examples of comparison between MCTF and AVC (GOP=16 frames)
2018/11/17Wavelet, MC 2009
MC 2007

43. Conclusions Scalability: spatial, temporal, SNR and complexity
2018/11/17
Conclusions
Scalability: spatial, temporal, SNR and complexity
(Wavelet coding which inherently possess scalability)
Providing flexible scalability of a single bitstream
(encoding once, decoding multiple times)
Error resilience
(a non-recursive coder)
2018/11/17Wavelet, MC 2009
MC 2007