1. Wavelets (Chapter 7)
CS474/674 – Prof. Bebis

2. STFT (revisited) Time/Frequency localization depends on window size.
Once you choose a particular window size, it will be the same for all frequencies.
Many signals require a more flexible approach - vary the window size to determine more accurately either time or frequency.

3. The Wavelet Transform
Overcomes the preset resolution problem of the STFT by using a variable length window:
Narrower windows are more appropriate at high frequencies (better time localization)
Wider windows are more appropriate at low frequencies (better frequency localization)

4. The Wavelet Transform (cont’d)
Wide windows do not provide good localization
at high frequencies.

5. The Wavelet Transform (cont’d)
A narrower window is more appropriate at high frequencies.

6. The Wavelet Transform (cont’d)
Narrow windows do not provide good localization
at low frequencies.

7. The Wavelet Transform (cont’d)
A wider window is more appropriate at low frequencies.

8. What is a wavelet?
It is a function that “waves” above and below the x-axis; it has (1) varying frequency, (2) limited duration, and (3) an average value of zero.
This is in contrast to sinusoids, used by FT, which have infinite energy.
Sinusoid Wavelet

9. Wavelets
Like sine and cosine functions in FT, wavelets can define basis functions ψk(t):
Span of ψk(t): vector space S containing all functions f(t) that can be represented by ψk(t).

10. Wavelets (cont’d) There are many different wavelets: Morlet Haar
Daubechies

11. Basis Construction - Mother Wavelet
(dyadic/octave grid)

12. Basis Construction - Mother Wavelet (cont’d)
scale =1/2j
(1/frequency) j k

13. Continuous Wavelet Transform (CWT)
scale parameter
(measure of frequency)
translation parameter, measure of time
Forward
CWT:
Mother wavelet (window)
normalization
constant
scale =1/2j=1/frequency

14. CWT: Main Steps
Take a wavelet and compare it to a section at the start of the original signal.
Calculate a number, C, that represents how closely correlated the wavelet is with this section of the signal. The higher C is, the more the similarity.

15. CWT: Main Steps (cont’d)
3. Shift the wavelet to the right and repeat steps 1 and 2 until you've covered the whole signal.

16. CWT: Main Steps (cont’d)
4. Scale the wavelet and repeat steps 1 through Repeat steps 1 through 4 for all scales.

17. Coefficients of CTW Transform
Wavelet analysis produces a time-scale view of the input signal or image.

18. Continuous Wavelet Transform (cont’d)
Inverse CWT:
note the double integral!

19. FT vs WT
weighted by F(u)
weighted by C(τ,s)

20. Properties of Wavelets
Simultaneous localization in time and scale
- The location of the wavelet allows to explicitly represent the location of events in time.
- The shape of the wavelet allows to represent different detail or resolution.

21. Properties of Wavelets (cont’d)
Sparsity: for functions typically found in practice, many of the coefficients in a wavelet representation are either zero or very small.
Adaptability: Can represent functions with discontinuities or corners more efficiently.
Linear-time complexity: many wavelet transformations can be accomplished in O(N) time.

22. Discrete Wavelet Transform (DWT)
(forward DWT)
(inverse DWT)
where

23. DFT vs DWT FT expansion: WT expansion one parameter basis or
two parameter basis

24. Multiresolution Representation using
fine
details
wider, large translations j
coarse
details

25. Multiresolution Representation using
fine
details j
coarse
details

26. Multiresolution Representation using
fine
details
narrower, small translations j
coarse
details

27. Multiresolution Representation using
high resolution
(more details) j …
low resolution
(less details)

28. Prediction Residual Pyramid - Revisited
In the absence of quantization errors, the approximation
pyramid can be reconstructed from the prediction residual
pyramid.
Prediction residual pyramid can be represented more
efficiently.
(with sub-sampling)

29. Efficient Representation Using “Details”
details D3
details D2
details D1 L0
(without sub-sampling)

30. Efficient Representation Using Details (cont’d)
representation: L0 D1 D2 D3
in general: L0 D1 D2 D3…DJ
A wavelet representation of a function consists of
a coarse overall approximation
detail coefficients that influence the function at various scales.

31. Reconstruction (synthesis)
H3=H2+D3
H2=H1+D2
details D3
details D2
H1=L0+D1
details D1 L0
(without sub-sampling)

32. Example - Haar Wavelets
Suppose we are given a 1D "image" with a resolution of 4 pixels:
[ ]
The Haar wavelet transform is the following:
L0 D1 D2 D3
(with sub-sampling)

33. Example - Haar Wavelets (cont’d)
Start by averaging the pixels together (pairwise) to get a new lower resolution image:
To recover the original four pixels from the two averaged pixels, store some detail coefficients.
[ ] 1

34. Example - Haar Wavelets (cont’d)
Repeating this process on the averages (i.e., low resolution image) gives the full decomposition: 1
Harr decomposition:

35. Example - Haar Wavelets (cont’d)
We can reconstruct the original image by adding or subtracting the detail coefficients from the lower-resolution versions.
1 -1 2

36. Example - Haar Wavelets (cont’d)
Detail coefficients
become smaller and
smaller as j increases. Dj
Dj-1
How to
compute Di ? D1 L0

37. Multiresolution Conditions
If a set of functions can be represented by a weighted sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j+1t - k):
high
resolution j
low
resolution

38. Multiresolution Conditions (cont’d)
If a set of functions can be represented by a weighted sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j+1t - k):
Vj: span of ψ(2jt - k):
Vj+1: span of ψ(2j+1t - k):

39. Nested Spaces Vj Basis functions: Vj : space spanned by ψ(2jt - k)
f(t) ϵ Vj
ψ(2t - k) V1 … Vj
ψ(2jt - k)
Multiresolution conditions  nested spanned spaces:
i.e., if f(t) ϵ V j then f(t) ϵ V j+1

40. How to compute Di ? (cont’d)
f(t) ϵ Vj
IDEA: define a set of basis
functions that span the
difference between Vj+1 and Vj

41. Orthogonal Complement Wj
Let Wj be the orthogonal complement of Vj in Vj+1
Vj+1 = Vj + Wj

42. How to compute Di ? (cont’d)
If f(t) ϵ Vj+1, then f(t) can be represented using basis functions φ(t) from Vj+1:
Vj+1
Alternatively, f(t) can be represented using two sets of basis functions,
φ(t) from Vj and ψ(t) from Wj:
Vj+1 = Vj + Wj

43. How to compute Di ? (cont’d)
Think of Wj as a means to represent the parts of a function in Vj+1 that cannot be represented in Vj
differences
between
Vj and Vj+1
Vj Wj

44. How to compute Di ? (cont’d)
 using recursion on Vj:
Vj+1 = Vj + Wj
Vj+1 = Vj-1+Wj-1+Wj = …= V0 + W0 + W1 + W2 + … + Wj
if f(t) ϵ Vj+1 , then:
V W0, W1, W2, …
basis functions basis functions

45. Summary: wavelet expansion (Section 7.2)
Wavelet decompositions involve a pair of waveforms (mother wavelets):
encode low
resolution info
encode details or
high resolution info
φ(t) ψ(t)
scaling function wavelet function

46. 1D Haar Wavelets φ(t) ψ(t) Haar scaling and wavelet functions:
computes average
computes details

47. 1D Haar Wavelets (cont’d)
V0 represents the space of one pixel images
Think of a one-pixel image as a function that is constant over [0,1)
width: 1
Example:

48. 1D Haar Wavelets (cont’d)
V1 represents the space of all two-pixel images
Think of a two-pixel image as a function having 21 equal-sized constant pieces over the interval [0, 1).
Note that
Examples: ½
width: 1/2
e.g., + =

49. 1D Haar Wavelets (cont’d)
V j represents all the 2j-pixel images
Functions having 2j equal-sized constant pieces over interval [0,1).
Note that
Examples:
width: 1/2j
ϵ Vj
e.g.,
ϵ Vj

50. 1D Haar Wavelets (cont’d)
V0, V1, ..., V j are nested
i.e., VJ … V1 V0
fine details
coarse details

51. Define a basis for Vj Mother scaling function:
Let’s define a basis for V j : 1
note alternative notation:

52. Define a basis for Vj (cont’d)

53. Define a basis for Wj (cont’d)
Mother wavelet function:
Let’s define a basis ψ ji for Wj : 1 -1 /
note alternative notation:

54. Define a basis for Wj Note that the dot product between basis
functions in Vj
and Wj is zero!

55. Define a basis for Wj (cont’d)
Basis functions ψ ji of W j
Basis functions φ ji of V j
form a basis in V j+1

56. Define a basis for Wj (cont’d)
V3 = V2 + W2

57. Define a basis for Wj (cont’d)
V2 = V1 + W1

58. Define a basis for Wj (cont’d)
V1 = V0 + W0

59. Example - Revisited
f(x)= V2

60. Example (cont’d)
f(x)=
φ2,0(x)
φ2,1(x)
φ2,2(x)
φ2,3(x) V2

61. Example (cont’d) φ1,0(x) φ1,1(x) ψ1,0(x) ψ1,1(x) V1and W1 V2=V1+W1
(divide by 2 for normalization)
V1and W1
φ1,0(x)
φ1,1(x)
ψ1,0(x)
ψ1,1(x)
V2=V1+W1

62. Example (cont’d)

63. Example (cont’d) φ0,0(x) ψ0,0(x) ψ1,0 (x) ψ1,1(x) V0 ,W0 and W1
(divide by 2 for normalization)
V0 ,W0 and W1
φ0,0(x)
ψ0,0(x)
ψ1,0 (x)
ψ1,1(x)
V2=V1+W1=V0+W0+W1

64. Example

65. Example (cont’d)

66. Filter banks
The lower resolution coefficients can be calculated from the higher resolution coefficients by a tree-structured algorithm (i.e., filter bank).
Subband
encoding
(analysis)
h0(-n) is a lowpass filter and h1(-n) is a highpass filter

67. Example (revisited) [9 7 3 5] (9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2
low-pass,
down-sampling
high-pass,
down-sampling
(9+7)/2 (3+5)/ (9-7)/2 (3-5)/2
V1 basis functions

68. Filter banks (cont’d)
Next level:

69. Example (revisited) [9 7 3 5] (8+4)/2 (8-4)/2 V1 basis functions
low-pass,
down-sampling
high-pass,
down-sampling
V1 basis functions
(8+4)/ (8-4)/2

70. Convention for illustrating 1D Haar wavelet decomposition
x x x x x x … x x
average
detail …
re-arrange:
V1 basis functions
re-arrange:

71. Examples of lowpass/highpass (analysis) filters
Haar h1 h0
Daubechies h1

72. Filter banks (cont’d)
The higher resolution coefficients can be calculated from
the lower resolution coefficients using a similar structure.
Subband
encoding
(synthesis)

73. Filter banks (cont’d)
Next level:

74. Examples of lowpass/highpass (synthesis) filters
Haar (same as
for analysis): g1 g0
Daubechies g1 +

75. 2D Haar Wavelet Transform
The 2D Haar wavelet decomposition can be computed using 1D Haar wavelet decompositions.
i.e., 2D Haar wavelet basis is separable
Two decompositions (i.e., correspond to different basis functions):
Standard decomposition
Non-standard decomposition

76. Standard Haar wavelet decomposition
Steps
(1) Compute 1D Haar wavelet decomposition of each row of the original pixel values.
(2) Compute 1D Haar wavelet decomposition of each column of the row-transformed pixels.

77. Standard Haar wavelet decomposition (cont’d)
average
(1) row-wise Haar decomposition:
detail
re-arrange terms … …
x x x … x
… …
x x x x …
… …
… … …

78. Standard Haar wavelet decomposition (cont’d)
average
(1) row-wise Haar decomposition:
detail
row-transformed result … … … …
… …
… … …

79. Standard Haar wavelet decomposition (cont’d)
average
detail
(2) column-wise Haar decomposition:
row-transformed result
column-transformed result … … … … …
… …
… … … …

80. Example …
… …
row-transformed result …
… …
re-arrange terms

81. Example (cont’d)
column-transformed result … …
… … …

82. 2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.
φ0,0(x)
ψ0,0(x)
ψ1,0(x)
ψ1,1(x)
Example:
V2=V0+W0+W1

83. 2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.
φ00(x), φ00(x)
ψ00(x), φ00(x)
ψ10(x), φ00(x)

84. 2D Haar basis of standard decompositionV2

85. Non-standard Haar wavelet decomposition
Alternates between operations on rows and columns.
(1) Perform one level decomposition in each row (i.e., one step of horizontal pairwise averaging and differencing).
(2) Perform one level decomposition in each column from step 1 (i.e., one step of vertical pairwise averaging and differencing).
(3) Repeat the process on the quadrant containing averages only (i.e., in both directions).

86. Non-standard Haar wavelet decomposition (cont’d)
one level, horizontal
Haar decomposition:
one level, vertical
Haar decomposition: … …
x x x … x
… …
x x x x … …
… …
… … … …

87. Non-standard Haar wavelet decomposition (cont’d)
re-arrange terms … …
one level, vertical
Haar decomposition
on “green” quadrant
one level, horizontal
Haar decomposition
on “green” quadrant
… … … … … … …
… … …

88. Example
re-arrange terms … …
… … … …

89. Example (cont’d) … …
… … …

90. 2D Haar basis for non-standard decomposition
Defined through 2D scaling and wavelet functions:

91. 2D Haar basis for non-standard decomposition (cont’d)V2

92. 2D Haar basis for non-standard decomposition (cont’d)
Three sets of detail coefficients (i.e., subband encoding)
LL: average LL
Detail coefficients
LH: intensity variations along columns (horizontal edges)
HL: intensity variations along
rows (vertical edges)
HH: intensity variations along
diagonals

93. 2D DWT using filter banks (analysis)LL LH HL HH

94. 2D DWT using filter banks (analysis)LL
H LH
V HL
D HH LL LH LL HL HH
The wavelet transform is applied again on the LL sub-image

95. 2D DWT using filter banks (analysis)

96. 2D Inverse DWT using filter banks (synthesis)
H LH
V HL
D HH

97. Wavelets Applications
Noise filtering
Image compression
Fingerprint compression
Image fusion
Recognition
G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by Fusing Thermal Infrared and Visible Imagery", Image and Vision Computing, vol. 24, no. 7, pp , 2006.
Image matching and retrieval
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast Multiresolution Image Querying", SIGRAPH, 1995.

98. Fast Multiresolution Image Querying
painted
low resolution
target
queries
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast Multiresolution Image Querying", SIGRAPH, 1995.