1. Wavelet-Based Denoising Using Hidden Markov Models
ELEC 631 Course Project
Mohammad Jaber Borran

2. Some properties of DWT Primary Secondary Locality  Match more signals
Multiresolution
Compression  Sparse DWT’s
Secondary
Clustering  Dependency within scale
Persistence  Dependency across scale

3. Probabilistic Model for an Individual Wavelet Coefficient
Compression  many small coefficients
few large coefficients S W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)

4. Probabilistic Model for a Wavelet Transform
Persistence 
Hidden Markov Tree Model t f
Ignoring the dependencies 
Independent Mixture (IM) Model t f
Clustering 
Hidden Markov Chain Model

5. Parameters of HMT Model
pmf of the root node
transition probability
(parameters of the)
conditional pdfs
e.g. if Gaussian Mixture is used
q : Model Parameter Vector

6. Dependency between Signs of Wavelet Coefficients
Signal
Wavelet t T w1
T/2 w2

7. New Probabilistic Model for Individual Wavelet Coefficients
Use one-sided functions as conditional probability densities S W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)
pS(3)
fW|S(w|3)
pS(4)
fW|S(w|4)

8. Proposed Mixture PDF
Use exponential distributions as components of the mixture distribution
If m is even:
If m is odd:

9. PDF of the Noisy Wavelet Coefficients
Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance s2, then we have
Noisy wavelet coefficient,
If m is even:
If m is odd:

10. Training the HMT Model y: Observed noisy wavelet coefficients
s: Vector of hidden states
q: Model parameter vector
Maximum likelihood parameter estimation:
Intractable, because s is unobserved (hidden).

11. Model Training Using Expectation Maximization Algorithm
Define the set of complete data, x = (y,s)
and then,

12. EM Algorithm (continued)
State a posteriori probabilities are calculated using Upward-Downward algorithm
Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U.
Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.

13. Denoising
MAP estimate:

14. Denoising (continued)
Conditional mean estimate:

18. Conclusion
Mixture distributions for individual wavelet coefficients can effectively model the non–Gaussian nature of the coefficients.
Hidden Markov Models can serve as a powerful tool for wavelet-based statistical signal processing.
One-sided exponential distributions for mixture components along with hidden Markov Tree model can achieve better performance in denoising.