1. Wavelet Packets  Shortcomings of standard orthogonal (bi-orthogonal) multi-resolution structure of DWT,  Insufficient flexibility for the analysis of signal under various application requirements

2. Wavelet Packets  Signal components with wide range of frequencies, are packed into details D1  Desire to focus on signal components with spectral bandwidths that are sufficiently narrow at high and low frequency ranges.

3. Wavelet Packets  Adaptive Decomposition There are application areas such as compression or pattern recognition/ classification problems where it is highly desirable to identify wavelets with certain properties suitable for the requirements of a given applications including the following

4. Application Requirements Demand for High Resolution both in Time and in Frequency Signal decomposition and approximation by a few number of large amplitude coefficients for high compression, Discriminatory separation of different classes in pattern recognition and classifications problems, High efficiency of computations,

5. Beyond Orthogonality Compression Requirements Slow rate of decay Fast rate of decay Coeffs 2Coeffs 1 Orthogonal ExpansionNon-orthogonal Expansion

6. Dictionary Generation Redundancy and “Best Basis ”  To use redundancy in wavelet packet to generate a dictionary that is composed of large number of basis functions ( basis functions that are not necessarily orthogonal) and as such to have a higher degree of freedom for selecting wavelets suitable for a given application

7. Time-Frequency Partitioning in Standard DWT Constant Q factor Fc F Fc / F = constant Phase-plane Diagram of standard DWT and Frequency Partitioning

8. Perfect Reconstruction QMF Filterbank High Pass Low PassLow pass High pass Perfect Reconstrution Filter bank Wavelet Subspace W

9. Standard QMF Structure of DWT

10. Wavelet Packet Structure  Extended QMF structure  Decomposition of signal i.e. splitting of signal into high and low resolution, is applied to high resolution details as shown below.  This is followed similarly during the synthesis stage

11. Subspace Illustration of Two Channel Wavelet Packet Wj-1,0 Vj-1 Vj Wj-1,1

12. Wavelet Packets High Pass Low Pass 2 2 2 Low pass High pass Wavelet Packets -Analysis Stages 2 High freq details High pass Low pass 2 2 V space of the signal

13. Standard and Wavelet Packet Structure

14. Frequency Partitioning

15. Wavelet Packets Tree

16. Binary Tree of DWT in Wavelet Packet

17. Wavelet, scaling Basis at WP Cells

18. Spectrum of Basis inWP

19. Redundancy in Wavelet Packet Decomposition  Wavelet packet decomposition of a signal, results in a considerable redundancy and an increase in the number of wavelet bases in which the size of the library will grow rapidly when the number of scale levels is increased. Redundancy results in a substantial increase of both computational and storage costs

20. Basis Selection Algorithm  A pruning algorithm is needed for selecting a subset of nodes for signal representation considered suitable for a given application and reducing computational costs. These nodes should provide a sufficiently accurate approximation to a given signal

21. Basis Selection Algorithm  a selection criteria is needed in which information cost of each node in the expansion tree is utilized as the basis for retaining or discarding the node. Different cost functions may result in different approximations ( see below).

22. Basis Designation Indices  Parent node of a pair of children nodes  A library of bases functions that are indexed by three parameters (instead of two parameters of scale and translation as used in standard multiresolution structure) as follows. scale index j, spatial location (translation) index k frequency bands indexed by n.

23. Alternative Decomposition, Orthogonal or non-orthogonal  A signal can be decomposed using orthogonal bases functions that reside at children nodes at a given scale.  However this may not be a ‘best representation for a given application though an orthogonal representation..  Computational cost of signal decomposition by wavelet packet is of order O(Nlog(N)) as compared with standard wavelet decomposition which is of order of N ( i.e linear cost).  Frequency bandwidths of the details and pproximation at a given scale are equal in size, similar to FFT having an order of computational complexity of Nlog(N).

24. Dictionary Construction, Best Basis Selection  Cost Functions as Criteria for Best Basis Selection  Cost functions are defined in accordance with the requirements of a particular application.  They are often described in terms of concentration of information contained in the coefficients  An Example: entropy-based cot function

25. Properties of Cost Functions  ‘Additive property’  Under additive property, total cost is given by the sum of costs of individual nodes in a tree. It relates information cost of a node to information costs of individual coefficients at that node. C({xi}= C{ (xi)}= C(xi) C(0)=0

26. Different Criteria and Cost Function  Entropy Measure Entropy is a measure of uncertainty in predicting a particular outcome of an experiment Pdf of random variable x Low Entropy High Entropy xx

27. Entropy Measure  We define Shannon-Weaver entropy of a sequence of x={xj, j=1,2,…n} as following.  x={xj}, H(x)= - pj log pj  pj=||xj||2/ ||x||2 and p.log p=0 if p=0  Coefficients at a node are used as xj.

28. Entropy Measure  Under an entropy criteria for best bases selection, wavelet packet nodes with the coefficients that are distributed across a narrow dynamic range are selected, leading to high compression rate for signal representation  Comparison with Karhunen-Loeve transform of PCA

29. Symmetric Entropy Measure  Symmetric Entropy and Discriminatory Classification.  x={xi}, H(x)= -[ pj log pj/ qj + qj log qj/ pj ]  pj=||xj||2/ ||x||2 and p.log p=0 if p=0  qj=||yj||2/ ||y||2 and q.log q=0 if q=0  coefficients at wavelet packet nodes are used for  x={xi} and y={yj}

30. Log of Energy Criteria  M(x) = log|xj|2,  log0=0  Log of energy may be interpreted as the entropy of so called Gauss- Markov process, a stochastic process composed of N uncorrelated Gaussian random variables with variances as  2 1 =|x 1 |2, …..2 N =|x N |2

31. Log of Energy Criteria  Minimizing log energy, finds a best basis and best approximation for signal representation using minimum variance criteria similar to principal component analysis (PCA) and Karhunen-Lo’ve Transform (KLT)

32. Concentration in l p norm, p<2.  For this norm, we choose p<2 and set M(x)=||{x}||p  Maximization of this norm results in selecting a few large amplitude coefficients with high concentration of signal energy in few coefficients. That is, in a binary tree of WP structure, nodes are selected that have a few large amplitude coefficients leading to a peaky pdf

33. Secondary Feature of WT and l p norm  In signal expansion by wavelets, it is also observed that often large amplitude coefficients are large across several neighboring scales as well as neighboring coefficients at the given scale.  This occurs at points of sudden changes and singularities where clustering of coefficients is observed both at within scale and cross-scales.  This property of wavelets, is often referred to as a secondary feature of wavelet analysis.

34. Other Cost Function, Selection Rules.  “Informative Wavelets”,  “Dictionary Projection Pursue”’  “Local Discriminatory Basis” selection criteria  “Matching Pursue” On items 1 and 3, papers have been included in ‘Archives of Project Reports’ posted on course web site

35. Best Bases and Best Tree  Best tree’ is a tree in the binary array of the signal decomposition that corresponds to minimal cost function.  And ‘best basis’ are the basis set that belong to the nodes of the ‘best tree’. For example in entropy-based basis selection, best basis are wavelets(and scaling functions) of those nodes having minimal entropy of the coefficients at the node.Best Tree: Refer to Matlab Wavemenu for Illustrations

36. Illustrative Examples of basis

37. Illustrative Examples of WP Basis

38. On the Illustrative Examples of WP Basis  Note Db 45 (of longer length) generates higher oscillatory wavelets (higher # of oscillations) as compared with those of Db 5 (of short length). Also note equi- frequency bandwidth of different wavelets of the same scale.  Translation of wavelets are also seen easily for different wavelets indicating the shift in the location of center of the wavelets.  Note time span of illustrations are all equal to the length of wavelet at lowest scale