1. Adaptive Wavelet Packet Models for Texture Description and Segmentation. Karen Brady, Ian Jermyn, Josiane Zerubia Projet Ariana - INRIA/I3S/UNSA June 5, 2003

2. Objectives Coherent set of probabilistic models for texture. Capture important periodicities and directionalities. Solution to boundary problem in texture. Ground truth images for objective evaluation.

3. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

4. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

5. The Segmentation Task Original Image I Possible classifications of original image I Label Set = { F - forest, S - sand, W - water } m[ I ] = c2 F S S W F m[ I ] = c3 Given the original image, and H, information on the textures represented, find the class map m[ I ] which yields the highest probability: S m[ I ] = c1 F S F W

6. Expanding the Probability Given a classification c, pixel values inside a region with a constant label do not depend on pixel values outside that region. Assumption 1: The probability of the pixel values inside the region labelled l does not depend on the class map outside this region. Assumption 2: F S S W F

7. What Do We Need? F S S W F Proposed classification of original image I Requirement 1:Probability distribution for a texture on an arbitrary finite region. Requirement 2:Prior probability on the class map, m[ I ], given the set H.

8. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

9. Texture Issues (1) Underlying Structure –Capturing important periodicities in the texture T1:FT(T1):

10. Texture Issues (2) Regional Nature of Texture a texture a single pixel coloured in green Needing a region to define texture makes it difficult to accurately identify the boundary of the texture

11. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

12. The Infinite Extendibility of Texture describing texture requires probability distributions on infinite images this is a probability distribution for the weave texture above, where I is an infinite image.

13. What We Have Vs. What We Need How do we go from the infinite image to an arbitrary finite region in a coherent manner? We have a distribution for sand on the space of infinite images In order to segment, we need a distribution for sand, but this time on an arbitrary finite region. Question:

14. Infinite Image To Finite Region (1) F S S W F Proposed classification of original image I Concentrate on the region labelled S. We want the probability of this region, given that it has label S and the information about texture S. The region labelled S divides the infinite image into two new images: - : the image on the region with label S - : the infinite image on the complement of the region with label S

15. Infinite Image to Finite Region (2) Define the maps: They form an orthogonal decomposition of the space of infinite images : Marginalizing over yields the probability measure for the image on the finite region R : In principle, this solves the boundary problem for texture.

16. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

17. The Gaussian Distribution Simplest starting point: Gaussian distribution on infinite images. where I is the infinite image and F is the inverse var-cov operator. Here F captures spatial correlations. It is not diagonal in the position basis unless pixels are independent. In the position basis the exponent is

18. Gaussian Distribution: Translation Invariance We require the distribution on infinite images to be translation invariant, making the operator F diagonal in the Fourier basis: The distribution is characterised by a function f on the Fourier domain.

19. Gaussian Distribution: Marginalization (1) Corresponding to the orthogonal decomposition of the space of infinite images I by the maps the operator F can be split up as follows:

20. Gaussian Distribution: Marginalization (2)

21. Gaussian Distribution: Marginalization (3) Marginalizing over yields the probability measure for the image on the finite region, In order to calculate this probability, we need to diagonalize the operator. For notational simplicity we denote the operator on by

22. Diagonalization of the Operator (1) If we can find a set of functions on the region R such that the infinite images are eigenfunctions of the operator F (with eigenvalues ), the application of the operator F on just results in the values of being multiplied by a constant value F then, …

23. Diagonalization of the Operator (2) The support of lies in R In addition, if the set B forms an orthonormal basis for the image on the finite region R, then we can write that: Thus we have diagonalized our operator.

24. To recap, diagonalization of the operator is a difficult task. However, if we can find a set to satisfy the following sufficient conditions then we can diagonalize the operator and write the exponent of the probability distribution as: Condition 1: The infinite images are eigenfunctions of the operator F (with eigenvalues ). Condition 2: The set B forms an orthonormal basis for the space of images on the finite region R. Diagonalization of the Operator (3)

25. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

26. Recall that translation invariance For an arbitrary function f(k) it is impossible to find a set B that satisfies these sufficient conditions. Choose a set of functions f that: –Is varied enough to capture structure. –Allows the two conditions to be satisfied. Consider: –The set of wavelet packet decompositions T of the Fourier domain. –The set Using Wavelets to Satisfy Condition 1 For any f in this set, the wavelet packet basis elements satisfy condition 1 up to “frequency leakage”.

27. Using Wavelets to Satisfy Condition 2 Now that we have shown that: for a function f which is piecewise constant on a particular wavelet packet decomposition, the set of basis elements forming that decomposition will satisfy condition 1 up to frequency leakage we must ask ourselves the question: how do we complete the set of wavelets inside the region to make a basis for the region and in doing so, satisfy condition 2? The answer to this question depends on the shape of the region...

28. Using Wavelets to Satisfy Condition 2 Square Dyadic-Sized Regions: Use a decimated wavelet packet decomposition to obtain a basis for the region. If we know and for a texture S, takes on the following form: : the number of pixels in subband : the eigenvalue of f on subband : the wavelet coefficient at pixel p in subband, : index for subbands in, the wavelet packet decomposition of texture S

29. Using Wavelets to Satisfy Condition 2 Arbitrary-Shaped Regions -- Optimal Solution Use decimated wavelets in the interior of the region and ‘ boundary wavelets ‘ on the boundary of the region. For future work. Interior plus boundary wavelets form a basis for the region. The frequency support of the boundary wavelets is coherent with that of the interior wavelets. Thus boundary wavelets solve the boundary problem.

30. Using Wavelets to Satisfy Condition 2 bad alignment with the boundary Problem 1Problem 2 region shifting wrt the basis Arbitrary-Sized Regions Using decimated wavelets all over the region leads to two problems:

31. Using Wavelets to Satisfy Condition 2 Arbitrary-Sized Regions -- Sub-optimal Solution Average over shifted subbands to get a ‘shift invariant’ measure of the energy in each subband. This results in using undecimated wavelets over the region. In this case the distribution,, for the texture takes on the following form: : the redundancy factor for subband : index for subbands in, the wavelet packet decomposition of texture S : the eigenvalue of f on subband : the wavelet coefficient at pixel p in subband

32. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

33. Parameter Estimation To find the optimal parameters for a given texture we examine the probability using Bayes’ law to arrive at we assume to be uniform, and choose the prior,, to penalise against large decompositions. Decimated form of the probability distribution for texture S Taking logs and differentiating wrt gives us the MAP estimate for :

34. Parameter Estimation: Algorithm A depth-first search algorithm finds the exact MAP estimates for T and f.

35. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

36. Pixel-wise Classification

37. Overview The Segmentation Task Texture Issues Abstract Development of a Texture Model Specific Development Using the Gaussian Distribution The Need For Wavelets Training of Texture Model Segmentation of Texture Mosaics Results

38. Classification: Synthetic Data

39. Classification: Real Data