1. Multiscale Geometric Signal Processing in High Dimensions
Hyeokho Choi Richard Baraniuk
Wai Lam Chan Mike Wakin
I’d like to share some ideas we recently had on multiscale image modeling in the wavelet domain. We will define a new wavelet in 2-D, something called quarternion wavelet, and then we use their phase and magnitude to analyze and model some key features of images.

2. Geometric Image Features
Edges:
“location”
“orientation”
Edges are important in images. We can divide up the image into local blocks and they are either edges, smooth regions or texture.
In this work, we are mainly interested in edges. For edge blocks shown here, it is important to encode the location, or offset, and angle or orientation. By this analysis, our goal is to build a multiscale edge models for natural images.
LOCALLY STRAIGHT EDGE GOING THRU A BLOCK.
Goal: multiscale feature analysis

3. Location Information in 1-D
Fourier phase analysis?
First, let’s look at how the “location” information can be analyzed. The traditional way of encoding location info is thru the phase of Fourier transform coefficients. If you look at the cross section of an image, it is a piecewise smooth signal. Thus, analyzing the edges corresponds to analyzing the location of jumps. As you all know the Fourier transform phase is proportional to the signal shift and you might think that you can look at the Fourier phase to tell the locations of jumps. However, this is not easy because the Fourier transform has no time localization. That is, the Fourier phase represent the shift or “location” of entire signal rather than local jump locations.
DO WE NEED THE PIC?
Linear phase change as signal shifts
No time localization

4. Local Fourier Analysis
Local Fourier analysis for “local” location
Short time Fourier transform (Gabor analysis)
Local bandpass filtering
Uniform time-frequency tiling
To solve the problem of no time localization, we can use local Fourier analysis, that is, short time Fourier transform. You use windowed Fourier analysis and then, the phase encodes local signal shift. Essentially, STFT is local bandpass filtering and on the time-frequency plane, you have uniform partitioning.
DO WE NEED FOURIER PIC? t f

5. Wavelet Analysis “Multiscale” analysis
Sparse representation of piecewise smooth signals
Orthonormal basis / tight frame
Fast computation by filter banks
Although STFT has both time and frequency localization, it’s half success. For example, it’s very expensive to compute and hard to invert STFT. Instead, you can analyze using so called wavelet transform that is much better suited for analysis of real world signals. Again, wavelet transform is a local bandpass filtering, but unlike STFT, it’s a “multiscale” analysis. And, you can easily invert it because it’s basically expansion of the signal in terms of orthonormal basis. The the computation is fast using filterbank implementation. Most importantly, for piecewise smooth signals, the wavelet coefficeints are sparse. Sparcity is huge advantage because you essentially have very small number of coefficients to process.
Make this filter bank kind of picture.

6. 1-D Wavelet Transform HP LP

7. 1-D Wavelet Transform HP HP LP LP

8. 1-D Wavelet Transform 2 HP LP 2 HP 2 2 LP

9. Short Time Fourier vs. Wavelet
Once again, comparing the atoms of short time Fourier transform and wavelets,
We have windowed cos and sin functions of different frequencies for STFT, and we have dilation and shrinkage of mother wavelet function at dyadic shifts for wavelets. In the time-frequency plane, STFT divides up the plane uniformly, while the wavelets divide it in a multiscale way so that we have better time resolution in high frequencies.
“Real” f f t t

10. Phase for Wavelets ? Need to have quadrature component phase shift of
“Hilbert Transform”

11. Complex Wavelet Complex wavelet transform (CWT)
[Kingsbury,Selesnik,Lina,…]
Phase linear to local signal shift

12. 1-D Complex Wavelet Transform (CWT)
Complex (analytic) wavelet
+ j ×
wavelet
Hilbert Transform
Explain the multiplication of j and -j =
+ j* +1 +1 +j -j +2

13. 2-D Fourier Analysis
Phase ambiguity
cannot obtain from phase shift

14. Quaternion Fourier Transform (QFT)
[Bulow et al.]
Separate 4 “quadrature” components
Organize as quaternion
Quaternions:
Multiplication rules: and

15. QFT Phase Quaternion phase angles: Shift theorem QFT shift theorem:
invariant to signal shift
linear to signal shift

16. Quaternion Wavelet!

17. “Real” 2-D Wavelet Transform
This need better explantaion

18. “Real” 2-D Wavelet Transform
This need better explantaion

19. “Real” 2-D Wavelet TransformHH HL
This need better explantaion LH LL

20. 2-D Hilbert Transform u v u v
HT in u
HT in v
HT in both u v u v

21. 2-D Hilbert Transform u v u v u v u v

22. Quaternion Wavelet +1 +1 +1 +1
Fix the figure so that they animate.

23. Quaternion Wavelet +1 +1 +j -j +j +1 +1 HTx HTy HTy +1 +1 +j -j +1 -1

24. Quaternion Wavelet Transform (QWT)
Quaternion basis function
3 subbands (HH, HL, LH)
single-quadrant spectrum (QFT domain) v
LH subband
Like we have only positive freq portion in 1-d analytic (complex) wavelet, we have only 1st quadrant of the QFT plane for 2-D QWT.
HH subband
HL subband u

25. QWT Applications Edge parameter (offset/orientation) estimation
 edge offset
QWT magnitude  edge orientation
Get rid of the plot?

26. QWT phase-based flow estimation
Image 2
Image 1
QWT
QWT
Phase difference
Local image shift

27. QWT phase-based flow estimation
Rotated image
Test image
Local estimation of image shift : motion field
Phase as local image features

28. Flow Estimation Example

30. Summary: Quaternion Wavelets
Quaternion wavelets for 2-D image processing
To date:
2-D quaternion wavelet implementation
flow analysis using quaternion phase
Current and future work:
- theoretical analysis of quaternion phase
statistical geometry modeling in QWT domain
extension to higher dimensions

31. Part II : Multiscale Image Manifold

32. Multiple View Images
“Light Field”

33. Multiple View Images “Light field Manifold” RN2
What is the dimensionality of the manifold?
RN2

34. Manifold Signal Processing
Often data can be interpreted as living along a low-dimensional manifold in a high-dimensional ambient space
Ex: each NxN image is a point in RNxN but most points in RNxN look like “noise”
Samples on manifold: point clouds

35. 3-D pose estimation

36. Manifold Navigation
Navigation guided by multiscale tangents

37. 3-D pose estimation

38. s = 1/2

39. s = 1/4

40. s = 1/16

41. s = 1/256

42. Summary: Multiscale Manifolds
Multiscale manifold analysis in high dimensions
To date:
multiscale tangents and navigation
image articulation manifold
imaging parameter estimation
Current and future work:
- geometric properties of articulation manifold
application to ATR problem