Geometric diffusions as a tool for harmonic analysis and structure definition of data By R. R. Coifman et al. The second-round discussion* on * The first-round discussion was led by Xuejun; * The third-round discussion is to be led by Nilanjan.

Diffusion Maps Purpose - finding meaningful structures and geometric descriptions of a data set X. - dimensionality reduction Why? The high dimensional data is often subject to a large quantity of constraints (e.g. physical laws) that reduce the number of degrees of freedom.

Markov Random Walk Symmetric Kernel Diffusion Maps Many works propose to use first few eigenvectors of A as a low representation of data (without rigorous justification). Relationship

Diffusion maps Spectral Decomposition of A Diffusion Maps where Spectral Decomposition of A m

Diffusion distance of m-step Interpretation Diffusion Distance The diffusion distance measures the rate of connectivity between x i and x j by paths of length m in the data.

Diffusion vs. Geodesic Distance

Data Embedding By mapping the original data into (often ) The diffusion distance can be accurately approximated

Example: curves Umist face database: 36 pictures (92x112 pixels) of the same person being randomly permuted. Goal: recover the geometry of the data set.

Original orderingRe-ordering The natural parameter (angle of the head) is recovered, the data points are re-organized and the structure is identified as a curve with 2 endpoints.

Original set: 1275 images (75x81 pixels) of the word “3D”. Example: surface

Diffusion Wavelet A function f defined on the data admits a multiscale representation of the form: Need a method compute and efficiently represent the powers A m.

Multi-scale analysis of diffusion Discretize the semi-group {A t :t>0} of the powers of A at a logarithmic scale which satisfy Diffusion Wavelet

The detail subspaces Downsampling, orthogonalization, and operator compression  - diffusion maps: X is the data set A - diffusion operator, G – Gram-Schmidt ortho-normalization, M - A  G

Diffusion multi-resolution analysis on the circle. Consider 256 points on the unit circle, starting with  0,k =  k and with the standard diffusion. Plot several scaling functions in each approximation space V j.

Diffusion multi-resolution analysis on the circle. We plot the compressed matrices representing powers of the diffusion operator. Notice the shrinking of the size of the matrices which are being compressed at the different scales.

Multiscale Analysis of MDPs [1] S. Mahadevan, “Proto-value Functions: Developmental Reinforcement Learning”, ICML05. [2] S. Mahadevan, M. Maggioni, “Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions”, NIPS05. [3] M. Maggioni, S. Mahadevan, “Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes”, ICML06.

To be discussed a third-round led by Nilanjan Thanks!