Hilbert Space Embeddings of Conditional Distributions -- With Applications to Dynamical Systems Le Song Carnegie Mellon University Joint work with Jonathan Huang, Alex Smola and Kenji Fukumizu

Hilbert Space Embeddings Summary statistics for distributions: (mean) (covariance) (expected features)

Hilbert Space Embeddings Reconstruct from for certain kernels! Eg. RBF kernel. Sample average converges to true mean at.

Hilbert Space Embeddings Embed Conditional Distributions p(y|x)  µ[p(y|x)] Embed Conditional Distributions p(y|x)  µ[p(y|x)] Structured Prediction Dynamical Systems

Embedding P(Y|X) Simple case: X binary, Y vector valued Need to embed Solution: – Divide observations according to their – Average feature for each group What if X is continuous or structured?

Goal: Estimate embedding from finite sample Problem: Some X=x are never observed… Embed p(Y|X=x) for x never observed? Embedding P(Y|X)

If and are close, and will be similar. Key Idea Generalize to unobserved X=x via a kernel on X

Conditional Embedding Conditional Embedding Operator

generalization of covariance matrix

Kernel Estimate Empirical estimate converges at

Sum and Product Rules Probabilistic Relation Hilbert Space Relation Sum Rule Product Rule Can do probabilistic inference with embeddings!

Dynamical Systems Maintain belief using Hilbert space embeddings Hidden State observation

Advantages Applies to any domain with a kernel Eg. reals, rotations, permutations, strings Applies to more general distributions Eg. multimodal, skewed

Linear dynamics model (Kalman filter optimal): Evaluation with Linear Dynamics Trajectory of Hidden States Observations

Kalman   N(0,10 I),   N(0,10 -2 I) Training Size ( Steps) RMS Error RKHS Gaussian process noise, Gaussian observation noise

   (1,1/3),   N(0,10 -2 I) Training Size ( Steps) RMS Error RKHS Kalman Skewed process noise, small observation noise

 0.5N(-0.2,10 -2 I)+0.5N(0.2,10 -2 I),  N(0,10 -2 I) Training Size ( Steps) RMS Error RKHS Kalman Bimodal process noise, small observation noise

Camera Rotation Rendered using POV-ray renderer

Predicted Camera Rotation Ground truth RKHSKalman filterRandom Trace measure Dynamical systems with Hilbert space embeddings works better

Video from Predicted Rotation Original Video Video Rendered with Predicted Camera Rotation

Conclusion Embedding conditional distributions – Generalizes to unobserved X=x – Kernel estimate – Convergence of empirical estimate – Probabilistic inference Application to dynamical systems learning and inference

The End Thanks

Future Work Estimate the hidden states during training? Dynamical system = chain graphical model; how about tree and general graph topology?

  N(0,10 -2 I),   N(0,10 I) Training Size (2 n Steps) RMS Error RKHS Kalman Small process noise, large observation noise

Dynamics can Help log10(Noise Variance) Trace Measure

Dynamical Systems Maintain belief using Hilbert space embeddings Hidden State observation

Conditional Embedding Operator Desired properties: Covariance operator: generalization of covariance matrix