Variational filtering in generated coordinates of motion Approximate Inference in Stochastic Processes and Dynamical Systems Abstract This presentation reviews variational treatments of dynamic models that furnish time-dependent conditional densities on the path or trajectory of a system's states and the time-independent densities of its parameters. These obtain by maximizing a variational action with respect to conditional densities. The action or path-integral of free-energy represents a lower-bound on the model’s log-evidence or marginal likelihood required for model selection and averaging. This approach rests on formulating the optimization in generalized co-ordinates of motion. The resulting scheme can be used for on-line Bayesian inversion of nonlinear dynamic causal models and is shown to outperform existing approaches, such as Kalman and particle filtering. Furthermore, it provides for multiple inference on a models states, parameters and hyperparameters using exactly the same principles. Free-form (Variational filtering) and fixed form (Dynamic Expectation Maximization) variants of the scheme will be demonstrated using simulated (bird-song) and real data (from hemodynamic systems studied in neuroimaging).

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs notation

Hierarchical dynamic causal models

and generalised coordinates Dynamic models and generalised coordinates likelihood prior

Energies and generalised precisions Instantaneous energy General and Gaussian forms Precision matrices in generalised coordinates and time

Hierarchal forms and empirical priors Dynamic priors Structural priors

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs

Variational learning (steady-state) Aim: To optimise a free-energy bound on model evidence Free-energy: Expected energy: Entropy: Lemma 1: The free energy is maximised with respect to when Variational density: Mean-field approximation: Variational energy: Internal energy:

Ensemble learning (steady-state) Let the equations of motion for each particle be Because particles are conserved, their density over parameter space is governed by the free energy Fokker-Plank equation It is trivial to show that the stationary solution for the ensemble density is the variational density by substitution

Variational learning (dynamic) In a dynamic setting, the variational density and energy become functionals of time. By analogy with Lagrangian mechanics, denote the action by the anti-derivative or path-integral of free energy We now seek variational densities that maximise action. It is fairly easy to show that the solutions are functionals of the instantaneous energy Where and are the prior energies 21

Ensemble learning (dynamic) Lemma 2: is the stationary solution, in a moving frame of reference, for an ensemble whose equations of motion and ensemble dynamics are Proof: Substituting the variational density; gives This describes a stationary density under a moving frame of reference, with velocity as seen using the co-ordinate transform

Ensemble dynamics in generalized coordinates 20 40 60 80 100 120 -2 -1 1 2 3 4 5 -2 2 -5 5

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs

The Laplace approximation – why is it useful? Under the Laplace approximation, the variational density assumes a Gaussian form The conditional precision (a function of the mode) obtains by extremising the free-energy

… for dynamic models The conditional precisions obtain by extremising the action Conditional precisions: which leaves the variational modes that optimise variational energy and action Variational energy and actions: Mean-field terms:

Approximating the mode Lemma 3: The path of a particle, whose motion is converges exponentially to the mode Where the trajectory of the conditional mode can be realised with a local linearisation (Ozaki 1992) to give the update

… an augmented gradient ascent In dynamic systems, the trajectory of the conditional mode maximises variational action, which is the solution to the ansatz Here, can be regarded as motion in a frame of reference that moves along the trajectory encoded in generalised coordinates. The stationary solution, in this moving frame of reference, maximises variational action. This can be seen easily by noting This is sufficient for the mode to maximise variational action (by the Fundamental lemma); c.f., Hamilton's principle of stationary action

DEM and belief propagation D-Step inference E-Step learning M-Step uncertainty A dynamic recognition system that minimises prediction error

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs

A linear convolution model Prediction error Generation Inversion

Variational filtering on states and causes 5 10 15 20 25 30 -1 -0.5 0.5 1 1.5 hidden states time cause 5 10 15 20 25 30 -0.4 -0.2 0.2 0.4 0.6 0.8 1 1.2 cause time {bins}

Linear deconvolution with variational filtering (SDEs) Linear deconvolution with Dynamic expectation maximisation (ODE)

The order of generalised motion Precision in generalised coordinates The order of generalised motion 10 20 30 40 -1.5 -1 -0.5 0.5 1 1.5 2 time Accuracy and embedding (n) 1 3 5 7 9 11 13 2 4 6 sum squared error (causal states) 5 10 15 20 25 30 35 -1.5 -1 -0.5 0.5 1 1.5 2 time

DEM and extended Kalman filtering hidden states DEM and extended Kalman filtering 5 10 15 20 25 30 35 -1.5 -1 -0.5 0.5 1 time DEM(0) DEM(4) EKF true With convergence when sum of squared error (hidden states) EKF 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 DEM(0) DEM(4) 10 20 30 40 -1 -0.5 0.5 1 time hidden states DEM(0) EKF

A nonlinear convolution model level This system has a slow sinusoidal input or cause that excites increases in a single hidden state. The response is a quadratic function of the hidden states (c.f., Arulampalam et al 2002).

DEM and particle filtering Comparative performance Sum of squared error

Triple estimation (DEM) Inference on states Triple estimation (DEM) Learning parameters

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs

An fMRI study of attention Stimuli 250 radially moving dots at 4.7 degrees/s Pre-Scanning 5 x 30s trials with 5 speed changes (reducing to 1%) Task: detect change in radial velocity Scanning (no speed changes) 4 x 100 scan sessions; each comprising 10 scans of 4 different conditions F A F N F A F N S ................. A – dots, motion and attention (detect changes) N – dots and motion S – dots F – fixation PPC V5+ Buchel et al 1999

Output: a mixture of intra- and extravascular signal A hemodynamic model Visual input Motion Attention convolution kernel state equations output equation Output: a mixture of intra- and extravascular signal

Hemodynamic deconvolution Inference on states Hemodynamic deconvolution Learning parameters

… and a closer look at the states

Overview Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs

Synthetic song-birds syrinx hierarchy of Lorenz attractors

Song recognition with DEM

… and broken birds

Summary Dynamic causal models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics Bird songs