Tractable Inference for Complex Stochastic Processes X. Boyen & D. Koller Presented by Shiau Hong Lim Partially based on slides by Boyen & Koller at UAI ‘98

Dynamic Systems Filtering in stochastic, dynamic systems: Monitoring freeway traffic (from an autonomous driver or for traffic analysis) Monitoring freeway traffic (from an autonomous driver or for traffic analysis) Monitoring patient’s symptoms Monitoring patient’s symptoms Models to deal with uncertainty and/or partial observability in dynamic systems: Hidden Markov Models (HMMs), Kalman Filters etc Hidden Markov Models (HMMs), Kalman Filters etc All are special cases of Dynamic Bayesian Networks (DBNs) All are special cases of Dynamic Bayesian Networks (DBNs)

Dynamic Bayesian Networks Markov assumption Partial observability: true state rarely actually known Maintain a belief state Maintain a belief state Belief state = probability distribution over all system states Belief state = probability distribution over all system states Propagate the belief state at time t to time (t+1) using a state evolution model and an observation model

Monitoring in practice Sometimes, belief states admit compact representations & manipulations: E.g. Kalman filters (assuming Gaussian processes) E.g. Kalman filters (assuming Gaussian processes) What about general Dynamic Bayesian Networks ?

DBN Myth Bayesian Network: a decomposed structure to represent the full joint distribution Does it imply easy decomposition for the belief state? No!

Tractable, approximate representation Exact inference in DBN is intractable Need approximation Maintain an approximate belief state Maintain an approximate belief state E.g. assume Gaussian processes E.g. assume Gaussian processes This paper: Factored belief state Factored belief state

Idea Use a decomposable representation for the belief state (pre-assume some independency)

Problem What about the approximation errors? It might accumulate and grow unbounded… It might accumulate and grow unbounded…

Contraction property Main result of the paper: Under reasonable assumptions about the stochasticity of the process, every state transition results in a contraction of the distance between the two distributions by a constant factor Under reasonable assumptions about the stochasticity of the process, every state transition results in a contraction of the distance between the two distributions by a constant factor Since approximation errors from previous steps decrease exponentially, the overall error remains bounded indefinitely Since approximation errors from previous steps decrease exponentially, the overall error remains bounded indefinitely

Basic framework Definition 1: Prior belief state: Prior belief state: Posterior belief state: Posterior belief state: Monitoring task:

Simple contraction Distance measure: Relative entropy (KL-divergence) between the actual and the approximate belief state Relative entropy (KL-divergence) between the actual and the approximate belief state Contraction due to O: Contraction due to T (can we do better?):

Simple contraction (cont) Definition: Minimal mixing rate: Minimal mixing rate: Theorem 3 (the single process contraction theorem): For process Q, anterior distributions φ and ψ, ulterior distributions φ’ and ψ’, For process Q, anterior distributions φ and ψ, ulterior distributions φ’ and ψ’,

Simple contraction (cont) Proof Intuition:

Compound processes Mixing rate could be very small for large processes The trick is to assume some independence among subprocesses and factor the DBN along these subprocesses Fully independent subprocesses: Theorem 5: Theorem 5: For L independent subprocesses T 1, …, T L. Let γ l be the mixing rate for T l and let γ = min l γ l. Let φ and ψ be distributions over S 1 (t), …, S L (t), and assume that ψ renders the S l (t) marginally independent. Then:

Compound processes (cont) Conditionally independent subprocesses Theorem 6 (the main theorem): For L independent subprocesses T 1, …, T L, assume each process depends on at most r others, and each influences at most q others. Let γ l be the mixing rate for T l and let γ = min l γ l. Let φ and ψ be distributions over S 1 (t), …, S L (t), and assume that ψ renders the S l (t) marginally independent. Then: For L independent subprocesses T 1, …, T L, assume each process depends on at most r others, and each influences at most q others. Let γ l be the mixing rate for T l and let γ = min l γ l. Let φ and ψ be distributions over S 1 (t), …, S L (t), and assume that ψ renders the S l (t) marginally independent. Then:

Efficient, approximate monitoring If each approximation incurs an error bounded by ε, then Total error Total error =>error remains bounded Conditioning on observations might introduce momentary errors, but the expected error will contract

Approximate DBN monitoring Algorithm (based on standard clique tree inference): 1. Construct a clique tree from the 2-TBN 2. Initialize clique tree with conditional probabilities from CPTs of the DBN 3. For each time step: a.Create a working copy of the tree Y. Create σ (t+1). b.For each subprocess l, incorporate the marginal σ (t) [X (t) l ] in the appropriate factor in Y. c.Incorporate evidence r (t+1) in Y. d.Calibrate the potentials in Y. e.For each l, query Y for marginal over X l (t+1) and store it in σ (t+1).

Conclusion Accuracy-efficiency tradeoff: Small partition => Small partition => Faster inference Better contraction Worse approximation Key to good approximation: Discover weak/sparse interactions among subprocesses and factor the DBN along these lines Discover weak/sparse interactions among subprocesses and factor the DBN along these lines Domain knowledge helps Domain knowledge helps