1. Dynamic Bayesian Networks (DBNs)
Dave, Hsieh Ding Fei
Frank, Yip Keung

2. Outline Introduction to DBNs Inference in DBNs Applications Conclusion
Type of inference
Exact inference
Approximate inference
Applications
Conclusion

3. Introduction to DBNs Motivation Bayesian Network (BN) Models DBN
Static nature of the problem domain
Observable quantity is observed once for all
Confidence in the observation is true for all time
DBN
Domains involving repeated observations
Process dynamically evolves over time
Examples: Monitoring a patient, traffic monitoring, etc.

4. Introduction to DBNs Assumptions
The process is modeled as discrete time-slice
At time 1, state is X(1) , at time t, state is X(t)
P(X(1),…, X(t))=P(X(1)) P(X(1)|X(2) )…P(X(t)|X(1),…, X(t-1))
Markov property
Given current state, the next state is independent of previous states
P(X(1),…, X(t))=P(X(1)) P(X(1)|X(2) )…P(X(t)|X(t-1))

5. Introduction to DBNs DBN model (DAG representation)
Edge means how tight the coupling is between nodes
Effect is immediateedge within same time slice
Effect is long termedge between time slices

6. Introduction to DBNs Special case of DBN  HMM
State of HMM evolves in a Markovian way
Model HMM as a simple DBN
Each time slice contains two variables which are state q and observation o

7. Inference Type of Inference Prediction Monitoring
Given a probability distribution over current state, predict the distribution over future states
Monitoring
Given the observation (evidence) in every time slice t, maintain the distribution over the current state
Belief state at time T P(X(T) | o(1) ,…, o(T))

8. Inference Probability Estimation Explanation
Given a sequence of observations in every time slice t, determine the distribution over each intermediate state
P(X(t) | o(1) ,…, o(T)) for t = 1, 2, … , T
Explanation
Given an initial state and a sequence of observations o(1) ,…, o(T), determine the most likely sequence of states X(1) ,…, X(T)

9. Exact inference
For most inference tasks, a belief state need to be maintained
belief state
A probability distribution over the current state
This state summarize all information about history
Need to be maintained compactly

10. Exact inference How to accomplish exact inference
How to do this in a simple DBN  HMM
Given a number of time slices, the DBN is just a very long BN with regular structure
Standard Bayesian network algorithms can be used
Probability estimation task
Clique tree propagation algorithm
Forward-backward algorithm

11. Exact inference Monitoring task Explanation task Prediction task
Only the forward pass of forward-backward algorithm
Explanation task
Viterbi’s algorithm
Prediction task
Only base on the current belief state because it already have the history information

12. Exact inference dHugin : an exact inference computational system
Inference method of classical discrete time-series analysis
Allows discrete multivariate dynamic system

13. dHugin introduce notion of dynamic time window
Contain several time slice and represent by junction tree
Operations: window expansion and reduction
Expand window to perform forecasting
Inference are formulated in terms of message passing in junction tree

14. dHugin Window expansion
Move k new consecutive time slices to the forecast model
Move the k oldest time slices of the forecast model to the time window
Moralize the compound graph including the graph in window and the new k slices
Triangulate the time window
Construct new junction tree

15. dHugin Window reduction Suppose has k+1 time slices in time window
make the k oldest slices in time window become k backward smoothing models
The remain (k+1)’st slices is the new time window

16. Forecasting
Calculate estimates of the distributions of future variables given past observations and present variables
Forecasting within window
Propagation
Forecasting beyond the window
A series of alternating expansion and reduction step
Propagation performed in each step

17. Problem of Exact inference
Drawback: complex and require large space for computations
Key issue is how to maintain the belief state
Represent it naively
Require an exponential number of entries
Cannot represent it compactly by exploiting the structure
no conditional independence structure
Variables becomes correlated each other when time goes on
Prevent using factorization ideas
Not even conditionally independent within this time slice

18. Approximate Inference
Objective
Try to maintain and propagate an approximate belief state when the state space is very large in dynamic process
It improves the complexity of probabilistic inference

19. Approximate Inference
Two approaches
Structural approximation
Ignore weak correlations between variables in a belief state
Stochastic simulation
Randomly sample from the states in the belief state

20. Structural Approximation
Problems in exact inference
All variables in a belief state are correlated
Belief state is expressed as full joint distribution  Need exponential number of table entries
Objective of structural approximation
Use factorization in order to represent complex system compactly by exploiting the fact that each variable has weak interaction with each other

21. Structural Approximation
Example (monitor a freeway with multiple cars)
States of different cars (e.g velocity,location..etc) become correlated after a certain period of time
Approximation is to assume that the correlations are not very strong
Each car can be considered as independent
The approximate belief state can be represented in a factorized way, as a product of separate distributions, one for each car

22. Structural Approximation
We can define a set of disjoint clusters Y1,…, Yk such that Y = Y1  Y2  …  Yk . We maintain an approximate belief state :
If this approximate belief state of time t is simply propagated forward to time t+1, all variables would become correlated again

23. Structural Approximation
It can be solved by executing the below process
At each time t, we take and propagate it to time t+1, obtain a new distribution
Approximate using independent marginal
Compute for every I
Ie.
The product of each marginal is

24. Structural Approximation
Two sources of error
The accumulated error results from propagation
The error results from approximation of
Errors are bounded due to two opposing forces
Propagation from time t to time t+1 adds noise to exact and approximate belief state  reduce difference between them  reduce error
Approximation  increase error

25. Stochastic Simulation
Likelihood Weighting (LW)
Find the approximate belief state using sampling
Algorithm of LW

26. Stochastic Simulation
Drawback
LW generates the samples at time t according to prior distribution (depends on condition of samples at time t-1)
Observation affects the weights, but not the choice of samples
Samples generated get increasingly irrelevant when time grows as some samples are not likely to happen to explain the current observation
Example of monitoring car’s location

27. Stochastic Simulation
Samples at t = 5 are more distributed, far away from exact location of vehicle
An improved algorithm called Survival-Of-Fittest is used

28. Stochastic Simulation
Survival-Of-Fittest (SOF)
Propagate likely samples more often than unlikely samples
Algorithm of SOF

29. Stochastic Simulation
Belief state propagation over time
(a) exact belief state
(b) belief state by using LW
(b) belief state by using SOF

30. Application Robot localization
Track a robot moving around in an environment
State variables
x, y location
Orientation
Transition model corresponds to motion
Next position is a Gaussian around a linear function of current position
Observation model
Probability that sonar detect an obstacle

31. Conclusion Concept DBNs Inference in DBNs Applications
Four types of inference
Exact inference
dHugin
Approximate inference
Structural approximation
Search –based
Stochastic simulation
Applications
robot localization