Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computer Science CPSC 502 Lecture 11 Probability and Time (Ch. 6.5)

Similar presentations


Presentation on theme: "Computer Science CPSC 502 Lecture 11 Probability and Time (Ch. 6.5)"— Presentation transcript:

1 Computer Science CPSC 502 Lecture 11 Probability and Time (Ch. 6.5)

2 Where are we?  Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Value Iteration Variable Elimination Belief Nets Decision Nets Markov Processes Static Sequential Representation Reasoning Technique Variable Elimination Belief Nets extended to cover temporal processes Belief Nets Markov Processes Value Iteration Variable Elimination Decision Nets Markov Processes Variable Elimination Approximate Inference Temporal Inference

3 Overview  Modelling Evolving Worlds with DBNs  Markov Chains  Hidden Markov Models  Inference in Temporal Models

4 Modeling Evolving Worlds  So far we have looked at techniques for probabilistic reasoning in a static world  E.g., keep collecting evidence to diagnose the cause of a fault in a system.  The true cause does not change as one gathers new evidence, what changes is the …………… over the possible causes.

5 Dynamic Bayesian Networks (DBN)‏  DBN are an extension of Bayesian networks devised for reasoning under uncertainty in dynamic environments  Basic approach World’s dynamics captured via series of snapshots, or time slices, each representing the state of the world at a specific point in time Each time slice contains a set of random variables, representing the state of the world at time t: state variables X t E.g., student’s knowledge and morale during a tutoring session  This assumes discrete time; step size depends on problem Notation: X a:b = X a, X a+1,…, X b-1, X b Knows-Sub 1 Knows-Add 1 Morale 1 Knows-Sub 2 Knows-Add 2 Morale 2 Knows-Sub 3 Knows-Add 3 Morale 3

6 Stationary Processes  How do we build a Bnet from these times slices and their variables?  Could use the procedure we defined for building static Bnets order variables (temporally), insert them in the networks one at time, find suitable parents by checking conditional dependencies given predecessors  First Problem – we could have a very long sequence of time slices: how to specify CPTs for all of them? Assumption of stationary process: the mechanism that regulates how state variables change overtime is stationary, that is it can be described by a single transition model P(X t |X t-1 ) Note that X t is a vector representing a set of state variables

7 Markov Assumption  Second Problem: there could be an infinite number of parents for each node, coming from all previous time-slices Markov assumption: current state X t depends on bounded subset of previous states X 0:t-1  Processes satisfying this assumption are called Markov Processes or Markov Chains

8 Slide 8 Simplest Possible DBN Intuitively S t conveys all of the information about the history that can affect the future states. “The future is independent of the past given the present.” One random variable for each time slice : let’s assume S t represents the state at time t with domain {s 1 …s n } Each random variable depends only on the previous one, thus P(S t |S t-1, ….S 0 ) =

9 Simplest Possible DBN (cont’) Stationary process assumption: the mechanism that regulates how state variables change overtime is stationary, that is it can be described by a single transition model P(S t |S t-1 ) How many CPTs do we need to specify? Slide 9

10 Stationary Markov Chain (SMC) A stationary Markov Chain : for all t >0 P (S t+1 | S 0,…,S t ) = P (S t+1 |S t ) P (S t +1 |S t ) is the same for every t We only need to specify P (S 0 ) and P (S t +1 |S t ) Simple model, easy to specify Often the natural model The network can extend indefinitely Variations of SMC are at the core of most Natural Language Processing (NLP) applications! Slide 10 Markov Assumption (first order) Stationary Process

11 Slide 11 Stationary Markov-Chain: Example Probability of initial state t tqp q p a h e ahe Domain of variable S i is {t, q, p, a, h, e} We only need to specify… t q.6.4 p a h e Stochastic Transition Matrix P (S 0 ) P (S t+1 |S t ) P (S t+1 | S t =q) P (S t+1 | S t =a)

12 Slide 12 Markov-Chain: Inference Probability of a sequence of states S 0 … S T Example: t q.6.4 p a h e t tqp q p a h e ahe P (S 0 ) P (S t+1 |S t )  P (S 0,…,S t ) = P (S 0 ) P (S 1 |S 0 ) P (S 2 |S 1 ) …………. P (t,q,p) =

13 13 Key problems in NLP  Assign a probability to a sentence Part-of-speech tagging Word-sense disambiguation, Probabilistic Parsing  Predict the next word Speech recognition Hand-writing recognition Augmentative communication for the disabled Impossible to estimate  “I made her duck” Summarization, Machine Translation…..... w1 w2 w3 w4

14 14 Impossible to estimate!  Assuming 10 5 words in Dictionary and average sentence contains 10 words, how may possible worlds (entries in the JPD), would I need to specify  Google language repository (22 Sept. 2006) contained “only”: 95,119,665,584 sentences ~ Not enough to learn the probabilities from frequencies in this dataset (or corpus) Most sentences will not appear or appear only once

15 15 What can we do?  Make a strong simplifying assumption! Sentences are generated by a Markov Chain P(The big red dog barks)= P(The| ) * These probabilities can be assessed in practice!

16 How can we minimally extend Markov Chains? A useful situation to model is the one in which: the reasoning system does not have access to the states but can make observations that give some information about the current state Slide 16

17 Slide 17 Hidden Markov Model P (S 0 ) specifies initial conditions P (S t+1 |S t ) specifies the dynamics P (O t |S t ) specifies the sensor model A Hidden Markov Model (HMM) starts with a Markov chain, and adds a noisy observation about the state at each time step: |domain(S)| = k |domain(O)| = h k probabilities k × k matrix of probabilities k × h matrix of probabilities Markov Assumption on Evidence) P(O t |S 0:t, O 0:t-1 ) = P(O t | S t )

18 Simple Example (We’ll use this as a running example)  Guard stuck in a high-security bunker  Would like to know if it is raining outside  Can only tell by looking at whether his boss comes into the bunker with an umbrella every day Transition model State variables Observable variables Observation model

19 Discussion  Note that the first-order Markov assumption implies that the state variables contain all the information necessary to characterize the probability distribution over the next time slice  Sometime this assumption is only an approximation of reality Whether it rains or not today may depend on the weather on more days than just the previous one  Possible fixes Increase the order of the Markov Chain (e.g., add Rain t-2 as a parent of Rain t ) Add state variables that can compensate for the missing temporal information Such as?

20 Rain Network  We could add Month to each time slice to include season statistics Rain t-1 Umbrella t-1 Rain t Umbrella t Rain t+1 Umbrella t+1 Month t-1 Month t Month t+1

21 Rain Network  Or we could add Temperature, Humidity and Pressure to include meteorological knowledge in the network Rain t-1 Umbrella t-1 Rain t Umbrella t Rain t+1 Umbrella t+1 Temperature t-1 Humidity t-1 Pressure t-1 Temperature t Humidity t Pressure t Temperature t+1 Humidity t+1 Pressure t+1

22 Rain Network  However, adding more state variables may require modelling their temporal dynamics in the network  Trick to get away with it Add sensors that can tell me the value of each new variable at each specific point in time The more reliable a sensor, the less important to include temporal dynamics to get accurate estimates of the corresponding variable Rain t-1 Umbrella t-1 Rain t Umbrella t Temperature t-1 Humidity t-1 Pressure t-1 Temperature t Humidity t Pressure t Barometer t Barometer t-1 Thermometer t-1 Thermometer t

23 Overview  Modelling Evolving Worlds with DBNs  Marcov Chains  Hidden Markov Models  Inference in Temporal Models

24 Inference Tasks in Temporal Models  Filtering (or monitoring): P(X t |e 0:t ) Compute the posterior distribution over the current state given all evidence to date In the rain example, this would mean computing the probability that it rains today given all the observations on umbrella made so far Important if a rational agent needs to make a decision in the current situation  Prediction: P(X t+k | e 0:t ) Compute the posterior distribution over a future state given all evidence to date In the rain example, this would mean computing the probability that it rains in two days given all the observations on umbrella made so far Useful for an agent to evaluate possible courses of actions

25 Inference Tasks in Temporal Models  Smoothing: P(X t-k | e 0:t ) Compute the posterior distribution over a past state given all evidence to date In the rain example, this would mean computing the probability that it rained five days ago given all the observations on umbrella made so far Useful to better estimate what happened by incorporating additional evidence to the evidence available at that time  Most Likely Explanation: argmax X 0:t P(X 0:t | e 0:t ) Given a sequence of observations, find the sequence of states that is most likely to have generated them Useful in many applications, e.g., speech recognition: find the most likely sequence of words given a sequence of sounds

26 Filtering Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2 TRUE 0.5 FALSE 0.5  Idea: recursive approach Compute filtering up to time t-1, and then include the evidence for time t (recursive estimation)‏

27 Filtering  Idea: recursive approach Compute filtering up to time t-1, and then include the evidence for time t (recursive estimation)‏  P(S t | o 0:t ) = P(S t | o 0:t-1,o t ) dividing up the evidence = α P(o t | S t, o 0:t-1 ) P(S t | o 0:t-1 ) WHY? = α P(o t | S t ) P(S t | o 0:t-1 ) WHY? Prediction of current state given evidence up to t-1 Inclusion of new evidence: this is available from..  So we only need to compute P(S t | o 0:t-1 )

28 Filtering  Compute P(S t | o 0:t-1 ) P(S t | o 0:t-1 ) = ∑ S t-1 P(S t, S t-1 |o 0:t-1 ) = ∑ S t-1 P(S t | S t-1, o 0:t-1 ) P( S t-1 | o 0:t-1 ) = = ∑ S t-1 P(S t | S t-1 ) P( S t-1 | o 0:t-1 ) because of..  Putting it all together, we have the desired recursive formulation P(S t | o 0:t ) = α P(o t | S t ) ∑ S t-1 P(S t | S t-1 ) P( S t-1 | o 0:t-1 )  P( S t-1 | o 0:t-1 ) can be seen as a message f 0:t-1 that is propagated forward along the sequence, modified by each transition and updated by each observation Filtering at time t-1 Inclusion of new evidence (sensor model)‏ Propagation to time t why? Filtering at time t-1 Transition model! Product Rule P(A,B) = P(A|B)P(B)

29 Filtering P(S t | o 0:t ) = α P(o t | S t ) ∑ S t-1 P(S t | S t-1 ) P( S t-1 | o 0:t-1 )  Thus, the recursive definition of filtering at time t in terms of filtering at time t-1 can be expressed as a FORWARD procedure f 0:t = α FORWARD (f 0:t-1, o t )  which implements the update described in Filtering at time t-1 Inclusion of new evidence (sensor model) Propagation to time t

30 Analysis of Filtering  Because of the recursive definition in terms for the forward message, when all variables are discrete the time for each update is constant (i.e. independent of t)  The constant depends of course on the size of the state space and the type of temporal model

31 Rain Example Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2  Suppose our security guard came with a prior belief of 0.5 that it rained on day 0, just before the observation sequence started.  Without loss of generality, this can be modeled with a fictitious state R 0 with no associated observation and P(R 0 ) = P(R 1 | o 0:t-1 ) = P(R 1 ) = ∑ r 0 P(R 1 | r 0 ) P(r 0 ) = * * 0.5 =  Day 1: umbella appears (u 1 ). Thus TRUE 0.5 FALSE R t-1 P(R t ) tftf RtRt P(U t ) tftf

32 Rain Example Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2  Updating P(R 1 ) with evidence from for t =1 (umbrella appeared) gives P(R 1 | u 1 ) = α P(u 1 | R 1 ) P(R 1 ) = α = α ~  Day 2: umbella appears (u 2 ). Thus P(R 2 | o 0:t-1 ) = P(R 2 | u 1 ) = ∑ r 1 P(R 2 | r 1 ) P(r 1 | u 1 ) = = * * ~ TRUE 0.5 FALSE R t-1 P(R t ) tftf RtRt P(U t ) tftf Vector dot product

33 Rain Example Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2  Updating this with evidence from for t =2 (umbrella appeared) gives P(R 2 | u 1, u 2 ) = α P(u 2 | R 2 ) P(R 2 | u 1 ) = α = α ~  Intuitively, the probability of rain increases, because the umbrella appears twice in a row TRUE 0.5 FALSE

34 Prediction: P(S t+k+1 | o 0:t )  Can be seen as filtering without addition of new evidence  In fact, filtering already contains a one-step prediction P(S t | o 0:t ) = α P(o t | S t ) ∑ s t-1 P(S t | s t-1 ) P( s t-1 | e 0:t-1 ) Filtering at time t-1 Inclusion of new evidence (sensor model)‏ Propagation to time t  We just need to show how to recursively predict the state at time t+k +1 from a prediction for state t + k P(S t+k+1 | o 0:t ) = ∑ s t+k P(S t+k+1, s t+k |o 0:t ) = ∑ s t+k P(S t+k+1 | s t+k, o 0:t ) P( s t+k | o 0:t ) = = ∑ s t+k P(S t+k+1 | s t+k ) P( s t+k | o 0:t )  Let‘s continue with the rain example and compute the probability of Rain on day four after having seen the umbrella in day one and two: P(R 4 | u 1, u 2 ) Prediction for state t+ k Transition model

35 Rain Example Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2  Prediction from day 2 to day 3 P(S 3 | o 1:2 ) = ∑ s 2 P(S 3 | s 2 ) P( s 2 | o 1:2 ) = ∑ r 2 P(R 3 | r 2 ) P( r 2 | u 1 u 2 ) = = * *0.117 = + = Rain 3 Umbrella  Prediction from day 3 to day 4 P(S 4 | o 1:2 ) = ∑ s 3 P(S 4 | s 3 ) P( s 3 | o 1:2 ) = ∑ r 3 P(R 4 | r 3 ) P( r 3 | u 1 u 2 ) = = * *0.347= + = Rain 3 Umbrella R t-1 P(R t ) tftf RtRt P(U t ) tftf

36 Rain Example  Intuitively, the probability that it will rain decreases for each successive day, as the influence of the observations from the first two days decays  What happens if I try to predict further and further into the future? It can be shown that the predicted distribution converges to the stationary distribution of the Markov process defined by the transition model ( for the rain example)‏( When the convergence happens, I have basically lost all the information provided by the existing observations, and I can’t generate any meaningful prediction on states from this point on The time necessary to reach this point is called mixing time. The more uncertainty there is in the transition model, the shorter the mixing time will be Basically, the more uncertainty there is in what happens at t+1 given that I know what happens in t, the faster the information that I gain from evidence on the state at t dissipates with time

37 Slide 37 Another Example: Localization for “Pushed around” Robot Localization (where am I?) is a fundamental problem in robotics Suppose a robot is in a circular corridor with 16 locations There are four doors at positions: 2, 4, 7, 11 The Robot initially doesn’t know where it is The Robot is pushed around. After a push it can stay in the same location, move left or right. The Robot has a noisy sensor telling whether it is in front of a door

38 This scenario can be represented as… Example Stochastic Dynamics: when pushed, it stays in the same location p=0.2, moves left or right with equal probability P(Loc t + 1 | Loc t ) P(Loc 0 ) /16 for each location in the corridor

39 Slide 39 This scenario can be represented as… Example of Noisy sensor telling whether it is in front of a door. If it is in front of a door P(O t = T) =.8 If not in front of a door P(O t = T) =.1 P(O t | Loc t ) Loc t P(O t = t) …… 16

40 Slide 40 This scenario can be represented as… Example of Noisy sensor telling whether it is in front of a door. If it is in front of a door P(O t = T) =.8 If not in front of a door P(O t = T) =.1 P(O t | Loc t ) Loc t P(O t = t) …… 16

41 Useful inference in this problem Localization: Robot starts at an unknown location and it is pushed around t times. It wants to determine where it is This is an instance of Filtering: compute the posterior distribution over the current state given all evidence to date P(S t | o 0:t ) Slide 41 P(Loc t | o 0, o 1,…,o t )

42 Slide 42 More complex Example: Robot Localization Suppose a robot wants to determine its location based on its actions and its sensor readings Three actions: goRight, goLeft, Stay This can be represented by an augmented HMM

43 Slide 43 More complex Example: Robot Localization Suppose a robot wants to determine its location based on its actions and its sensor readings Three actions: goRight, goLeft, Stay This can be represented by an augmented HMM

44 Slide 44 Robot Localization Sensor and Dynamics Model Sample Sensor Model (assume same as for pushed around) Sample Stochastic Dynamics:P(Loc t + 1 | Action t, Loc t ) P(Loc t + 1 = L | Action t = goRight, Loc t = L) = 0.1 P(Loc t + 1 = L+1 | Action t = goRight, Loc t = L) = 0.8 P(Loc t + 1 = L + 2 | Action t = goRight, Loc t = L) = P(Loc t + 1 = L’ | Action t = goRight, Loc t = L) = for all other locations L’ All location arithmetic is modulo 16 The action goLeft works the same but to the left The action Stay is deterministic

45 Dynamics Model More Details Sample Stochastic Dynamics:P(Loc t + 1 | Action, Loc t ) P(Loc t + 1 = L | Action t = goRight, Loc t = L) = 0.1 P(Loc t + 1 = L+1 | Action t = goRight, Loc t = L) = 0.8 P(Loc t + 1 = L + 2 | Action t = goRight, Loc t = L) = P(Loc t + 1 = L’ | Action t = goRight, Loc t = L) = for all other locations L’

46 Slide 46 Example Inference What is the probability distribution for the robot location at time 2, given the following sequence of observations and actions

47 Slide 47 Robot Localization additional sensor Additional Light Sensor: there is light coming through an opening at location 10. The light sensor detects if there is light or not at a given location. What do we need to specify?

48 Slide 48 Robot Localization additional sensor Additional Light Sensor: there is light coming through an opening at location 10 P (L t | Loc t ) Info from the two sensors is combined :“Sensor Fusion” Do we need to do anything special to make this sensor fusion happen? P (L t =t) P (L t =f)

49 Slide 49  The Robot starts at an unknown location and must determine where it is  The model appears to be too ambiguous Sensors are too noisy Dynamics are too stochastic to infer anything But inference actually works pretty well. check: It uses a generalized form of filtering (not just a sequence of observations, but a sequence of observation pairs (from the two sensors) + the actions

50 HMMs have many other applications….  Natural Language Processing: e.g., Speech Recognition States: phoneme \ word Observations: acoustic signal \ phoneme  Bioinformatics: Gene Finding States: coding / non-coding region Observations: DNA Sequences For these problems the critical inference is: find the most likely sequence of states given a sequence of observations

51 Most Likely Sequence  Suppose that in the rain example we have the following umbrella observation sequence [true, true, false, true, true]  Is it the perfect reflection on the rain situation? [rain, rain, no-rain, rain, rain]  Or perhaps it did rain on the third day but the boss forgot to bring the umbrella?  If it did not rain on day 3, perhaps it also did not rain on day 4, but the boss brought the umbrella just in case  2 5 possible sequences of states

52 Most Likely Sequence (Explanation)  Most Likely Sequence: argmax x 0:t P(X 0:t | e 0:t )  General idea: search in a graph whose nodes are the possible states at each time step

53 Most Likely Sequence ( assuming a single state )  Suppose we want to find the most likely path to state S t+1.  Because of the Markov assumption, this can be found by finding The most likely path to each state s t at step t the state s t, at step t that maximizes the path to S t+1.  Recursive relationship between most likely path to state S t+1 and most likely path to state S t, which we can express as  max s1,...st P(s 1,.... s t,S t+1 |o 1:t+1 ) = P(o t+1 |S t+1 ) max st [(P(S t+1 |s t ) max s 1,...s t-1 P(s 1,.... s t-1,s t |o 1:t )] See derivation in next slide if you need

54 Most Likely Sequence max s 1,...s t P(s 1,.... s t,S t+1 |o 1:t+1 )= max s 1,...s t P(s 1,.... s t,S t+1 |o 1:t, o t+1 )= = max s 1,...s t αP(o t+1 |o 1:t, s 1,.... s t,S t+1 ) P(s 1,.... s t,S t+1 |o 1:t )= = max s 1,...s t αP(o t+1 |S t+1 ) P(s 1,.... s t,S t+1 |o 1:t )= = max s 1,...s t αP(o t+1 |S t+1 ) P(S t+1 | s 1,.... s t, e 1:t )P(s 1,.... s t, |o 1:t )= = max s 1,...s t P(o t+1 |S t+1 ) P(S t+1 |s t ) P(s 1,.... s t-1,s t |o 1:t ) P(o t+1 |S t+1 ) max s t (P(S t+1 |s t ) max s 1,...s t-1 P(s 1,.... s t-1,s t |o 1:t ) Bayes Rule Markov Assumption Product Rule Markov Assumption

55 Most Likely Sequence  Identical to filtering, P(S t | o 0:t ) = α P(o t | S t ) ∑ s t-1 P(S t | s t-1 ) P( s t-1 | o 0:t-1 )‏ max s 1,...s t P(s 1,.... s t,S t+1 |o 1:t+1 ) = P(o t+1 |S t+1 ) max s t (P(S t+1 |s t ) max s 1,...s t-1 P(s 1,.... s t-1,s t |o 1:t )  f 0:t-1 = P(S t-1 |o 0:t-1 ) is replaced by m 1:t = max s 1,...s t-1 P(s 1,.... s t-1,S t |o 1:t ) (*)‏  the summation in the filtering equations is replaced by maximization in the maximization equations Recursive call

56 Viterbi Algorithm  Computes the most likely sequence to S t+1 by running forward along the sequence computing the m message at each time step, using (*) in the previous slide in the end it has the most likely sequence to each of the final states, and we can pick the most likely See derivation of the first 3 m 1:I in the next couple of slides

57 Rain Example max s 1,...s t P(s 1,.... s t,S t+1 |o 1:t+1 ) = P(o t+1 |S t+1 ) max s t [(P(S t+1 |s t ) m 1:t ] m 1:t = max s 1,...s t-1 P(s 1,.... s t-1,S t |o 1:t ) -m 1:1 is just P(R 1 |u) = -m 1:2 = P(u 2 |R 2 )

58 Rain Esample m 1:3 = P(⌐ u 3 |R 3 )

59 Viterbi Algorithm  Time complexity, O(t)  The space is also linear in t, because the algorithm needs to save the pointers indicating the best sequence to each state

60 DBN and HMM Every HMM is a DBN with only one state variable Every DBN can be turned into a HMM –Just create a mega-variable with values that represent all possible combinations of values for the variables in the DBN

61 DBN and HMM Student learning example in DBN format –Knows-Add (boolean), Knows-Sub (boolean), Morale {low, high} Knows-Sub t-1 Knows-Add t-1 Morale t-1 Knows-Sub t Knows-Add t Morale t Equivalent HMM: only one variable –StudentState with 8 possible states –representing all possible combinations of knowing/not knowing addition and subtraction and having a high/low morale

62 DBN and HMM If they are equivalent, why bother making a distinction? Main difference is that decomposing the state into variables allows one to explicitly represent dependencies among them –Sparse dependencies => Exponentially fewer parameters Suppose we have Bnet with –20 Boolean state variables –Each with 3 parents in previous time slice –Transition model has ……………. Equivalent HMM –1 state variable with ……. states –……………. entries in the transition matrix –Really bad space and time complexity –Really bad having to specify so many parameters

63 DBN and HMM If they are equivalent, why bother making a distinction? Main difference is that decomposing the state into variables allows one to explicitly represent dependencies among them –Sparse dependencies => Exponentially fewer parameters Suppose we have Bnet with –20 Boolean state variables –Each with 3 parents in previous time slice –Transition model has 20 x 2 3 = 160 probabilities Equivalent HMM –1 state variable with 2 20 states –2 20 x 2 20 = 2 40 ~ entries in the transition matrix –Really bad space and time complexity –Really bad having to specify so many parameters

64 DBN and HMM Even when the number of states in the equivalent HMM is not that large (e.g. student learning model)… …specifying the numbers in the transition matrix can be less intuitive than specifying the more modular conditional probabilities in the Bnets What is the probability that the student goes –From knowing addition, not knowing subtraction, having low morale –To knowing addition, not knowing subtraction and having high morale?

65 Exact Inference in DBN Since DBN are Bayesian networks, we can use any of the existing algorithms for exact inference Given a sequence of observations –Construct the full Bayesian networks by creating as many time slices as they are needed to fit the whole sequence –This technique is called unrolling But this is not such a great idea, as it would requite O(t) space to unroll the network Besides if we update the network anew for each new piece of evidence, time complexity would also be O(t)

66 Exact Inference in DBN Better to use the recursive definition of filtering that we have seen earlier P(X t | e 0:t ) = α P(e t | X t ) ∑ x t-1 P(X t | x t-1 ) P( x t-1 | e 0:t-1 ) Filtering at time t-1 Inclusion of new evidence (sensor model)‏ Propagation to time t Filtering step: basically sums out the state variables in the previous time slice to get the distribution in the current time slice This is exactly variable elimination run with the variables in temporal order X t is now a set of state variables, and e is a set of observations

67 Rollup Filtering This algorithm (rollup filtering) keeps only two time slices in memory at any given time –Start with Slice 0 –Add slice 1 and sum out slice 0 –Add slice 2 and sum out slice 1 –Add slice n and sum out slice n -1 Rain 0 Rain 1 Umbrella 1 Rain 2 Umbrella 2 TRUE 0.5 FALSE

68 Rollup Filtering Rollup filtering requires “constant” (independent of t) space and time for each update Unfortunately the “constant” is, in most cases, exponential in the number of state variables –As variable elimination proceeds, the factors grow to include all state variables that have parents in the previous time slice Thus, while in principle DBN allow to represent temporal processes with many sparsely connected variables, in practice we cannot reason efficiently and exactly about those processes Need to use sampling algorithms! –The most promising is particle filtering

69 Why not Likelihood Weighting ?  LW Avoids the inefficiency of rejection sampling by generating only events that are consistent with evidence e Fixes the values for the evidence variables E Samples only the remaining variables Z, i.e. the query X and hidden variables Y  Still needs to account for the influence of the given evidence on the probability of the samples  LW approximates the posterior distribution by weighing the samples by the probability they afford to the evidence

70 Example: P(Rain|sprinkler, wet-grass)‏ TFTF P(S|C)‏C Cloudy Sprinkler 0.5 P(C)‏ Rain Wet Grass TFTF C P(R|C)‏ 0.1FF 0.9TF FT 0.99TT P(W|S,R)‏RS Random => 0.4 Sample=> cloudy Sprinkler is fixed No sampling, but adjust weight w 2 = w 1 * P(sprinkler|cloudy)‏ Random => 0.4 Sample=> rain w 1 = 1 w 2 = w 1* 0.1 = 0.1 Wet Grass is fixed No sampling, but adjust weight w 3 = w 2 * P(wet-grass|sprinkler, rain)‏ w 3 = w 2* 0.99 = 0.099

71 Why not Likelihood Weighting Two problems 1)Standard algorithm runs each sample in turn, all the way through the network time and space requirements grow with t!

72 Why not Likelihood Weighting 2) Does not work well when evidence comes “late” in the network topology Samples are largely independent on the evidence True in HMM, where often state variables don’t have evidence as parents In the Rain network, I could observe umbrella every day, and still get samples that are sequences of sunny days Number of samples necessary to get good approximations increases exponentially with t!

73 Why not Likelihood Weighting

74 Particle Filtering Designed to fix both problems 1.Run all N samples together through the network, one slice at a time Form of filtering, where the N samples are the forward message Essentially use the sample themselves as an approximation of the current state distribution No need to unroll the network, all that is needed is current and next slice => “constant” update per time step STEP 0: Generate a population on N initial-state samples by sampling from initial state P(X t ) N = 10

75 Particle Filtering  STEP 1: Propagate each sample for x t forward by sampling the next state value x t+1 based on P(X t+1 |x t ) R t-1 P(R t ) tftf

76 Particle Filtering  STEP 2: Weight each sample by the likelihood it assigns to the evidence E.g. assume we observe not umbrella at t+1 RtRt P(U t ) tftf

77 Particle Filtering Now we need to take care of problem #2: generate samples that are a better simulation of reality  IDEA: Focus the set of samples on the high probability regions of the state space Throw away samples with very low weight according to evidence Replicate those with high weight, to obtain a sample closer to reality STEP 3: Create a new sample from the population at X t+1, i.e.  resample the population so that the probability that each sample is selected is proportional to its weight

78 Particle Filtering STEP 3: Create a new sample from the population at X t+1, i.e.  resample the population so that the probability that each sample is selected is proportional to its weight  Start the Particle Filtering cycle again from the new sample

79 Particle Filtering

80 N = 10

81 Particle Filtering

82

83

84  In practice, approximation error of particle filtering remains bounded overtime  It is also possible to prove that the approximation maintains bounded error with high probability (with specific assumptions) Is PF Efficient?

85 Representational Dimensions  Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Value Iteration Variable Elimination Belief Nets Decision Nets Markov Processes Static Sequential Representation Reasoning Technique Variable Elimination Approximate Temporal Inference This concludes the module on answering queries in stochastic environments

86 Explain what is a dynamic Bayesian network and when it is a useful R&R tool. Indentify real world scenarions that can be modeled via DBN State the stationary process assumption, Markov assumption and Markov Assumption on Evidence. Explain why they are made. Define markov Chains, and explain when they are useful. Define Hidden Markov Models, and explain when they are useful. Explain/write in pseudocode/implement/trace/debug algorithms for temporal inference: filtering, prediction, Viterbi. Explain the relation between DN and HHM. Translate a problem represented as a HMM into a corresponding DN (and vice versa). Compare the pros and cons of the the two representations. Explain/implement roll-up filtering. Explain Particle Filtering for discrete networks 86 Learning Goals For Probability andTime


Download ppt "Computer Science CPSC 502 Lecture 11 Probability and Time (Ch. 6.5)"

Similar presentations


Ads by Google