Download presentation

Presentation is loading. Please wait.

Published byLexus Barlet Modified over 2 years ago

1
ST3236: Stochastic Process Tutorial 3 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 4

2
Question 1 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix and initial distributions, p 0 = P(X 0 = 0) = 0.3, p 1 = P(X 0 = 1) = 0.4 and p 2 = P(X 0 = 2) = 0.3. Determine P(X 0 = 0, X 1 = 1, X 2 = 2) and draw the state- diagram with transition probability.

3
Question 1 P(X 0 = 0,X 1 = 1,X 2 = 2) = P(X 0 = 0)P(X 1 = 1 | X 0 = 0)P(X 2 = 2 | X 0 = 0,X 1 = 1) = P(X 0 = 0)P(X 1 = 1 | X 0 = 0)P(X 2 = 2 | X 1 = 1) = p 0 x p 01 x p 12 = 0.3 x 0.2 x 0 = 0.

5
Question 2 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix Determine the conditional probabilities P(X 2 = 1,X 3 = 1|X 1 = 0) and P(X 1 = 1,X 2 = 1|X 0 = 0).

6
Question 2 P(X 2 = 1, X 3 = 1 | X 1 = 0) = P(X 2 = 1 | X 1 = 1)P(X 3 = 1 | X 1 = 0, X 2 = 1) = P(X 2 = 1 | X 1 = 0)P(X 3 = 1 | X 2 = 1) = p 01 x p 11 = 0.2 x 0.6 = 0.12 Similarly (or by stationarity), P(X 1 = 1, X 2 = 1 | X 0 = 0) = 0.12 In general, P(X n+1 = 1, X n+2 = 1 | X n = 0) = 0.12 for any n. That is, it doesn’t matter when you start.

7
Question 3 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix If we know that the process starts in state X 0 = 1, determine probability P(X 0 = 1,X 1 = 0,X 2 = 2).

8
Question 3 P(X 0 = 1,X 1 = 0,X 2 = 2) = P(X 0 = 1)P(X 1 = 0| X 0 = 1)P(X 2 = 2 | X 0 = 1,X1 = 0) = P(X 0 = 1)P(X 1 = 0| X 0 = 1)P(X 2 = 2 | X 1 = 0) = p 1 x p 10 x p 02 = 1 x 0.3 x 0.1 = 0.03

9
Question 4 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix

10
Question 4a Compute the two-step transition matrix P(2). Note: Observe that the rows must always sum to one for all transition matrices.

11
Question 4b What is P(X 3 = 1|X 1 = 0)? P(X 3 = 1|X 1 = 0) = 0.13 In general, P(X n+2 = 1 | X n = 0) = 0.13 for any n.

12
Question 4c What is P(X 3 = 1|X 0 = 0)? Note that: Thus, P(X 3 = 1|X 0 = 0) = 0.16 In general, P(X n+3 = 1 | X n = 0) = 0.16 for any n.

13
Question 5 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix It is known that the process starts in state X 0 = 1, determine probability P(X 2 = 2).

14
Question 5 Note that: P(X 2 = 2) = P(X 0 = 0) x P(X 2 = 2 | X 0 = 0) +P(X 0 = 1) x P(X 2 = 2 | X 0 = 1) +P(X 0 = 2) x P(X 2 = 2 | X 0 = 2) = p 0 p 02 + p 1 p 12 + p 2 p 22 = 1 x p 12 = 0.35

15
Question 6 Consider a sequence of items from a production process, with each item being graded as good or defective. Suppose that a good item is followed by another good item with probability and by a defective item with probability 1- . Similarly, a defective item is followed by another defective item with probability and by a good item with probability 1- . Specify the transition probability matrix. If the first item is good, what is the probability that the first defective item to appear is the fifth item?

16
Question 6 Let X n be the grade of the nth product. P(X n+1 = g | X n = g) = , P(X n+1 = d | X n = g) = 1 - P(X n+1 = d | X n = d) = , P(X n+1 = g | X n = d) = 1 - Thus, the transition probability matrix is

17
Question 6 The probability is, P(X 5 = d,X 4 = g,X 3 = g,X 2 = g | X 1 = g) = P(X 2 = g | X 1 = g) x P(X 3 = g | X 2 = g) x P(X 4 = g | X 3 = g) x P(X 5 = d | X 4 = g) (why?) = p gg x p gg x p gg x p gd = 3 (1 - )

18
Question 7 The random variables 1, 2,... are independent and with common probability mass function Set X 0 = 0 and let X n = max { 1, 2,... }. Determine the transition probability matrix for the MC {X n }. Draw the state-diagram associated with transition probability

19
Question 7 Observe: X 0 = 0, X 1 = max {X 0, 1 }, X 2 = max {X 1, 2 }, … X n = max {X n-1, n } Hence, X n recursively compares the previous maximum and the current input to obtain the new maximum.

20
Question 7 The state space is S = {0, 1, 2, 3} P(X n+1 = 0 | X n = 0) = P( n+1 = 0)= 0.1 P(X n+1 = 1 | X n = 0) = P( n+1 = 1)= 0.3 P(X n+1 = 2 | X n = 0) = 0.2 P(X n+1 = 3 | X n = 0) = 0.4 P(X n+1 = 1 | X n = 1) = P( n+1 = 0) + P( n+1 = 1) = 0.1 + 0.3 = 0.4 P(X n+1 = 2 | X n = 1) = 0.2 … P(X n+1 = 2 | X n = 2) = 0.1 + 0.3 + 0.2 = 0.6 … P(X n+1 = 3 | X n = 3) = 0.1 + 0.3 + 0.2 + 0.4 = 1 … P(X n+1 = j | X n = i) = 0 if j < i. (Cannot Happen!)

21
Question 7 The transition probability matrix is

Similar presentations

OK

Tutorial 8 Markov Chains. 2 Consider a sequence of random variables X 0, X 1, …, and the set of possible values of these random variables is {0, 1,

Tutorial 8 Markov Chains. 2 Consider a sequence of random variables X 0, X 1, …, and the set of possible values of these random variables is {0, 1,

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Download ppt on respiration in humans Free ppt on product management Ppt on life of guru gobind singh ji Make appt online Ppt on carbon monoxide poisoning Ppt on story of taj mahal A ppt on srinivasa ramanujan Ppt on computer assisted language learning Ppt on duty roster templates Ppt on effective speaking skills