Download presentation

Presentation is loading. Please wait.

Published byLexus Barlet Modified about 1 year ago

1
ST3236: Stochastic Process Tutorial 3 TA: Mar Choong Hock Exercises: 4

2
Question 1 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix and initial distributions, p 0 = P(X 0 = 0) = 0.3, p 1 = P(X 0 = 1) = 0.4 and p 2 = P(X 0 = 2) = 0.3. Determine P(X 0 = 0, X 1 = 1, X 2 = 2) and draw the state- diagram with transition probability.

3
Question 1 P(X 0 = 0,X 1 = 1,X 2 = 2) = P(X 0 = 0)P(X 1 = 1 | X 0 = 0)P(X 2 = 2 | X 0 = 0,X 1 = 1) = P(X 0 = 0)P(X 1 = 1 | X 0 = 0)P(X 2 = 2 | X 1 = 1) = p 0 x p 01 x p 12 = 0.3 x 0.2 x 0 = 0.

4

5
Question 2 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix Determine the conditional probabilities P(X 2 = 1,X 3 = 1|X 1 = 0) and P(X 1 = 1,X 2 = 1|X 0 = 0).

6
Question 2 P(X 2 = 1, X 3 = 1 | X 1 = 0) = P(X 2 = 1 | X 1 = 1)P(X 3 = 1 | X 1 = 0, X 2 = 1) = P(X 2 = 1 | X 1 = 0)P(X 3 = 1 | X 2 = 1) = p 01 x p 11 = 0.2 x 0.6 = 0.12 Similarly (or by stationarity), P(X 1 = 1, X 2 = 1 | X 0 = 0) = 0.12 In general, P(X n+1 = 1, X n+2 = 1 | X n = 0) = 0.12 for any n. That is, it doesn’t matter when you start.

7
Question 3 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix If we know that the process starts in state X 0 = 1, determine probability P(X 0 = 1,X 1 = 0,X 2 = 2).

8
Question 3 P(X 0 = 1,X 1 = 0,X 2 = 2) = P(X 0 = 1)P(X 1 = 0| X 0 = 1)P(X 2 = 2 | X 0 = 1,X1 = 0) = P(X 0 = 1)P(X 1 = 0| X 0 = 1)P(X 2 = 2 | X 1 = 0) = p 1 x p 10 x p 02 = 1 x 0.3 x 0.1 = 0.03

9
Question 4 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix

10
Question 4a Compute the two-step transition matrix P(2). Note: Observe that the rows must always sum to one for all transition matrices.

11
Question 4b What is P(X 3 = 1|X 1 = 0)? P(X 3 = 1|X 1 = 0) = 0.13 In general, P(X n+2 = 1 | X n = 0) = 0.13 for any n.

12
Question 4c What is P(X 3 = 1|X 0 = 0)? Note that: Thus, P(X 3 = 1|X 0 = 0) = 0.16 In general, P(X n+3 = 1 | X n = 0) = 0.16 for any n.

13
Question 5 A markov chain X 0,X 1,… on state 0, 1, 2 has the transition probability matrix It is known that the process starts in state X 0 = 1, determine probability P(X 2 = 2).

14
Question 5 Note that: P(X 2 = 2) = P(X 0 = 0) x P(X 2 = 2 | X 0 = 0) +P(X 0 = 1) x P(X 2 = 2 | X 0 = 1) +P(X 0 = 2) x P(X 2 = 2 | X 0 = 2) = p 0 p 02 + p 1 p 12 + p 2 p 22 = 1 x p 12 = 0.35

15
Question 6 Consider a sequence of items from a production process, with each item being graded as good or defective. Suppose that a good item is followed by another good item with probability and by a defective item with probability 1- . Similarly, a defective item is followed by another defective item with probability and by a good item with probability 1- . Specify the transition probability matrix. If the first item is good, what is the probability that the first defective item to appear is the fifth item?

16
Question 6 Let X n be the grade of the nth product. P(X n+1 = g | X n = g) = , P(X n+1 = d | X n = g) = 1 - P(X n+1 = d | X n = d) = , P(X n+1 = g | X n = d) = 1 - Thus, the transition probability matrix is

17
Question 6 The probability is, P(X 5 = d,X 4 = g,X 3 = g,X 2 = g | X 1 = g) = P(X 2 = g | X 1 = g) x P(X 3 = g | X 2 = g) x P(X 4 = g | X 3 = g) x P(X 5 = d | X 4 = g) (why?) = p gg x p gg x p gg x p gd = 3 (1 - )

18
Question 7 The random variables 1, 2,... are independent and with common probability mass function Set X 0 = 0 and let X n = max { 1, 2,... }. Determine the transition probability matrix for the MC {X n }. Draw the state-diagram associated with transition probability

19
Question 7 Observe: X 0 = 0, X 1 = max {X 0, 1 }, X 2 = max {X 1, 2 }, … X n = max {X n-1, n } Hence, X n recursively compares the previous maximum and the current input to obtain the new maximum.

20
Question 7 The state space is S = {0, 1, 2, 3} P(X n+1 = 0 | X n = 0) = P( n+1 = 0)= 0.1 P(X n+1 = 1 | X n = 0) = P( n+1 = 1)= 0.3 P(X n+1 = 2 | X n = 0) = 0.2 P(X n+1 = 3 | X n = 0) = 0.4 P(X n+1 = 1 | X n = 1) = P( n+1 = 0) + P( n+1 = 1) = = 0.4 P(X n+1 = 2 | X n = 1) = 0.2 … P(X n+1 = 2 | X n = 2) = = 0.6 … P(X n+1 = 3 | X n = 3) = = 1 … P(X n+1 = j | X n = i) = 0 if j < i. (Cannot Happen!)

21
Question 7 The transition probability matrix is

22

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google