Download presentation

Presentation is loading. Please wait.

Published byIrma Clark Modified over 2 years ago

1
ST3236: Stochastic Process Tutorial 4 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 5

2
Question 1 An urn initially contains a single red and single green ball. A ball is drawn at random, removed and replaced by a ball of the opposite color and this procedure repeats so that there are always two balls in the urn. Let X n be the number of red balls in the urn after n draws, with X 0 = 1. Specify the transitions probabilities for MC {X}.

3
Question 1 Case (X n =0): Both balls are green. One ball will certainly be replaced with red after a ball is drawn. P(X n+1 = 1 | X n = 0) = 1, P(X n+1 = 0 | X n = 0) = 0, P(X n+1 = 2 | X n = 0) = 0 Case (X n =2): Both balls are red. One ball will certainly be replaced with green after a ball is drawn. P(X n+1 = 1 | X n = 2) = 1, P(X n+1 = 0 | X n = 2) = 0, P(X n+1 = 2 | X n = 2) = 0

4
Question 1 Case (X n = 1): One ball is red. Outcome dependent on the colour of the ball drawn. Note: P(Green is drawn|X n =1) = P (Red is drawn|X n =1) = 0.5 P(X n+1 = 0 | X n = 1) = P(Red is drawn|X n = 1) = 0.5 P(X n+1 = 2 | X n = 1) = P(Green is drawn|X n = 1) = 0.5 P(X n+1 = 1 | X n = 1) = 0

5
Question 1 The transition matrix is given as follows:

6
Question 2 Find the mean time to reach state 3 starting from state 0 for the MC whose transition probability matrix is

7
Question 2 Let T = min{n : X n = 3} and v i = E(T | X 0 = i). The mean time to reach state 3 starting from state 0 is v 0. We apply first step analysis.

8
Question 2 Therefore, v 0 = 1 + 0.4v 0 + 0.3v 1 + 0.2v 2 + 0.1v 3 v 1 = 1 + 0v 0 + 0.7v 1 + 0.2v 2 + 0.1v 3 v 2 = 1 + 0v 0 + 0v 1 + 0.9v 2 + 0.1v 3 v 3 = 0 Solving the equations, we have v 0 = 10.

9
Question 3 Consider the MC with transition probability matrix (a) Starting in state 1, determine the probability that the MC ends in state 0 (b) Determine the mean time to absorption.

10
Question 3a Let T = min{n : X n = 0 and X n = 2}, u i = P(X T = 0|X 0 = i) and v i = E(T|X 0 = i). u 0 = 1 u 1 = 0.1u 0 + 0.6u 1 + 0.3u 2 u 2 = 0 we have u 1 = 0.25.

11
Question 3b Let T = min{n : X n = 0 and X n = 2}, u i = P(X T = 0|X 0 = i) and v i = E(T|X 0 = i). v 0 = 0 v 1 = 1 + 0.1v 0 + 0.6v 1 + 0.3v 2 v 2 = 0 we have v 1 = 2.5.

12
Question 4 Consider the MC with transition probability matrix (a) Starting in state 1, determine the probability that the MC ends in state 0 (b) Determine the mean time to absorption.

13
Question 4a Let T = min{n : X n = 0 and X n = 2}, u i = P(X T = 0|X 0 = i) and v i = E(T|X 0 = i). u 0 = 1 u 1 = 0.1u 0 + 0.6u 1 + 0.1u 2 + 0.2u 3 u 2 = 0.2u 0 + 0.3u 1 + 0.4u 2 + 0.1u 3 u 3 = 0 we have, u 1 = 0.3810, u 2 = 0.5238 Starting in state 1, the probability that the MC ends in state 0 is u 1 = 0.3810

14
Question 4b Let T = min {n : X n = 0 and X n = 2}, u i = P(X T = 0|X 0 = i) and v i = E(T|X 0 = i). v 0 = 0 v 1 = 1 + 0.1v 0 + 0.6v 1 + 0.1v 2 + 0.2v 3 v 2 = 1 + 0.2v 0 + 0.3v 1 + 0.4v 2 + 0.1v 3 v 3 = 0 we have v 1 = v 2 = 3.33.

15
Question 5 A coin is tossed repeatedly until two successive heads appear. Find the mean number of tosses required. [Hint: Let X n be the cumulative number of successive heads. Then the state space is 0,1,2 before stop]

16
Question 5 – Method 1 Let Y n be the outcome {H, T} of each toss and (Y n-1, Y n ) denotes the sample point for the sucessive tosses. There are 4 possible sample points.

18
Question 5 – Method 1 The transition probability matrix is

19
Question 5 – Method 1 Let v i denote the mean time to each state 3 starting from state i. By the first step analysis, we have the following equations: v 0 = 1 + 0.5v 0 + 0.5v 1 v 1 = 1 + 0.5v 2 + 0.5v 3 v 2 = 1 + 0.5v 0 + 0.5v 1 v 3 = 0 Therefore, v 0 = 6 (v 1 = 4, v 2 = 6)

20
Question 5 – Method 2 Let X n be the cumulative number of successive heads. The 3-state state space now is {0,1,2}. Example: n01234 YnYn THTHH XnXn 01012

21
Question 5 – Method 2 Case (X n = 0) :Previous two tosses are tails Or a head followed by tail. Note: P(Head) = P(Tail) = 0.5 P(X n+1 = 1 | X n = 0) = P(Head) = 0.5, P(X n+1 = 0 | X n = 0) = P(Tail) = 0.5, P(X n+1 = 2 | X n = 0) = 0 Case (X n = 1): A tail is followed by a head P(X n+1 = 2 | X n = 1) = P(Head) = 0.5 P(X n+1 = 0 | X n = 1) = P(Tail) = 0.5 P(X n+1 = 1 | X n = 1) = 0

22
Question 5 – Method 2 Case (X n = 2): Previous two tosses are heads. We make this state absorbing. P(X n+1 = 1 | X n = 2) = 0 P(X n+1 = 2 | X n = 2) = 1, P(X n+1 = 0 | X n = 2) = 0

23
Question 5 – Method 2 The transition probability matrix is

24
Question 5 – Method 2 Let v i denote the mean time to each state 2 starting from state i. By the first step analysis, we have the following equations: v 0 = 1 + 0.5v 0 + 0.5v 1 v 1 = 1 + 0.5v 0 + 0.5v 2 v 2 = 0 Thus, we have, v 0 = 6, v 1 = 4

Similar presentations

Presentation is loading. Please wait....

OK

ST3236: Stochastic Process Tutorial 6

ST3236: Stochastic Process Tutorial 6

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on media advertising Ppt on french revolution Ppt on time management for students Ppt on standing order template Ppt on weapons of mass destruction bush Converter pub to ppt online templates Ppt on acute and chronic diseases Ppt on airline industry in india Ppt on english language learning Ppt on turbo c compiler