1. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Chapter 4. Discrete Probability Distributions Section 4.11: Markov Chains Jiaping Wang Department of Mathematical Science 03/18/2013, Monday

2. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline Introduction Formal Definition and Formulas

3. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 1. Introduction

4. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Consider a system that can be in any of a finite number of states. Assume that the system moves from state to state according to some prescribed probability law. For example, the states represent whether the economy is in a bull market, a bear market or a recession during a given week. From week to week, the states will transit each other, eg., from bull market to bear market, or from recession to bear market with some probabilities. From the figure, if we denote X i as the state in i-th week, S1=Bull market, S2=Bear market, S3=Recession, then we can have P(X i =S1|X i-1 =S2)=0.15=p 21, P(X i =S2|X i-1 =S2)=0.80=p 22, P(X i =S2|X i-1 =S1)=0.075=p 12, P(X i =S1|X i-1 =S1)=0.90 =p 11, P(X i =S3|X i-1 =S1)=0.025 =p 13, P(X i =S3|X i-1 =S2)=0.05 =p 23, P(X i =S1|X i-1 =S3)=0.25 =p 31, P(X i =S2|X i-1 =S3)=0.25 =p 32, P(X i =S3|X i-1 =S3)=0.50 =p 33, Consider a system that can be in any of a finite number of states. Assume that the system moves from state to state according to some prescribed probability law. For example, the states represent whether the economy is in a bull market, a bear market or a recession during a given week. From week to week, the states will transit each other, eg., from bull market to bear market, or from recession to bear market with some probabilities. From the figure, if we denote X i as the state in i-th week, S1=Bull market, S2=Bear market, S3=Recession, then we can have P(X i =S1|X i-1 =S2)=0.15=p 21, P(X i =S2|X i-1 =S2)=0.80=p 22, P(X i =S2|X i-1 =S1)=0.075=p 12, P(X i =S1|X i-1 =S1)=0.90 =p 11, P(X i =S3|X i-1 =S1)=0.025 =p 13, P(X i =S3|X i-1 =S2)=0.05 =p 23, P(X i =S1|X i-1 =S3)=0.25 =p 31, P(X i =S2|X i-1 =S3)=0.25 =p 32, P(X i =S3|X i-1 =S3)=0.50 =p 33, An Example

5. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Transition Matrix

6. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 2. Formal Definitions

7. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Markov Chain

8. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Denote p (0) =[p 1 (0),p 2 (0),…, p m (0) ] with p k (0) =P(X 0 =S k ) where X 0 denote the starting state of the system. The p (n) =[p 1 (n),…,p m (n) ] is the probability after n steps. So we have p (1) =p (0) P, and p (n) =p (n-1) P Where P is the transition matrix. If P is regular, then there is a limit π=(π 1,…, π m )=lim n  ∞ p (n), which satisfies π= πP. Denote p (0) =[p 1 (0),p 2 (0),…, p m (0) ] with p k (0) =P(X 0 =S k ) where X 0 denote the starting state of the system. The p (n) =[p 1 (n),…,p m (n) ] is the probability after n steps. So we have p (1) =p (0) P, and p (n) =p (n-1) P Where P is the transition matrix. If P is regular, then there is a limit π=(π 1,…, π m )=lim n  ∞ p (n), which satisfies π= πP. Basic Operation: Probability after n steps

9. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Example 4.29 A supermarket stocks three brands of coffee – A, B and C – and customers switch from brand to brand according to the transition matrix Where S1 corresponds to a purchase of brand A, S2 to brand B, S3 to brand C; that is ¾ of the customers buying brand A also buy brand A the next time they purchase coffee, whereas ¼ of these customers switch to brand B. 1.Find the probability that a customer who buys brand A today will again purchase brand A 2 weeks from today, assuming that he or she purchase coffee once a week. 2.In the long run, what fractions of customers purchase the respective brands? A supermarket stocks three brands of coffee – A, B and C – and customers switch from brand to brand according to the transition matrix Where S1 corresponds to a purchase of brand A, S2 to brand B, S3 to brand C; that is ¾ of the customers buying brand A also buy brand A the next time they purchase coffee, whereas ¼ of these customers switch to brand B. 1.Find the probability that a customer who buys brand A today will again purchase brand A 2 weeks from today, assuming that he or she purchase coffee once a week. 2.In the long run, what fractions of customers purchase the respective brands?

10. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Solution Answer: 1.Assuming that the customer is chosen at random, his or her transition probabilities are given by P. The given information indicates that P(0)=(1, 0, 0); that is, the customer starts with a purchase of brand A. Then we have p (1) =p (0) P=(3/4, ¼, 0) Gives the probabilities for the next week’s purchase. The probabilities for the two weeks from now are given by p (2) =p (1) P=(9/16, 17/18, 1/12). That is, the chance of the customer to purchase brand A 2 weeks from now is 9/16. 2. The answer to the long-run frequency ratio is given by π, which is the stationary distribution. The equation π = πP yields π1=(3/4) π1+(1/4) π2 π2=(1/4) π1+(2/3) π2+(1/4) π3 π3=(1/3) π2+(1/2) π3 π1+ π2+ π3=1 Solve them to π=(2/7, 3/7, 2/7).

11. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Chapter 5. Continuous Probability Distributions Section 5.1: Continuous Random Variables and Their Probability Distributions Jiaping Wang Department of Mathematical Science 03/18/2013, Monday

12. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline Introduction Density Function Distribution Function More Examples

13. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 1. Introduction

14. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL An Experiment Measuring the life length X of 50 batteries of certain type. 0.4060.6854.7781.7258.523 2.3431.4011.5070.2942.230 0.5380.2344.0253.3232.920 5.0881.4581.0640.7740.761 5.5870.5173.2462.3301.064 2.5630.5113.2462.3301.064 6.5230.2251.5143.2143.810 3.3342.3250.3337.5540.968 3.4912.9211.6240.3344.490 1.2671.7022.6341.8490.186

15. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 2. Density Function

16. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Definition 5.1 A random variable X is said to be continuous if there is a function f(x), called probability density function, such that Notice that P(X=a)=P(a ≤ X ≤ a)=0. A random variable X is said to be continuous if there is a function f(x), called probability density function, such that Notice that P(X=a)=P(a ≤ X ≤ a)=0.

17. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Example 5.1 The random variable X of the life lengths of batteries discussed earlier is associated with a probability density function of the form Find the probability that the life of a particular battery of this type is less than 200 or greater than 400 hours. The random variable X of the life lengths of batteries discussed earlier is associated with a probability density function of the form Find the probability that the life of a particular battery of this type is less than 200 or greater than 400 hours. Answer: Let A be the event that X less than 2 (Hundreds of hours), B be that X is Greater than 4, also A and B are mutually exclusive, so P(AU B)=P(A)+P(B)=P(X≤2)+P(X≥4) =(1-e -1 )+(e -2 )=0.767

18. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Example 5.2 Refer to Example 5.1. Find the probability that a battery of this type lasts more than 3 (Hundreds of hours), given that it already has been in use for more than 2 (Hundreds of hours). Answer: Let A be the event that X has been in use for more than 2, ie., A={X>2}, let B Denote the event that X lasts more than 3, ie., B={X>3}, so we are interested in The new event C={B|A}={X>3|X>2}, so the probability of P(C) is P(X>3|X>2)=P(X>3, X>2)/P(X>2)=P(X>3)/P(X>2)=e -3/2 /e -1 =0.606

19. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 3. Distribution Function

20. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Definition 5.2 The distribution function for a random variable X is defined as F(b)=P(X ≤ b). If X is continuous with probability density function f(x), then Notice that F’(x)=f(x). The distribution function for a random variable X is defined as F(b)=P(X ≤ b). If X is continuous with probability density function f(x), then Notice that F’(x)=f(x). For example, we are given Thus,

21. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Properties The cumulative distribution function F(x) of the continuous random variable X follows the properties:

22. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Example 5.3 The distribution function of the random variable X, the time (in months) from the diagnosis age until death for one population of patients with AIDS, is as follows: F(x)=1-e -0.03x 1.2, x>0. 1.Find the probability that a randomly selected person from this population survives at least 12 months. 2.Find the probability density function of X. The distribution function of the random variable X, the time (in months) from the diagnosis age until death for one population of patients with AIDS, is as follows: F(x)=1-e -0.03x 1.2, x>0. 1.Find the probability that a randomly selected person from this population survives at least 12 months. 2.Find the probability density function of X. Answer: 1. P(X ≥ 12) = 1- P(X≤12) = 1-F(12) = 1-(1- e -0.03(12) 1.2 ) = 1-0.45=0.55. 2. f(x)=F’(x)=0.036x 0.2 e -0.03x 1.2, x>0; and 0, otherwise.

23. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Part 3. More Examples

24. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Additional Example 1 Determine the value of c so that the following function is a density function Answer: Based on the requirement for a density function ∫f(x)dx=1, we have ∫c/(x+1) 3 dx=1  c=1/[∫1/(x+1) 3 dx ]= 2.

25. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Additional Example 2 Let X have the density function If P(X>1)=7/8, find the value of θ. Let X have the density function If P(X>1)=7/8, find the value of θ. Answer: first find F(b)=(b/θ) 3, then we have P(X>1)=1-P(X≤1)=1-F(1)=1-(1/θ) 3 =7/8, So from 1-(1/θ) 3 =7/8, we have θ=2.