1. S TOCHASTIC M ODELS L ECTURE 3 P ART II C ONTINUOUS -T IME M ARKOV P ROCESSES Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Oct 21, 2015

2. Outline 1.Birth-Death Processes

3. 3.3 B IRTH -D EATH P ROCESSES

4. Birth-Death Processes Birth-death processes are one of the most important examples for continuous-time Markov chains. Consider a system whose state at any time is represented by the number of people. Suppose that whenever there are people in the system, then – new arrivals enter it (people born) at an exponential rate – People leave the system (people die) at a rate

5. Birth-Death Processes (Continued) The above system can be modeled as a continuous-time Markov chain with states for which the transition parameters are given by –

6. Example I: The Poisson process The Poisson process, as a special example of birth-death processes, has the following parameter specification: –

7. Example II: A Birth Process with Linear Birth Rate (Yule Process) Consider a population whose members can give birth to new members but cannot die. Suppose that each member acts independently of the others and takes an exponentially distributed amount of time, with mean to give a birth. Consider the population number. We have –

8. Example III: A Linear Growth Model with Immigration A model in which – is called a linear growth process with immigration. In it, each individual is assumed to give birth at an exponential rate and die at rate ; in addition, there is an rate of increase of the population due to an external source such as immigration.

9. Kolmogorov Equations for B-D Processes We can easily have the backward equations for a birth-death process:

10. Kolmogorov Equations for B-D Processes (Continued) As for the forward equations, we have

11. Example IV: Pure Birth Processes Calculate for the process in Example II, using the forward equations. We have

12. Example IV: Pure Birth Processes (Continued) The solution is

13. Limiting Probabilities Let us now determine the limiting probabilities for a birth-death process. From the general equation we should have – State 0: – State 1: – State 2: – State

14. Limiting Probabilities (Continued) We can solve the above equations to obtain ……

15. Limiting Probabilities (Continued) Then, the limiting probabilities of a general birth-death process are given by

16. Example V: A Machine Repair Model A job shop consists of machines and one serviceman. Suppose that the amount of time each machine runs before breaking down is exponentially distributed with mean Suppose that it takes the serviceman an exponential time (with mean ) to fix a machine. What is the average number of machine not in use?

17. Example V: A Machine Repair Model Establish a birth-death process to model the system. The process is in state whenever machines are not in use. Then, the process has the following parameters: –

18. Example V: A Machine Repair Model (Continued) By the formula of the limiting probabilities, we have

19. Example V: A Machine Repair Model (Continued) Hence, the average number of machines not in use is given by

20. Homework Assignments Read Ross Chapter 6.3. Answer Questions: – Exercises 12 (Page 400, Ross) – Exercises 15 (Page 401, Ross) – Exercises 23 (Page 402, Ross) – (Optional, Extra Bonus) Exercise 25 (Page 402, Ross).