CSE 3504: Probabilistic Analysis of Computer Systems Topics covered: Continuous time Markov chains (Sec )

Introduction  Example: Counting the number of cars at a service station each time a car departs.  Discrete-space, continuous-time:

Introduction (contd..)  Time spent in each state:

Introduction (contd..)  Transition rate:  Generator matrix:  Diagonal entries:

Introduction (contd..)  State probabilities:

Example of CTMC  Example: State of a component  Generator matrix:  How to compute the probabilities of being in each one of the states:

Example of CTMC (contd..)  Balance equations:  Steady-state probabilities:  Numerical example:

Birth-death process  Example: Number of cars in a service station, each time a car arrives and departs.  Transitions in the state-space:

Birth-death process (contd..)  Generator matrix:  Balance equations:

Birth-death process (contd..)  Steady-state or limiting probabilities

Birth-death process (contd..)  Finite state space:  Transitions in the state space:

Birth-death process (contd..)  Generator matrix:  Balance equations:

Birth-death process (contd..)  Steady-state or limiting probabilities: