1. CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Discrete time Markov chains (Sec. 7.2-7.3)

2. Computation of n-step probabilities  One-step transition probability matrix:  What is the probability of going from state i to state j in n steps?

3. Absolute probability distribution  Definition:  Mathematical representation:

4. Absolute probability distribution (contd..)  How to compute:

5. Steady-state or limiting probabilities  Definition:  Classification of DTMCs based on limiting probabilities:

6. Absorbing chains  Example:  Transient states:  Absorbing states:

7. Absorbing chains (contd..)  Number of visits to transient state 1 before reaching state 0:  What is the “average” number of visits to each transient state before being absorbed?

8. Absorbing chains (contd..)  Arrange the entries of matrix P:  Partition the matrix P:

9. Absorbing chains (contd..)  Fundamental matrix M:  Mean number of visits:

10. Absorbing chains (contd..)  Example:

11. Irreducible chains  Example:  Definition:  Steady-state or equilibrium state:

12. Irreducible chains (contd..)  Computation of steady-state probabilities:

13. Irreducible chains (contd..)  Example: