1. Problems 10/3 1. Ehrenfast’s diffusion model:

2. Problems, cont. 2. Discrete uniform on {0,...,n}

3. Problems, cont. 3. where k=0?

4. Classification of states A state i for a Markov chain X k is called persistent if and transient otherwise. Let and. j is persistent iff f jj =1. Let

5. Some results Theorem: (a)P ii (s)=1+F ii (s)P ii (s) (b)P ij (s)=F ij (s)P ij (s) for i ≠ j. Proof: As for the random walk case we deduce from the Markov property that Multiply both sides by s m, sum over m≥1 to get

6. Some results, cont. Corollary: (a)State j is persistent if and then for all i. (b)State j is transient if and thenfor all i. Proof: SInce we see that But(by Abel’s thm)

7. A final consequence If i is transient, then Why? Example: Branching process What states are persistent? Transient? State 0 is called absorbing, since once the process reaches 0, it never leaves again.

8. Mean recurrence time Let T i = min{n>0: X n = i} and  i = E(T i |X 0 =i). For a transient state  i = ∞. For a persistent state We call a recurrent state positive persistent if  i < ∞, null persistent otherwise. Example: Simple random walk positive recurrent = non-null persistent

9. A model for radiation damage Initial damage from radiation can either heal or get worse until it is visible. 0 is a healthy organism (absorbing) 3 visible damage (absorbing) 1 initial damage 2 amplified damage

10. Radiation damage, cont. Recovery probability 0 is probability of reaching 0 before 3. Last step must go Thus

11. Communication Two states i and j communicate,, if for some m. i and j intercommunicate,, if and. Theorem: is an equivalence relation. What do we need to prove?

12. Equivalence classes of states Theorem: If then (a)i is transient iff j is transient (b)i is persistent iff j is persistent Proof of (a): Since there are m,n with By Chapman-Kolmogorov so summing over r we get

13. Closed and irreducible sets A set C of states is closed if p ij =0 for all i in C, j not in C C is irreducible if for all i,j in C. Theorem: S=T+C 1 +C 2 +... where T are all transient, and the C i are irreducible disjoint closed sets of persistent states Note: The C i are the equivalence classes for

14. Example S={0,1,2,3,4,5} {0,1},{4,5} closed irreducible persistent {2,3} transient. Why?

15. Long-term behavior Recall from the 0-1 process that When does this not depend on n? (a)p 01 = p 11 (b) (c)

16. Stationary distribution Case (b) is the general one. Here is the idea: Recall that  (n) =  (0) P n. In order to get the same distribution for all n, we use  (0) =  where  solves  P =   (1) =  P =   P 2 =  P = ...  P n = 

17. Snoqualmie Falls so or