Homework 2 Question 2: For a formal proof, use Chapman-Kolmogorov Question 4: Need to argue why a chain is persistent, periodic, etc. To calculate mean recurrence times, use the stationary distribution Question 5: Interpretation issue Question 6: Geometric distribution is memoryless

Review Distribution of a stochastic process Discrete time: conditional distributions Markov property Equivalent forms Proof of Markov property: X n =X n-1 +Y n where Y n depends only on X n-1 OR calculate conditional distributions

The opposite approach To disprove Markov property: try to show Example: Y n =1,2,3 or 4 iid uniform. For j=1,2,3 A nj ={Y n =j or 4}. X 3m+j =1(A mj )

Description of Markov chain Need matrix P of transition probabilities Vector  0 of initial probabilities  n =  0 P n Chapman-Kolmogorov P (m+n) = P (m) P (n) P (n) matrix of n-step transition probabilities P (n) =P n P1 T =? P n 1 T =?

Example What happens as ?

Classification of states Communicating states Persistent/transient, periodic Compute first return distribution P ii (s)=1+F ii (s)P ii (s) where

Example Hence so

Stationary distribution  P =  If    then  n  P n =  Also regardless of the initial distribution Fact: When there is a solution to  P =  which is a probability distribution then the chain is irreducible positive persistent Another fact: the mean recurrence time for an ergodic class is

Example S={0,1,2,3} ST=SP=ST=SP= u=P 2 (absorption in {0,1})

Example, cont. Stationary distribution for {0,1}: so and

Branching process Individuals have independent offspring according to a distribution with pgf G(s). Given Z n-1 =z, the nth generation Z n is determined by the sum of the z offspring sizes. Z 0 = 1. Facts: The pgf of Z n is the nth iterate EZ n =  n where  = G’(1). The extinction probability q is the smallest solution to G(s)=s.