Solutions to group exercises 1. (a) Truncating the chain is equivalent to setting transition probabilities to any state in {M+1,...} to zero. Renormalizing the transition matrix to make it stochastic we get (b) We have so

and yielding detailed balance for the truncated chain. 2. Let Z n = (X 2n,X 2n+1 ). Then Z n is a Markov chain with transition matrix Let T k = P(hit (0,1) before (1,0)|start at k). First step analysis yields the equations

(b) If q 0 =p 1 =p we get p 01,01 =1/2, fair coin. 3. (a) so whence the differential equation follows by letting. (b) Since P 1 (t)=1-P 0 (t) we get

which can either be solved directly, or one can check that the given solution satisfies the differential equation. (c) Letting we get This value as a starting distribution also yields a marginal distribution that is free of t, so it behaves like a stationary distribution (which we will define later).

Announcement MathAcrossCampus Colloquium ( Evolutionary trees, coalescents, and gene trees: can mathematicians find the woods? JOE FELSENSTEIN Genome Sciences, UW Thursday, November 13, 2008, 3:30 Kane Hall 210 Reception to follow

The Markov property X(t) is a Markov process if for any n for all j, i 0,...,i n in S and any t 0 <t 1 <...<t n <t. The transition probabilities are homogeneous if p ij (s,t)=p ij (0,t-s). We will usually assume this, and write p ij (t).

Semigroup property Let P t be [p ij (t)]. Then P t is a substochastic semigroup, meaning that (a)P 0 = I (b)P s+t = P s P t (c)P t is a substochastic matrix, i.e. has nonnegative entries with row sums at most 1.

Proof (a)? (b) (c)

Standard semigroup P t,t≥0 is a standard semigroup if as. Theorem: For a standard semigroup the transition probabilities are continuous. Proof: By Chapman-Kolmogorov Unless otherwise specified we will consider standard semigroups.

Infinitesimal generator By continuity of the transition probabilities, Taylor expansion suggests We must have g ij ≥0, g ii ≤0. Let G=[g ij ]. Then (under regularity conditions) G is called the infinitesimal generator of P t.

Birth process G = Under regularity conditions we have so we must have

Forward equations so and or

Backward equations Instead of looking at (t,t+h] look at (0,h]: so

Formal solution In many cases we can solve both these equations by But this can be difficult to actually calculate.

The 0-1 case

0-1 case, continued Thus

Marginal distribution Let. Then for a starting distribution  (0) we have  (t) =  (0) P t. For the 0-1 process we get

Exponential holding times Suppose X(t)=j. Consider Let  be the time spent in j until the next transition after time t. By the Markov property, P(stay in j in (u,u+v], given stays at least u) is precisely P(stay in j v time units). Mathematically Let g(v)=P(  >v). Then we have g(u+v)=g(u)g(v), and it follows that g(u)=exp(- u). By the backward eqn and P(  >v)=p jj (v).

Jump chain Given that the chain jumps from i at a particular time, the probability that it jumps to j is -g ij /g ii. Here is why (roughly): Suppose t<  <t+h, and there is only one jump in (t,t+h] (likely for small h). Then

Construction The way the continuous time Markov chains work is: (1)Draw an initial value i 0 from  (0) (2)If, stay in i 0 for a random time which is (3)Draw a new state from the distribution where

Death process Let g i,i-1 =  i = - g i,i. The forward equation is Write. Then This is a Lagrange equation with sln or