15 October 2012 Eurecom, Sophia-Antipolis Thrasyvoulos Spyropoulos / Absorbing Markov Chains 1

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  A mouse is trapped in the above maze with 3 rooms and 1 exit  When inside a room with x doors, it chooses any of them with equal probability (1/x) Q: How long will it take it on average to exit the maze, if it starts at room i? Q: How long if it starts from a random room? exit

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Def: T i = expected time to leave maze, starting from room I  T 2 = 1/3*1 + 1/3*(1+T 1 )+1/3*(1+T 3 ) = 1 + 1/3*(T 1 +T 3 )  T 1 = 1 + T 2  T 3 = 1 + T 2  T 2 = 5, T 3 = 6, T 1 = 6 Q: Could you have guessed it directly? A: times room 2 is visited before exiting is geometric(1/3)  on average, the wrong exit will be taken twice (each time costing two steps) and the 3 rd time the mouse exits exit

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  A packet must be routed towards the destination over the above network  “Hot Potato Routing” works as follows: when a router receives a packet, it picks any of its outgoing links randomly (including the incoming link) and send the packet immediately. Q: How long does it take to deliver the packet? 4 destination

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  First Step Analysis: We can still apply it!  But it’s a bit more complicated: 9x9 system of linear equations  Not easy to guess solution either!  We’ll try to model this with a Markov Chain 5 destination

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  9 transient states: 1-9  1 absorbing state: A Q: Is this chain irreducible? A: No! Q: Hot Potato Routing Delay  expected time to asborption? A 1 1/4 1/2 1/3 1/2 1/4 1/3 1/2 1/3 1/2 1/4 1 1/3

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  We can define transition matrix P (10x10) Q: What is P (n) as n  ∞? A: every row converges to [0,0,…,1] Q: How can we get ET iA ?  (expected time to absorption starting from i) Q: How about ? A: No, the sum goes to infinity! A 1 1/4 1/2 1/3 1/2 1/4 1/3 1/2 1/3 1/2 1/4 1 1/3

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Transition matrix can be written in canonical form  Transient states written first, followed by absorbing ones  Calculate P (n) using canonical form Q: Q n as n  ∞? A: it goes to O Q: where does the (*) part of the matrix converge to if only one absorbing state? A: to a vector of all 1s 8

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Theorem: The matrix (I-Q) has an inverse  N = (I-Q) -1 is called the fundamental matrix  N = I + Q + Q 2 + …  n ik : the expected number of times the chain is in state k, starting from state i, before being absorbed Proof: 9

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Theorem:  Let T i be the expected number of steps before the chain is absorbed, given that the chain starts in state i,  let T be the column vector whose i th entry is T i. then T = Nc,  where c is a column vector all of whose entries are 1 Proof:  Σ k n ik :add all entries in the i th row of N  expected number of times in any of the transient states for a given starting state i  the expected time required before being absorbed.  T i = Σ k n ik. 10

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Theorem:  b ij :probability that an absorbing chain will be absorbed in (absorbing) state j, if it starts in (transient) state i.  B: (t-by-r) matrix with entries b ij. then B = NR,  R as in the canonical form. Proof: 11

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Use Matlab to get matrices  Matrix N =  Vector T =

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  A wireless path consisting of H hops (links)  link success probability p  A packet is (re-)transmitted up to M times on each link  If it fails, it gets retransmitted from the source (end-to-end) Q: How many transmissions until end-to-end success? 13