Download presentation

Presentation is loading. Please wait.

Published byVernon Jennings Modified over 2 years ago

1
Matrix Analytic methods in Markov Modelling

2
Continous Time Markov Models X: R -> X µ Z (integers) X(t): state at time t X: state space (discrete – countable) R: real numbers (continuous time line)

3
Continous Time Markov Models X is piecewise constant X is typically cadlag ("continue à droite, limite à gauche") = RCLL (“right continuous with left limits”)

4
Transition probabilities P(X(t+h)=j|X(t)=i) ¼ h ¸ ij for i j P(X(t+h)=j|X(t)=j) = 1- i j P(X(t+h)=i|X(t)=j) ¼ 1- h i j ¸ ji P(X(t+h)=j)= i P(X(t+h)=j and X(h)=i) = i P(X(t+h)=j | X(h)=i) P(X(h)=i) = i j ¸ ij h P(X(h)=i) + h(1- i j ¸ ji ) P(X(h)=j)

5
Transition probabilities P(X(t+h)=j)= i P(X(t+h)=j and X(h)=i) + (1- k j ¸ ik ) P(X(h)=j) P(t)=[P(X(t)=0) P(X(t)=1) P(X(t)=2)..] P(t+h) ¼ P(t)H H=I+hQ Q ij = ¸ ij Q jj = 1- i j ¸ ji

6
Taking limits P(t+h) ¼ P(t)H H=I+hQ P(t+h) ¼ P(t)(I+hQ)=P(t)+hP(t)Q (P(t+h)-P(t))/h ¼ P(t)Q d/dt P(t) = P(t)Q P(t)=P(0)exp(Qt)

7
Irreducibility X is irreducible when states are mutually reachable. X is irreducible iff for every i,j 2 X there is a sequence {i(1),i(2),..,I(N) 2 X} such that i(1)=i i(N)=j and ¸ i(k),i(k+1) > 0 for every k 2 1,..,N-1

8
Recurrence Assume X(t n - )=j and X(t n )=i j then t n is a transition time Let {t n } be the sequence of consequetive transition times. {Xn=X(t n )} is called the embedded chain Let t 0 =0, X 0 =i then k(i)=inf{k>0: X(k)=i}=inf{k>1: X(k)=i} T i =t k(i) T i is the time to next visit at i X is recurrent if P(T i < 1 )=1 for all i (almost certain return to all states) X is positive recurrent if E(T i ) · 1 for all i

9
Stationary probability d/dt P(t) = P(t)Q P(t)=P(0)exp(Qt) When X is irreducible and positive recurrent there is a unique probability vector ¦ such that P(t) -> ¦ ¦ solves ¦ Q=0

10
Stationary probability Ergodicity: ¦ i = E(D i )/E(T i ) ¦ i is the fraction of the time in state i Statistically intuitively appealing X(t) ==i TiTi DiDi time

11
Example Poisson Counting Process Q i,i+1 = ¸ ¸¸¸¸ 1023 Counts Poisson events Birth Chain Not irreducible Not recurrent

12
Example Birth/Death(BD)-chain Q i,i+1 = ¸ Q i+1,i = ¹ ¸¸¸¸ 1023 Models a queueing system with Poisson arrival process and independent exponentially distributed service times ¸ is arrival rate ¹ is service rate Irreducible Positive recurrent for ¸ < ¹ ¹¹¹¹

13
Example Birth/Death(BD)-chain ¸¸¸¸ 1023 ½ = ¸ / ¹ ¦ n = ½¦ n-1 ¦ n = ½ n ¦ 0 P 0 =1/ n=0 1 ½ n =1- ½ ¦ n = ½ n (1- ½ ) E(X) = n=0 1 n ¦ n = ½ /(1- ½ ) ¹¹¹¹

14
Example Birth/Death(BD)-chain ¸1¸2¸3¸4 1023 ½ n = ¸ n / ¹ n ¦ n = ½ n ¦ n-1 ¦ n = ¦ i=1 n ½ i ¦ 0 P 0 =1/ n=0 1 ¦ i=1 n ½ i ¹1¹2¹3¹4

15
Markov Modulated Poisson Process Has two modes: Modes={ON,OFF} M(t) 2 Modes is a two state CTMC Transmits with rate ¸ in ON mode. Counting proces combines state spaces, i.e. X = Modes £ {0,1,2,..} Q (ON,i),(ON,i+1) = ¸ Q (ON,i),(OFF,i) = ¯ Q (OFF,i),(ON,i) = ® Q i,j =0 otherwise

16
Markov Modulated Poisson Process Q (ON,i),(ON,i+1) = ¸ Q (ON,i),(OFF,i) = ¯ Q (OFF,i),(ON,i) = ® Q i,j =0 otherwise 1OFF23 ¸¸¸¸ 1023 0 ON ® ¯®®®¯¯¯

17
MMPP with exponential service Q (ON,i),(ON,i+1) = ¸ Q (ON,i),(OFF,i) = ¯ Q (OFF,i),(ON,i) = ® Q (ON,i),(ON,i-1) = ¹ Q (OFF,i),(OFF,i-1) = ¹ Q i,j =0 otherwise 1OFF23 ¸¸¸¸ 1023 0 ON ® ¯®®®¯¯¯ ¹¹¹¹ ¹¹¹¹

18
State ordering For a state (i,M) we denote i the level of the state We order states so that equal levels are gathered (i,OFF),(i,ON)(i+1,OFF)(i+1,ON)(i+2,OFF)(i+2,ON)

19
Generator matrix

20
Sub matrices

21
Generator matrix by submatrices Balance equations: P 0 A 0 + P 1 B = 0 eq(0) P 0 C + P 1 A + P 2 B = 0 eq(1) P i C + P i+1 A + P i+2 B = 0 eq(i+1) We look for a matrix geometric solution, i.e. P 1 =P 0 R P i+1 = P i R Inserting in eq(0): P 0 A 0 + P 0 R B = 0 and eq(i+1) P i (C+R A + R 2 B)=0 for all P i

22
Solving for R P i (C+R A + R 2 B)=0 for all P i Sufficient that C+R A + R 2 B=0 (Ricatti equation) Iterative solution R 0 =0 repeat R n+1 =-(C+R n 2 B) A -1 Converges for irreducible positive recurrent Q

23
MM - service Q (ON,i),(ON,i+1) = ¸ Q (OFF,i),(OFF,i+1) = ¸ Q (ON,i),(OFF,i) = ¯ Q (OFF,i),(ON,i) = ® Q (ON,i),(ON,i-1) = ¹ Q (OFF,i),(OFF,i-1) =0 Q i,j =0 otherwise 1OFF23 ¸¸¸¸ 1023 0 ON ® ¯®®®¯¯¯ ¹¹¹¹ ¸ ¸¸¸

24
Generator matrix

25
Sub matrices

26
Generally We still look for a matrix geometric solution: Now: i=0 1 R i A i = 0 A 0 + R A 1 i=2 1 R i A i Iteration: R n = -A 1 -1 (A 0 + i=2 1 R n-1 i A i ) Solving for P 0 : P 0 i=0 1 R i B i = 0 Conditions for solution: Irreducibility and pos. recurrence

27
Miniproject (i) Let traffic be generated by an on/off Markov process with on rate: ¸ =1, mean rate 0.1 ¸, average on time: T=1 Let service be exponential with rate ¹ = 0.2 ¸ Construct the generator matrix for the data given above. Use the iterative algoritme to solve for the load matrix R Solve for P 0 and P i Compute the mean queue length Compare with M/M/1 results

28
Miniproject (ii) Collect file size or web session duration data Check for power tails and estimate tail power Find appropriate parameters for a hyperexponential approximation of the reliability estimated reliability Construct the generator matrix of an equivalent ME/M/1 queue

Similar presentations

OK

Many useful applications, especially in queueing systems, inventory management, and reliability analysis. A connection between discrete time Markov chains.

Many useful applications, especially in queueing systems, inventory management, and reliability analysis. A connection between discrete time Markov chains.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on condition based maintenance certification Ppt on seven segment display schematic diagram Ppt on vodafone company profile Ppt on coalition government big Ppt on domain and range Ppt on landscape photography Ppt on html tags with examples Ppt on articles of association definition Ppt on history of mathematics Ppt on climate and weather