Download presentation
Presentation is loading. Please wait.
1
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
2
Last time: Birth-death process
State space Markov chain, transition rates Definition: Global balance Definition: Detailed balance equations Theorem: A distribution that satisfied detailed balance is a stationary distribution Theorem: Assume that then is the equilibrium distrubution of the birth-death process X.
3
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
4
Reversibility; stationarity
Definition: Stationary process: A stochastic process is stationary if for all t1,…,tn, Theorem: If the initial distribution is a stationary distribution, then the process is stationary Definition: Reversible process: A stochastic process is reversible if for all t1,…,tn,
5
Reversibility; stationarity
Lemma: A reversible process is stationary. Theorem: A stationary Markov chain is reversible if and only if there exists a collection of positive numbers π(j), jS, summing to unity that satisfy the detailed balance equations When there exists such a collection π(j), jS, it is the equilibrium distribution Proof
6
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
7
Truncation of reversible processes
10 Truncation of reversible processes Lemma 1.9 / Corollary 1.10: If the transition rates of a reversible Markov process with state space S and equilibrium distribution are altered by changing q(j,k) to cq(j,k) for where c>0 then the resulting Markov process is reversible in equilibrium and has equilibrium distribution where B is the normalizing constant. If c=0 then the reversible Markov process is truncated to A and the resulting Markov process is reversible with equilibrium distribution A S\A
8
Time reversed process X(t) reversible Markov process X(-t) also, but
Lemma 1.11: time-homogeneity not inherited for non-stationary process Theorem 1.12 : If X(t) is a stationary Markov process with transition rates q(j,k), and equilibrium distribution π(j), jS, then the reversed process X(-t) is a stationary Markov process with transition rates and the same equilibrium distribution Theorem 1.13: Kelly’s lemma Let X(t) be a stationary Markov processwith transition rates q(j,k). If we can find a collection of numbers q’(j,k) such that q’(j)=q(j), jS, and a collection of positive numbers (j), jS, summing to unity, such that then q’(j,k) are the transition rates of the time-reversed process, and (j), jS, is the equilibrium distribution of both processes.
9
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
10
Kolmogorov’s criteria
Theorem 1.8: A stationary Markov chain is reversible iff for each finite sequence of states Notice that
11
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
12
Poisson process Definition : Poisson process : Let S1,S2,… be a sequence of independent exponential() r.v. Let Tn=S1+…+Sn, T0=0 and N(s)=max{n,Tn≤s}. The counting process {N(s),s≥0} is called Poisson process. Theorem : If {N(s),s≥0} is a Poisson process, then (i) N(0)=0, (ii) N(t+s)-N(s)=Poisson( t), and (iii) N(t) has independent increments. Conversely, if (i), (ii), (iii) hold, then {N(s),s≥0} is a Poisson process
13
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
14
PASTA: Poisson Arrivals See Time Averages
fraction of time system in state n probability outside observer sees n customers at time t probability that arriving customer sees n customers at time t (just before arrival at time t there are n customers in the system) in general
15
PASTA: Poisson Arrivals See Time Averages
Let C(t,t+h) event customer arrives in (t,t+h) For Poisson arrivals q(n,n+1)= so that Alternative explanation; PASTA holds in general! PASTA
16
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
17
Output simple queue Simple queue, Poisson() arrivals, exponential() service X(t) number of customers in M/M/1 queue: in equilibrium reversible Markov process. Forward process: upward jumps Poisson () Reversed process X(-t): upward jumps Poisson () = downward jump of forward process Downward jump process of X(t) Poisson () process
18
Output simple queue (2) Let t0 fixed. Arrival process Poisson, thus arrival process after t0 independent of number in queue at t0. For reversed process X(-t): arrival process after –t0 independent of number in queue at –t0 Reversibility: joint distribution departure process up to t0 and number in queue at t0 for X(t) have same distribution as arrival process to X(-t) up to –t0 and number in queue at –t0. In equilibrium the departure process from an M/M/1 queue is a Poisson process, and the number in the queue at time t0 is independent of the departure process prior to t0 Holds for each reversible Markov process with Poisson arrivals as long as an arrival causes the process to change state
19
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
20
Tandem network of simple queues
Simple queue, Poisson() arrivals, exponential() service Equilibrium distribution Tandem of J M/M/1 queues, exp(i) service queue i Xi(t) number in queue i at time t Queue 1 in isolation: simple queue. Departure process queue 1 Poisson, thus queue 2 in isolation: simple queue State X1(t0) independent departure process prior to t0, but this determines (X2(t0),…, XJ(t0)), hence X1(t0) independent (X2(t0),…, XJ(t0)). Similar Xj(t0) independent (Xj+1(t0),…, XJ(t0)). Thus X1(t0), X2(t0),…, XJ(t0) mutually independent, and
21
Flows and Networks Plan for today (lecture 3):
Last time / Questions? Reversibility, stationarity Truncation Kolmogorov’s criteria Poisson process PASTA Output simple queue Tandem network Summary / Next Exercises
22
Summary / next: Reversibility Poisson process Arrival process
Output reversible Markov process Tandem network NEXT: Jackson network
23
Exercises [R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5, , 1.6.3, 1.6.4, 2.1.1, 2.1.2
![Exercises [R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5, 1.6.2, 1.6.3, 1.6.4, 2.1.1, 2.1.2](http://slideplayer.com/slide/13216110/79/images/23/Exercises+%5BR%2BSN%5D+1.3.2%2C+1.3.3%2C+1.3.5%2C+1.5.1%2C+1.5.2%2C+1.5.5%2C+1.6.2%2C+1.6.3%2C+1.6.4%2C+2.1.1%2C+2.1.2.jpg "Exercises [R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5, 1.6.2, 1.6.3, 1.6.4, 2.1.1, 2.1.2")
Similar presentations
