1. Lecture 5 This lecture is about: Introduction to Queuing Theory
Queuing Theory Notation
Bertsekas/Gallager: Section 3.3
Kleinrock (Book I)
Basics of Markov Chains
Bertsekas/Gallager: Appendix A
Markov Chains by J. R. Norris

2. Queuing Theory
Queuing Theory deals with systems of the following type:
Typically we are interested in how much queuing occurs or in the delays at the servers.
Server Process(es)
Input
Process
Output

3. Queuing Theory Notation
A standard notation is used in queuing theory to denote the type of system we are dealing with.
Typical examples are:
M/M/1 Poisson Input/Poisson Server/1 Server
M/G/1 Poisson Input/General Server/1 Server
D/G/n Deterministic Input/General Server/n Servers
E/G/ Erlangian Input/General Server/Inf. Servers
The first letter indicates the input process, the second letter is the server process and the number is the number of servers.
(M = Memoryless = Poisson)

4. The M/M/1 Queue The simplest queue is the M/M/1 queue.
Recall that a Poisson process has the following characteristics:
Where A(t) is the number of events (arrivals) up to time t.
Let us assume that the arrival process is a Poisson with mean  and the service process is a Poisson with a mean 

5. Poisson Processes (a refresher)
Interarrival times are i.i.d. and exponentially distributed with parameter .
tn is the time of packet n and n= tn+1 - tn then:
For every t  0 and   0:

6. Poisson Processes (a refresher)
If two or more Poisson processes (A1,A2...Ak) with different means(1, 2... k) are merged then the resultant process has a mean  given by:
If a Poisson process is split into two (or more) by independently assigning arrivals to streams then the resultant processes are both Poisson.
Because of the memoryless property of the Poisson process, an ideal tool for investigating this type of system is the Markov chain.

7. On the Buses (a paradoxical property of Poisson Processes)
You are waiting for a bus. The timetable says that buses are every 30 minutes. (But who believes bus timetables?)
As a mathematician, you have observed that, in fact, the buses are a Poisson process with a mean arrival rate such that the expectation time between buses is 30 minutes.
You arrived at a random time at the bus stop. What is your expected wait for a bus? What is the expected time since the last bus?
15 minutes. After all, they are, on average, 30 minutes apart.
30 minutes. As we have said, a Poisson Process is memoryless so logically, the expected waiting time must be the same whether we arrive just after a previous bus or a full hour since the previous bus.

8. Introduction to Markov Chains
Some process (or time series) {Xn| n= 0,1,2,...} takes values in nonnegative integers.
The process is a Markov chain if, whenever it is in state i, the probability of being in state j next is pij
This is, of course, another way of saying that a Markov Chain is memoryless.
pij are the transition probabilities.

9. Visualising Markov Chains (the confused hippy hitcher example)B C
1/3
2/3
1/2
1/4
3/4
A hitchhiking hippy begins at A
town. For some reason he has
poor short-term memory and
travels at random according
to the probabilities shown. What
is the chance he is back at A after 2
days? What about after 3 days? Where is he likely to end up?

10. The Hippy Hitcher (continued)
After 1 day he will be in B town with probability 3/4 or C town with probability 1/4
The probability of returning to A via B after 1 day is 3/12 and via C is 1/8 total 3/8
We can perform similar
calculations for 3 or 4 days
but it will quickly
become tricky and
finding which city he
is most likely to end up
in is impossible. A B C
1/3
2/3
1/2
1/4
3/4

11. Transition Matrix Instead we can represent the transitions as a matrix
Prob of going to B from A A B C
1/3
2/3
1/2
1/4
3/4
Prob of going to A from C

12. Markov Chain Transition Basics
pij are the transition probabilities of a chain. They have the following properties:
The corresponding probability matrix is:

13. Transition Matrix
Define n as a distribution vector representing the probabilities of each state at time step n.
We can now define 1 step in our chain as:
And clearly, by iterating this, after m steps we have:

14. The Return of the Hippy Hitcher
What does this imply for our hippy?
We know the initial state vector:
So we can calculate n with a little drudge work.
(If you get bored raising P to the power n then you can use a computer)
But which city is the hippy likely to end up in?
We want to know

15. Invariant (or equilibrium) probabilities)
Assuming the limit exists, the distribution vector  is known as the invariant or equilibrium probabilities.
We might think of them as being the proportion of the time that the system spends in each state or alternatively, as the probability of finding the system in a given state at a particular time.
They can be found by finding a distribution which solves the equation:
We will formalise these ideas in a subsequent lecture.

16. Some Notation for Markov Chains
Formally, a process Xn is Markov chain with initial distribution  and transition matrix P if:
P{X0=i} = i (where i is the ith element of )
P{Xn+1=j| Xn=i, Xn-1=xn-1,...X0=x0}= P{Xn+1=j| Xn=i }=pij
For short we say Xn is Markov (,P)
We now introduce the notation for an n step transition:
And note in passing that:
This is the Chapman-Kolmogorov equation