1. Lecture 5 This lecture is about: Introduction to Queuing Theory Queuing Theory Notation Bertsekas/Gallager: Section 3.3 Kleinrock (Book I) Basics of Markov Chains Bertsekas/Gallager: Appendix A Kleinrock (Book I) Markov Chains by J. R. Norris

2. Queuing Theory deals with systems of the following type: Typically we are interested in how much queuing occurs or in the delays at the servers. Queuing Theory Input Process Server Process(es) Output

3. Queuing Theory Notation A standard notation is used in queuing theory to denote the type of system we are dealing with. Typical examples are: M/M/1Poisson Input/Poisson Server/1 Server M/G/1Poisson Input/General Server/1 Server D/G/nDeterministic Input/General Server/n Servers E/G/ Erlangian Input/General Server/Inf. Servers The first letter indicates the input process, the second letter is the server process and the number is the number of servers. (M = Memoryless = Poisson)

4. The M/M/1 Queue The simplest queue is the M/M/1 queue. Recall that a Poisson process has the following characteristics: Where A(t) is the number of events (arrivals) up to time t. Let us assume that the arrival process is a Poisson with mean and the service process is a Poisson with a mean

5. Poisson Processes (a refresher) Interarrival times are i.i.d. and exponentially distributed with parameter. t n is the time of packet n and n = t n+1 - t n then: For every t 0 and 0:

6. If two or more Poisson processes (A 1,A 2...A k ) with different means( 1, 2... k ) are merged then the resultant process has a mean given by: If a Poisson process is split into two (or more) by independently assigning arrivals to streams then the resultant processes are both Poisson. Because of the memoryless property of the Poisson process, an ideal tool for investigating this type of system is the Markov chain. Poisson Processes (a refresher)

7. On the Buses (a paradoxical property of Poisson Processes) You are waiting for a bus. The timetable says that buses are every 30 minutes. (But who believes bus timetables?) As a mathematician, you have observed that, in fact, the buses are a Poisson process with a mean arrival rate such that the expectation time between buses is 30 minutes. You arrived at a random time at the bus stop. What is your expected wait for a bus? What is the expected time since the last bus? 15 minutes. After all, they are, on average, 30 minutes apart. 30 minutes. As we have said, a Poisson Process is memoryless so logically, the expected waiting time must be the same whether we arrive just after a previous bus or a full hour since the previous bus.

8. Introduction to Markov Chains Some process (or time series) {X n | n= 0,1,2,...} takes values in nonnegative integers. The process is a Markov chain if, whenever it is in state i, the probability of being in state j next is p ij This is, of course, another way of saying that a Markov Chain is memoryless. p ij are the transition probabilities.

9. Visualising Markov Chains (the confused hippy hitcher example) A BC 1/3 2/3 1/2 1/4 3/4 A hitchhiking hippy begins at A town. For some reason he has poor short-term memory and travels at random according to the probabilities shown. What is the chance he is back at A after 2 days? What about after 3 days? Where is he likely to end up?

10. The Hippy Hitcher (continued) After 1 day he will be in B town with probability 3/4 or C town with probability 1/4 The probability of returning to A via B after 1 day is 3/12 and via C is 1/8 total 3/8 We can perform similar calculations for 3 or 4 days but it will quickly become tricky and finding which city he is most likely to end up in is impossible. A BC 1/3 2/3 1/2 1/4 3/4

11. Transition Matrix Instead we can represent the transitions as a matrix A BC 1/3 2/3 1/2 1/4 3/4 Prob of going to B from A Prob of going to A from C

12. Markov Chain Transition Basics p ij are the transition probabilities of a chain. They have the following properties: The corresponding probability matrix is:

13. Transition Matrix Define n as a distribution vector representing the probabilities of each state at time step n. We can now define 1 step in our chain as: And clearly, by iterating this, after m steps we have:

14. The Return of the Hippy Hitcher What does this imply for our hippy? We know the initial state vector: So we can calculate n with a little drudge work. (If you get bored raising P to the power n then you can use a computer) But which city is the hippy likely to end up in? We want to know

15. Invariant (or equilibrium) probabilities) Assuming the limit exists, the distribution vector is known as the invariant or equilibrium probabilities. We might think of them as being the proportion of the time that the system spends in each state or alternatively, as the probability of finding the system in a given state at a particular time. They can be found by finding a distribution which solves the equation: We will formalise these ideas in a subsequent lecture.

16. Some Notation for Markov Chains Formally, a process X n is Markov chain with initial distribution and transition matrix P if: 1. P{X 0 =i} = i (where i is the i th element of ) 2. P{X n+1 =j| X n =i, X n-1 =x n-1,...X 0 =x 0 }= P{X n+1 =j| X n =i }=p ij For short we say X n is Markov (,P) We now introduce the notation for an n step transition: And note in passing that: This is the Chapman-Kolmogorov equation