1. Introduction to Queuing Theory

2. 2 Queuing theory definitions  (Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory."  (Mathworld) “The study of the waiting times, lengths, and other properties of queues.” http://www2.uwindsor.ca/~hlynka/queue.html

3. 3 Queuing theory for studying networks  View network as collections of queues  FIFO data-structures  Queuing theory provides probabilistic analysis of these queues  Examples:  Average length  Average waiting time  Probability queue is at a certain length  Probability a packet will be lost

4. 4 Model Queuing System Server System Queuing System Queue Server Queuing System  Use Queuing models to  Describe the behavior of queuing systems  Evaluate system performance

5. 5 Characteristics of queuing systems  Arrival Process  The distribution that determines how the tasks arrives in the system.  Service Process  The distribution that determines the task processing time  Number of Servers  Total number of servers available to process the tasks

6. 6 Kendall Notation 1/2/3(/4/5/6)  Six parameters in shorthand First three typically used, unless specified 1. Arrival Distribution 2. Service Distribution 3. Number of servers 4. Total Capacity (infinite if not specified) 5. Population Size (infinite) 6. Service Discipline (FCFS/FIFO)

7. 7 Arrival time distributions  M: stands for "Markovian“, Poisson arrivials  D: Deterministic (e.g. fixed constant)  G: General (anything)

8. 8 Kendall Notation Examples  M/M/1:  Poisson arrivals and exponential service, 1 server, infinite capacity and population, FCFS (FIFO)  the simplest ‘realistic’ queue  M/M/m  Same, but M servers  G/G/3/20/1500/SPF  General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First

9. 9 Analysis of M/M/1 queue  Given: : Arrival rate of jobs (packets on input link)  : Service rate of the server (output link)  Solve:  L: average number in queuing system  L q average number in the queue  W: average waiting time in whole system  W q average waiting time in the queue

10. 10 M/M/1 queue model  WqWq W L LqLq

11. 11 Little’s Law  Little’s Law: Average number of jobs in system = average arrival rate of jobs × average service time  Observed before, Little was first to prove  Applies to any system in equilibrium, as long as nothing in black box is creating or destroying jobs ArrivalsDepartures System

12. 12 Formally, Little’s Law is L = W  : Average arrival rate  W: Average time job is in the system (average service time)  L: Average number of jobs in the system Average number of jobs in system = average arrival rate × average service time for each job

13. 13 Proof of Little’s Law: Let J be the sum of all service time, that is, the sum of time spent on each job. E.g., J = j 1 +j 2 +j 3 …., where j 1 is the service time of job 1, and so on. Let N be the number of jobs (packets). Let T be the total service time of the system on the jobs. Note that T ≤ J. We know is the average arrival rate, W is the average service time of jobs, and L is the average number of jobs in the system. On the right-hand side, by definitions, in a stable system we have W)( W T N )(  ))(( N J T N  T J  By definition, J is the sum of service time for all jobs, T is the total service time of the system. Thus, J/T is the average number of jobs in the system, which is L. Therefore, the Little’s Law holds.

14. 14 An example of J J = Shaded area = 9 Same in all cases! 1 2 3 4 5 6 7 8 Packet # Time 1 2 3 1 2 3 4 5 6 7 8 # in System 1 2 3 Time 1 2 3 Time in System Packet # 1 2 3 Arrivals Departures

15. 15 M/M/1 queue model  WqWq W L LqLq L=λW L q =λW q : Arrival rate of jobs (packets on input link)  : Service rate of the server (output link)  L: average number in queuing system  L q average number in the queue  W: average waiting time in whole system  W q average waiting time in the queue

16. 16 Poisson process  Used to model random events in time that are largely independent of one another  E.g., a event may be that a customer arrives at the ice cream shop  E.g., or you see a roadkill when driving  E.g., or a packet arrives at the router

17. 17 Poisson Process  For a poisson process with average arrival rate, the probability of seeing n arrivals in time interval Δt

18. 18 Poisson process & exponential distribution  Inter-arrival time t (time between arrivals) in a Poisson process follows exponential distribution with parameter

19. 19 M/M/1 queue model  WqWq W L LqLq L=λW L q =λW q W = W q + (1/μ) : Arrival rate of jobs (packets on input link)  : Service rate of the server (output link)  L: average number in queuing system  L q average number in the queue  W: average waiting time in whole system  W q average waiting time in the queue

20. 20 Solving queuing systems  4 unknowns: L, L q W, W q  Relationships:  L= W  L q = Wq  W = W q + (1/  )  If we know any 1, can find the others

21. 21 Analysis of M/M/1 queue  Goal: A closed form expression of the probability of the number of jobs in the queue (P i ) given only and   Online M/M/1 animation  http://www.dcs.ed.ac.uk/home/jeh/Simjava/que ueing/mm1_q/mm1_q.html

22. 22 Equilibrium conditions 10   P n is the probability that in equilibrium the system has n number of jobs n+1nn-1   

23. 23 Solving for P 0 and P n at equilibrium  Step 1  Step 2 Verify this after class

24. 24  Step 3  Step 4 Solving for P 0 and P n at equilibrium P n : probability of n jobs in the system

25. 25 Solving for L

26. 26 Solving W, W q and L q

27. 27 Response time vs. utilization (ρ)

28. 28 Stable Region linear region

29. 29 An example  On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model.  Measurement of a network gateway:  mean arrival rate (l): 125 Packets/s  mean response time (m): 2 ms  Assuming exponential arrivals:  What is the gateway’s utilization?  What is the probability of n packets in the gateway?  mean number of packets in the gateway?

30. 30 Example cont’d Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ / μ = 0.25 Prob. of n packets in gateway = P n = Mean number of packets in gateway = L =