1. Queueing Theory
Chapter 17

2. Basic Queueing Process
Arrivals
Arrival time distribution
Calling population (infinite or finite)
Queue
Capacity (infinite or finite)
Queueing discipline
Service
Number of servers (one or more)
Service time distribution
“Queueing System”

3. Examples and Applications
Call centers (“help” desks, ordering goods)
Manufacturing
Banks
Telecommunication networks
Internet service
Intelligence gathering
Restaurants
Other examples….

4. Labeling Convention (Kendall-Lee)
/ / / / /
Interarrival time distribution
Service time distribution
Number of servers
Queueing discipline
System capacity
Calling population size
M Markovian (exponential interarrival times, Poisson number of arrivals)
D Deterministic
Ek Erlang with shape parameter k
G General
FCFS first come, first served
LCFS last come, first served
SIRO service in random order
GD general discipline
Notes:

5. Labeling Convention (Kendall-Lee)
Examples: M/M/1 M/M/5 M/G/1 M/M/3/LCFS Ek/G/2//10 M/M/1///100

6. Terminology and Notation
State of the system Number of customers in the queueing system (includes customers in service)
Queue length Number of customers waiting for service
= State of the system - number of customers being served
N(t) = State of the system at time t, t ≥ 0
Pn(t) = Probability that exactly n customers are in the queueing system at time t

7. Terminology and Notation
n = Mean arrival rate (expected # arrivals per unit time) of new customers when n customers are in the system
s = Number of servers (parallel service channels)
n = Mean service rate for overall system (expected # customers completing service per unit time) when n customers are in the system
Note: n represents the combined rate at which all busy servers (those serving customers) achieve service completion.

8. Terminology and Notation
When arrival and service rates are constant for all n,
 = mean arrival rate (expected # arrivals per unit time)
 = mean service rate for a busy server
1/ = expected interarrival time
1/ = expected service time
 = /s = utilization factor for the service facility = expected fraction of time the system’s service capacity (s) is being utilized by arriving customers ()

9. Terminology and Notation Steady State
When the system is in steady state, then
Pn = probability that exactly n customers are in the queueing system
L = expected number of customers in queueing system
= …
Lq = expected queue length (excludes customers being served)
= …

10. Terminology and Notation Steady State
When the system is in steady state, then
 = waiting time in system (includes service time) for each individual customer
W = E[]
q = waiting time in queue (excludes service time) for each individual customer
Wq = E[q]

11. Little’s Formula
Demonstrates the relationships between L, W, Lq, and Wq
Assume n= and n= (arrival and service rates constant for all n)
In a steady-state queue,
Intuitive Explanation:

12. Little’s Formula (continued)
This relationship also holds true for  (expected arrival rate) when n are not equal.
Recall, Pn is the steady state probability of having n customers in the system

13. Heading toward M/M/s
The most widely studied queueing models are of the form M/M/s (s=1,2,…)
What kind of arrival and service distributions does this model assume?
Reviewing the exponential distribution….
If T ~ exponential(α), then
A picture of the distribution:

14. Exponential Distribution Reviewed
If T ~ exponential(), then
E[T] = ______
Var(T) = ______

15. Property 1 Strictly Decreasing
The pdf of exponential, fT(t), is a strictly decreasing function
A picture of the pdf:
fT(t)  t

16. Property 2 Memoryless The exponential distribution has lack of memory
i.e. P(T > t+s | T > s) = P(T > t) for all s, t ≥ 0.
Example:
P(T > 15 min | T > 5 min) = P(T > 10 min)
The probability distribution has no memory of what has already occurred.

17. Property 2 Memoryless Prove the memoryless property
Is this assumption reasonable?
For interarrival times
For service times

18. Property 3 Minimum of Exponentials
The minimum of several independent exponential random variables has an exponential distribution
If T1, T2, …, Tn are independent r.v.s, Ti ~ expon(i) and
U = min(T1, T2, …, Tn),
U ~expon( )
Example:
If there are n servers, each with exponential service times with mean , then U = time until next service completion ~ expon(____)

19. Property 4 Poisson and Exponential
If the time between events, Xn ~ expon(), then the number of events occurring by time t, N(t) ~ Poisson(t)
Note:
E[X(t)] = αt, thus the expected number of events per unit time is α

20. Property 5 Proportionality
For all positive values of t, and for small t,
P(T ≤ t+t | T > t) ≈ t
i.e. the probability of an event in interval t is proportional to the length of that interval

21. Property 6 Aggregation and Disaggregation
The process is unaffected by aggregation and disaggregation
Aggregation
Disaggregation
N1 ~ Poisson(1)
N1 ~ Poisson(p1) p1
N2 ~ Poisson(2)
N2 ~ Poisson(p2)
N ~ Poisson()
N ~ Poisson() p2 … …
 = 1+2+…+k pk
Nk ~ Poisson(k)
Nk ~ Poisson(pk)
Note: p1+p2+…+pk=1

22. Back to Queueing
Remember that N(t), t ≥ 0, describes the state of the system: The number of customers in the queueing system at time t
We wish to analyze the distribution of N(t) in steady state

23. Birth-and-Death Processes
If the queueing system is M/M/…/…/…/…, N(t) is a birth-and-death process
A birth-and-death process either increases by 1 (birth), or decreases by 1 (death)
General assumptions of birth-and-death processes:
1. Given N(t) = n, the probability distribution of the time remaining until the next birth is exponential with parameter λn
2. Given N(t) = n, the probability distribution of the time remaining until the next death is exponential with parameter μn
3. Only one birth or death can occur at a time

24. Rate Diagrams

25. Steady-State Balance Equations

26. M/M/1 Queueing System
Simplest queueing system based on birth-and-death
We define  = mean arrival rate  = mean service rate  =  /  = utilization ratio
We require  <  , that is  < 1 in order to have a steady state
Why?
Rate Diagram      1 2 3 4 …     

27. M/M/1 Queueing System Steady-State Probabilities
Calculate Pn, n = 0, 1, 2, …

28. M/M/1 Queueing System L, Lq, W, Wq
Calculate L, Lq, W, Wq

29. M/M/1 Example: ER Emergency cases arrive independently at random
Assume arrivals follow a Poisson input process (exponential interarrival times) and that the time spent with the ER doctor is exponentially distributed
Average arrival rate = 1 patient every ½ hour =
Average service time = 20 minutes to treat each patient =
Utilization
 =

30. M/M/1 Example: ER Questions
What is the…
probability that the doctor is idle?
probability that there are n patients?
expected number of patients in the ER?
expected number of patients waiting for the doctor?
expected time in the ER?
expected waiting time?
probability that there are at least two patients waiting?
probability that a patient waits more than 30 minutes?

31. What conclusions can you draw from your results?
Car Wash Example
Consider the following 3 car washes
Suppose cars arrive according to a Poisson input process and service follows an exponential distribution
Fill in the following table   L Lq W Wq P0
Car Wash A
0.1 car/min
0.5 car/min
Car Wash B
0.11 car/min
Car Wash C
What conclusions can you draw from your results?

32. M/M/s Queueing System
We define  = mean arrival rate  = mean service rate s = number of servers (s > 1)  =  / s = utilization ratio
We require  < s , that is  < 1 in order to have a steady state
Rate Diagram 1 2 3 4 …

33. M/M/s Queueing System Steady-State Probabilities
and
Pn = CnP0
where

34. M/M/s Queueing System L, Lq, W, Wq
How to find L? W? Wq?

35. M/M/s Example: A Better ER
As before, we have
Average arrival rate = 1 patient every ½ hour  = 2 patients per hour
Average service time = 20 minutes to treat each patient  = 3 patients per hour
Now we have 2 doctors s =
Utilization
 =

36. M/M/s Example: ER Questions
What is the…
probability that both doctors are idle?
probability that exactly one doctor is idle?
probability that there are n patients?
expected number of patients in the ER?
expected number of patients waiting for a doctor?
expected time in the ER?
expected waiting time?
probability that there are at least two patients waiting?
probability that a patient waits more than 30 minutes?

37. Performance Measurements s = 1 s = 2ρ
2/3
1/3 L 2
3/4 Lq
4/3
1/12 W
1 hr
3/8 hr Wq
2/3 hr
1/24 hr
P(at least two patients waiting in queue)
0.296
0.0185
P(a patient waits more than 30 minutes)
0.404
0.022

38. Travel Agency Example
Suppose customers arrive at a travel agency according to a Poisson input process and service times have an exponential distribution
We are given
= .10/minute = 1 customer every 10 minutes
 = .08/minute = 8 customers every 100 minutes
If there were only one server, what would happen?
How many servers would you recommend?

42. Single Queue vs. Multiple Queues
Would you ever want to keep separate queues for separate servers?
vs.
Single queue
Multiple queues

43. Bank Example Suppose we have two tellers at a bank
Compare the single server and multiple server models
Assume  = 2,  = 3 L Lq W Wq P0

44. Bank Example Continued
Suppose we now have 3 tellers
Again, compare the two models

45. M/M/s//K Queueing Model (Finite Queue Variation of M/M/s)
Now suppose the system has a maximum capacity, K
We will still consider s servers
Assuming s ≤ K, the maximum queue capacity is K – s
List some applications for this model:
Draw the rate diagram for this problem:

46. M/M/s//K Queueing Model (Finite Queue Variation of M/M/s)
Rate Diagram 1 2 3 4 …
Balance equations: Rate In = Rate Out

47. M/M/s//K Queueing Model (Finite Queue Variation of M/M/s)
Solving the balance equations, we get the following steady state probabilities:
Verify that these equations match those given in the text for the single server case (M/M/1//K)

48. M/M/s//K Queueing Model (Finite Queue Variation of M/M/s)
To find W and Wq:
Although L ≠ lW and Lq ≠ lWq because ln is not equal for all n,
and
where

49. M/M/s///N Queueing Model (Finite Calling Population Variation of M/M/s)
Now suppose the calling population is finite
We will still consider s servers
Assuming s ≤ K, the maximum queue capacity is K – s
List some applications for this model:
Draw the rate diagram for this problem:

50. M/M/s///N Queueing Model (Finite Calling Population Variation of M/M/s)
Rate Diagram 1 2 3 4 …
Balance equations: Rate In = Rate Out

51. M/M/s///N Results

52. Queueing Models with Nonexponential Distributions
M/G/1 Model
Poisson input process, general service time distribution with mean 1/ and variance 2
Assume  = / < 1
Results

53. Queueing Models with Nonexponential Distributions
M/Ek/1 Model
Erlang: Sum of exponentials
Think it would be useful?
Can readily apply the formulae on previous slide where
Other models
M/D/1
Ek/M/1
etc

54. Application of Queueing Theory
We can use the results for the queueing models when making decisions on design and/or operations
Some decisions that we can address
Number of servers
Efficiency of the servers
Number of queues
Amount of waiting space in the queue
Queueing disciplines

55. Number of Servers
Suppose we want to find the number of servers that minimizes the expected total cost, E[TC]
Expected Total Cost = Expected Service Cost + Expected Waiting Cost (E[TC]= E[SC] + E[WC])
How do these costs change as the number of servers change?
Expected cost
Number of servers

56. Repair Person Example
SimInc has 10 machines that break down frequently and 8 operators
The time between breakdowns ~ Exponential, mean 20 days
The time to repair a machine ~ Exponential, mean 2 days
Currently SimInc employs 1 repair person and is considering hiring a second
Costs:
Each repair person costs $280/day
Lost profit due to less than 8 operating machines: $400/day for each machine that is down
Objective: Minimize total cost
Should SimInc hire the additional repair person?

57. Repair Person Example Problem Parameters
What type of problem is this?
M/M/1
M/M/s
M/M/s/K
M/G/1
M/M/s finite calling population
M/Ek/1
M/D/1
What are the values of  and ?

58. Repair Person Example Rate Diagrams
Draw the rate diagram for the single-server and two-server case
Single server 1 2 3 4 … 8 9 10
Two servers 1 2 3 4 … 8 9 10
Expected service cost (per day) = E[SC] =
Expected waiting cost (per day) = E[WC] =

59. Repair Person Example Steady-State Probabilities
Write the balance equations for each case
How to find E[WC] for s=1? s=2?

60. Repair Person Example E[WC] Calculations
N=n
g(n)
s=1
s=2 Pn
g(n) Pn
0.271
0.433 1
0.217
0.346 2
0.173
0.139 3
400 56
0.055 24 4
800
0.097 78
0.019 16 5
1200
0.058 70
0.006 8 6
1600
0.029 46
0.001 7
2000
0.012
0.0003
2400
0.003 9
2800
0.0007 10
3200
E[WC]
$281/day
$48/day

61. Repair Person Example Results
We get the following results s
E[SC]:
E[WC]:
E[TC]: 1
$280/day
$281/day
$561/day 2
$560/day
$48/day
$608/day
≥ 3
≥ $840/day
≥ $0/day
What should SimInc do?

62. Supercomputer Example
Emerald University has plans to lease a supercomputer
They have two options
Supercomputer
Mean number of jobs per day
Cost per day
MBI
30 jobs/day
$5,000/day
CRAB
25 jobs/day
$3,750/day
Students and faculty jobs are submitted on average of 20 jobs/day, distributed Poisson i.e. Time between submissions ~ __________
Which computer should Emerald University lease?

63. Supercomputer Example Waiting Cost Function
Assume the waiting cost is not linear: h() = 500   2 ( = waiting time in days)
What distribution do the waiting times follow?
What is the expected waiting cost, E[WC]?

64. Supercomputer Example Results
Next incorporate the leasing cost to determine the expected total cost, E[TC]
Which computer should the university lease?