1. Queueing Theory

2. Overview Introduction Basic Queue Properties Stochastic Processes
Kendall Notation
Little’s Law
Stochastic Processes
Birth-Death Process
Markov Process
Queueing Models

3. Why Queueing Theory Mathematical properties of lines or “queues”
Useful to understand delays and congestions in computer networks
Tool for packet switched networks
Limited service/processing capability
Probabilistic arrivals
Server
Departures
Queue
Customers
(Arrivals)
Example Queueing System

4. Analysis
Analysis of the queue can help identify numerous transient and steady state properties about the system, including
# of customers in system/queue
Response time
Wait time
Utilization
Throughput

5. Queue Properties Arrival process Service patterns Number of servers
The rate and distribution that customers arrive to the system
Service patterns
The rate and distribution at which the system can process customers
Number of servers
Queueing discipline
First in-First out, Last in-first out, Round Robin

6. Queue System Notation A/B/X/Y/Z Kendal Notation
Interarrival time distribution
M – Exponential
D – Deterministic
G – General
# of parallel service channels
Queue discipline
FIFO – First in first out
LIFO – Last in first out
RSS – Random selection of service
GD – General discipline
(Default = FIFO)
A/B/X/Y/Z
Service time distribution
M – Exponential
D – Deterministic
G – General
Capacity of the
System
(Default = infinite)

7. Queue Performance Parameters
T – Time in system, W = E[T]
Time in the
system
Tq – Time in queue
(Wq = E[Tq])
S – Service time
(1/μ = E[S])
Arrival rate
(λ)
Server
Departures
Queue
Nq – # customers in queue
Ns – # customers in service
# customers in
system
Lq – avg. # customers in queue
N – # customers in system
L – avg. # customers in system

8. Queue Stability μ – service rate λ – arrival rate
λ/μ - traffic intensity
λ/μ < 1 for stable queue
λ/μ = .1 – light load
λ/μ = .5 – moderate load
λ/μ = .9 – heavy load

9. Little’s Law Intuitively, longer service times equals longer queues
Average number of customers in a queueing system equals the arrival rate of new customers times the customer service rate
Intuitively, longer service times equals longer queues
Examples
Traffic jams occur with accidents, bad weather
Jimmy John’s has less seating than Zoe’s

10. Little’s Law
Your computer networking professor receives 30 s per day, on average he has 15 unchecked messages, how long until he responds to your ?
L= λW
W = L/λ = 15/30 = .5 day
A switch receives 100 packets every second, the switch can process each packet in 8ms, what’s the number of packets in system?
L = λ/W
L= 100 pps x .008sec = 8 packets

11. Little’s Law Derivation4 3 2 1 N
Time t1 t2 t3 t4 t5 t6 t7 T
Show:
LT = WNc
Nc/T – arrival rate

12. G/G/c General Properties
Useful Equations (c = number of servers):
L = λW [Little’s Law (also Lq = λWq)]
r = λ/μ [work load rate]
ρ = λ/cμ [utilization/traffic intensity]
p0 = 1 – ρ [probability system is empty]

13. Utilization vs Queueing Delay
Avg. Queuing Delay
[W = 1/(μ-λ)]
Utilization
[ρ = λ/μ]
Can’t fully utilize network without long delays!!!!

14. Stochastic Processes

15. Stochastic Process
Probability process that takes random values, X(t)=x(t1),…,x(tn) for times t1,….,tn
Can be
Discrete-time
T = {0, 1, 2, ….}
Example: coin flip
Continuous-time
T = {0< t < ∞}
Example: stock market, weather

16. Probability Review Bernoulli trial
Random experiment with only two outcomes
Example, flip of coin (heads and tails)
If I flip 5 coins, what is the probability of 3 heads?
Binomial distribution
Possible combinations of k events
Probability of n-k
non-events
Probability of k events

17. Poisson Distribution
Assume arrival times of some event follow an exponential distribution
X(t) for t≥0 represents the number of arrivals up to time period t
px(t) = probability x arrivals in time t

18. Poisson Distribution Example: Probability of seeing exactly 5 packets
Assume 5 (λ=5) packets arrive per second
Probability of seeing exactly 5 packets
p(5)
Probability of seeing less than 10 packets in a second
1-[p(0) + p(1) + … + p(10)]
P(0)
.007
P(1)
.034
P(2)
.081
P(3)
.135
P(4)
.175
P(5)
P(6)
.141
P(7)
.101
P(8)
.061
P(9)
P(10)
.013

19. Poisson Process Superposition Decompositionλ1 λ2 λ
Superposition
Multiple Poisson processes aggregate to Poisson process with higher rate
Decomposition
Single Poisson process decomposes to multiple lower rate Poisson processes . . . λn λ1 λ2 λ . . . λn

20. Exponential Distribution
Used to model
Arrival rate
Service rate
Memoryless (Markov) property
Pr(T > s+t |T > s)= Pr(T > t)
λ=1

21. Markov Process Discrete or continuous process where Classified by:
Pr(Xn = xn |Xn-1 = xn-1, Xn-2 = xn-2, … ,X0 = x0) =
Pr(Xn = xn |Xn-1 = xn-1)
Memoryless process
Present state only the precious state, not those earlier
Classified by:
Index set (discrete, continuous)
State space
Markov Chain – discrete
Markov Process – continuous

22. Birth-Death Process (BDP)
Continuous time Markov chain
State n represents size of population
pn is probability system in state n
Transition types
λi – birth rate, moves system from state n to n+1
μi – death rate, moves system from n to n-1 λ0 λ1 λ2 λk 1 2 … k μ1 μ2 μ3
μk-1

23. BDP Probabilities Flow balance
pn is steady state probability of system being in state n
Steady State Probability

24. Queue Models

25. Types of Queues M/M/1 M/M/c M/M/c/m M/G/1
Single queue and single server
M/M/c
Single queue, c servers
M/M/c/m
Single queue, c servers, m buffer size
M/G/1
General service distribution

26. Single Server Queue M/M/1 Single queue and single server
Customer arrival –
Exponentially distributed with λ
Service time
Exponential distribution with μ
Server
Departures
Queue
Customers
(Arrivals)

27. M/M/1 Properties Birth-death process where: Flow equations: λ λ λ λ 11 2 … k μ μ μ μ

28. M/M/1 Probabilities
Steady State Probabilities for M/M/1

29. M/M/1 Properties L – average number of events in system
W – average time spent in system
Use Little’s Law (L=λW)

30. Example Assume a network: What is average # packets in system?
Receives packets at 100pps (λ=100)
Can process 200 pps (μ=200)
What is average # packets in system?
What is average time a packet is in system

31. Example Assume a network where:
Packet sent at 1000pps
Average packet size is 1000bits
Question: To ensure average delay less than 50ms, what should be link speed?
10ms?

32. Statistical Multiplexing vs FDM/TDM
Network has m users, each send packets at λ/m pps
What’s the average delay?
Statistical Multiplexing
Users share single network which can send μ pps
FDM/TDM
Users all allocated μ/m of network bandwidth
Essentially m independent M/M/1 queues
Usually better to have one big server/network!!!!

33. Multiple Server Queue M/M/c Single queue with c servers λ/c Server 1 λ. . .
λ/c
Departures
Server c
Queue
Customers
(Arrivals)

34. M/M/c Properties Birth/death rates: Utilization: λ λ λ λ λ 1 2 … c c+11 2 … c
c+1 μ 2μ 3μ cμ cμ

35. M/M/c Probabilities
Steady state probabilities:

36. M/M/c Properties
Average time in system:
Average number in system:

37. Examples Assume a network: What is average # packets in system?
Receives packets at 100pps (λ=100)
Two processor computes 100 pps
(μ=100, c=2)
What is average # packets in system?
What is average time a packet is in system