1. Delays in Packet Networks
Getting a Grip on
Delays in Packet Networks
Jorg Liebeherr Almut Burchard Dept. of ECE Dept. of Mathematics
University of Toronto

2. Packet Switch Fixed-capacity links
Variable delay due to waiting time in buffers
Delay depends on
Traffic
Scheduling

3. Traffic Arrivals
Peak rate
Frame size
Mean rate
Frame number

4. First-In-First-Out

5. Static Priority (SP) Blind Multiplexing (BMux):
All “other traffic” has higher priority

6. Earliest Deadline First (EDF)
Benchmark scheduling algorithm for meeting delay requirements

7. Network

8. Computing delays in such networks is notoriously hard …
Simplified Network
Computing delays in such networks is notoriously hard …

9. … but tempting
Over the last (almost) 20/15 years, we have worked on problems relating to network delays:
Worst-case delays
Scheduling vs. statistical multiplexing
Statistical bounds on end-to-end delays
Difficult traffic types
Scaling laws

10. Collaborators Domenico Ferrari Dallas Wrege Hui Zhang Ed Knightly
Robert Boorstyn
Chaiwat Oottamakorn
Stephen Patek
Chengzhi Li
Florin Ciucu
Yashar Ghiassi-Farrokhfal

11. Papers (relevant to this talk)
J.Liebeherr, D. E. Wrege, D. Ferrari, “Exact admission control for networks with a bounded delay service,” ACM/IEEE Trans. Netw. 4(6), 1996.
E. W. Knightly, D. E. Wrege, H. Zhang, J. Liebeherr, “Fundamental Limits and Tradeoffs of Providing Deterministic Guarantees to VBR Video Traffic,” ACM Sigmetrics, 1995.
R. Boorstyn, A. Burchard, J. Liebeherr, C. Oottamakorn. “Statistical Service Assurances for Packet Scheduling Algorithms”, IEEE JSAC, Dec
A. Burchard, J. Liebeherr, S. D. Patek, “A Min-Plus Calculus for End-to-end Statistical Service Guarantees,” IEEE Trans. on IT, 52(9), Sep
F. Ciucu, A. Burchard, J. Liebeherr, “A Network Service Curve Approach for the Stochastic Analysis of Networks”, ACM Sigmetrics 2005.
C. Li, A. Burchard, J. Liebeherr, “A Network Calculus with Effective Bandwidth,” ACM/IEEE Trans. on Networking, Dec
J. Liebeherr, A. Burchard, F. Ciucu, “Non-asymptotic Delay Bounds for Networks with Heavy-Tailed Traffic,” INFOCOM 2010.
J. Liebeherr, Y. Ghiassi-Farrokhfal, A. Burchard, “On the Impact of Link Scheduling on End-to-End Delays in Large Networks,” IEEE JSAC, May 2011.
J. Liebeherr, Y. Ghiassi-Farrokhfal, A. Burchard, “The Impact of Link Scheduling on Long Paths: Statistical Analysis and Optimal Bounds”, INFOCOM ’2011.,
A. Burchard, J. Liebeherr, F. Ciucu, “On Superlinear Scaling of Network Delays, ACM/IEEE Trans. Netw., to appear.

12. Disclaimer This talk makes a few simplifications
Please see papers for complete details

13. Traffic Description Traffic arrivals in time interval [s,t) is
Arrival function A
Traffic arrivals in time interval [s,t) is
Burstiness can be reduced by “shaping” traffic

14. Shaped Arrivals Traffic is shaped by an envelope such that:
Flow 1 . C
Flow N
Flows are shaped
Regulated
arrivals
Buffered Link
Traffic is shaped by an envelope such that:
Popular envelope: “token bucket”

15. What is the maximum number of shaped flows with delay requirements that can be put on a single buffered link?
Link capacity C
Each flows j has
arrival function Aj
envelope Ej
delay requirement dj

16. Delay Analysis of Schedulers
Consider a link scheduler with rate C
Consider arrival from flow i at t with t+di:
Limit
(Scheduler Dependent)
Deadline of
Tagged arrival
Arrivals from flow j
Tagged
arrival
Tagged arrival departs by if

17. Delay Analysis of Schedulers
Arrivals from flow j
with
FIFO:
Static Priority:
EDF:

18. Schedulability Condition
We have:
Therefore:
An arrival from class i never has a delay bound violation if
Condition is tight, when Ej is concave

19. Numerical Result (Sigmetrics 1995)
C = 45 Mbps
MPEG 1 traces:
Lecture:
d = 30 msec
Movie (Jurassic Park):
d = 50 msec
EDF
Static Priority (SP)
Peak Rate
strong effective
envelopes
Type 1 flows

20. Expected case
Deterministic
worst-case
Probable worst-case

21. Statistical Multiplexing Gain
Worst-case arrivals
Arrivals
Flow 1
Flow 2
Flow 3
Time
Backlog
Worst-case backlog
With statistical multiplexing
Arrivals
Flow 1
Flow 2
Flow 3
Time
Backlog
Backlog

22. Statistical Multiplexing Gain
Statistical multiplexing gain is the raison d’être for packet networks.

23. What is the maximum number of flows with delay requirements that can be put on a buffered link and considering statistical multiplexing?
Arrivals are random processes
Stationarity: The are stationary random processes
Independence: The and are stochastically independent

24. Envelopes for random arrivals
Statistical envelope bounds arrival from flow j with high certainty
Statistical envelope :
Statistical sample path envelope :
Statistical envelopes are non-random functions

25. Aggregating arrivals Arrivals from group of flows:
with deterministic envelopes:
with statistical envelopes:

26. Statistical envelope for group of indepenent (shaped) flows
Exploit independence and extract statistical multiplexing gain when calculating
For example, using the Chernoff Bound, we can obtain

27. Statistical vs. Deterministic Envelope Envelopes
(JSAC 2000)
Type 1 flows:
P =1.5 Mbps
r = .15 Mbps
s =95400 bits
Type 2 flows:
P = 6 Mbps
s = bits
statistical
envelopes
Type 1 flows

28. Statistical vs. Deterministic Envelope Envelopes (JSAC 2000)
Traffic rate at t = 50 ms Type 1 flows

29. Scheduling Algorithms
Work-conserving scheduler that serves Q classes
Class-q has delay bound dq
D-scheduling algorithm .
Scheduler
Deterministic Service
Never a delay bound violation if:
Statistical Service
Delay bound violation with if:

30. Statistical Multiplexing vs. Scheduling (JSAC 2000)
Example: MPEG videos with delay constraints at C= 622 Mbps
Deterministic service vs. statistical service (e = 10-6)
dterminator=100 ms dlamb=10 ms
Thick lines: EDF Scheduling Dashed lines: SP scheduling
Statistical multiplexing makes a big difference
Scheduling has small impact

31. More interesting traffic types
So far: Traffic of each flow was shaped
Next:
On-Off traffic
Fraction Brownian Motion (FBM) traffic
Approach:
Exploit literature on Effective Bandwidth
Derived for many traffic types
Peak rate
Mean rate
effective bandwidth

32. Statistical Envelopes and Effective Bandwidth
Effective Bandwidth (Kelly 1996)
Given , an effective envelope is given by

33. Different Traffic Types (ToN 2007)
Comparisons of statistical service guarantees for different schedulers and traffic types
Schedulers:
SP- Static Priority EDF – Earliest Deadline First GPS – Generalized Processor Sharing
Traffic:
Regulated – leaky bucket On-Off – On-off source FBM – Fractional Brownian Motion
C= 100 Mbps, e = 10-6

34. Delays on a path with multiple nodes:
Impact of Statistical Multiplexing
Role of Scheduling
How do delays scale with path length?
Does scheduling still matter in a large network?

35. Deterministic Network Calculus (1/3)
Systems theory for networks in (min,+) algebra developed by Rene Cruz, C. S. Chang, JY LeBoudec (1990’s)
Service curve S characterizes node
Used to obtain worst-case bounds on delay and backlog

36. Deterministic Network Calculus (2/3)
Worst-case view of
arrivals:
service :
Implies worst-case bounds
backlog:
delay :
(min,+) algebra operators
Convolution:
Deconvolution:

37. Deterministic Network Calculus (3/3)
Main result:
If describes the service at each node, then
describes the service given by the network as a whole.
Sender
Receiver
Sender
Receiver S3 S1 S3 S2
A (t)
network S1
D (t)
network S2
A (t)
network
D (t)
network
S = S * S * S
network 1 2 3
S = S * S * S
network 1 2 3
Receiver
Sender
Receiver
Sender S
network S
network
D (t)
network
A (t)
network
D (t)
network
A (t)
network

38. Stochastic Network Calculus
Probabilistic view on arrivals and service
Statistical Sample Path Envelope
Statistical Service Curve
Results on performance bounds carry over, e.g.:
Backlog Bound

39. Stochastic Network Calculus
Hard problem: Find so that
Technical difficulty:
is a random variable!

40. Statistical Network Service Curve (Sigmetrics 2005)
Notation:
Theorem: If are statistical service curves, then for any :
is a statistical network service curve with some finite violation probability.

41. EBB model Traffic with Exponentially Bounded Burstiness (EBB)
Sample path statistical envelope obtained via union bound

42. Example: Scaling of Delay Bounds
Traffic is Markov Modulated On-Off Traffic (EBB model)
All links have capacity C
Same cross-traffic (not independent!) at each node
Through flow has lower priority:

43. Example: Scaling of Delay Bounds
Two methods to compute delay bounds:
Add per-node bounds: Compute delay bounds at each node and sum up
Network service curve: Compute single-node delay bound with statistical network service curve

44. Example: Scaling of Delay Bounds (Sigmetrics 2005)
Peak rate: P = 1.5 Mbps Average rate: r = 0.15 Mbps
T= 1/m + 1/l = 10 msec
C = 100 Mbps
Cross traffic = through traffic
e = 10-9
Addition of per-node bounds grows O(H3)
Network service curve bounds grow O(H log H)

45. Result: Lower Bound on E2E Delay (Infocom 2007)
M/M/1 queues with identical exponential service at each node
Theorem: E2E delay satisfies for all
Corollary: -quantile of satisfies

46. Numerical examples Tandem network without cross traffic Node capacity:
Arrivals are compound Poisson process
Packets arrival rate:
Packet size:
Utilization:

47. Upper and Lower Bounds on E2E Delays (Infocom 2007)

48. Superlinear Scaling of Network Delays
For traffic satisfying “Exponential Bounded Burstiness”, E2E delays follow a scaling law of
This is different than predicted by
… worst-case analysis
… networks satisfying “Kleinrock’s independence assumption”

49. How do end-to-end delay bounds look like for different schedulers?
Back to scheduling …
So far: Through traffic has lowest priority and gets leftover capacity
 Leftover Service
or Blind Multiplexing
BMux C
How do end-to-end delay bounds look like for different schedulers?
Does link scheduling matter on long paths?

50. Service curves vs. schedulers (JSAC 2011)
How well can a service curve describe a scheduler?
For schedulers considered earlier, the following is ideal:
with indicator function and parameter

51. Example: End-to-End Bounds
Traffic is Markov Modulated On-Off Traffic (EBB model)
Fixed capacity link

52. Example: Deterministic E2E Delays (Infocom ‘11)
Peak rate: P = 1.5 Mbps Average rate: r = 0.15 Mbps
C = 100 Mbps
BMUX
EDF (delay-tolerant)
FIFO
EDF (delay intolerant

53. Example: Statistical E2E Delays (Infocom`11)
Peak rate: P = 1.5 Mbps Average rate: r = 0.15 Mbps
C = 100 Mbps
e = 10-9

54. Example: Statistical Output Burstiness (Infocom ‘11)
Peak rate: P = 1.5 Mbps Average rate: r = 0.15 Mbps
C = 100 Mbps
e = 10-9

55. Can we compute scaling of delays for nasty traffic ?

56. Heavy-Tailed Self-Similar Traffic
A heavy-tailed process satisfies
with
A self-similar process satisfies
Hurst Parameter

57. End-to-End Delays
Cross traffic
Cross traffic
Cross traffic
Node
Node …
Node H
Through traffic
Exponentially bounded traffic  (N log N) (Sigmetrics 2005, Infocom 2007)
End-to-end delay bound
Worst-case delays  (N)
(e.g., LeBoudec and Thiran 2000)
Number of nodes (N)

58. htts Traffic Envelope Heavy-tailed self-similar (htss) envelope:
Main difficulty: Backlog and delay bounds require sample path envelopes of the form
Key contribution (not shown): Derive sample path bound for htss traffic

59. Example: Node with Pareto Traffic (Infocom 2010)
Traffic parameters:
Node:
Capacity C=100 Mbps with packetizer
No cross traffic
Compared with:
Lower bound from Infocom 2007
Simulations

60. Example: Nodes with Pareto Traffic (End-to-end)
Parameters:
Compared with:
Lower bound from Infocom 2007
Simulation traces of 108 packets

61. Illustration of scaling bounds (Infocom 2010)
Upper Bound:
Lower Bound:
Bounds: a = 1.5
Upper Bound
Lower Bound
 (N log N)
 (N)

62. Summary of insights (1) (2) (3) (4), (5)
1995
2000
2005
2010
(1)
(2)
(3)
(4), (5)
Satisfying delay bounds does not require peak rate allocation for complex traffic
Statistical multiplexing gain dominates gain due to link scheduling
scaling law of end-to-end delays
New laws for heavy-tailed traffic
Link scheduling plays a role on long path

63. Example: Pareto Traffic
Size of i–th arrival:
Arrivals are evenly spaced with gap :
With Generalized Central Limit Theorem …
… and tail bound
... we get htss envelope
a-stable distribution

64. Example: Envelopes for Pareto Traffic (Infocom 2010)
Parameters:
Comparison of envelopes:
htss GCLT envelope
Average rate
Trace-based
deterministic envelope
htts trace envelope

65. Single Node Delay Bound
htss envelope:
ht service curve:
Delay bound:

66. Conclusions Stochastic network calculus Broad Yes Requirements
Queueing networks
Effective bandwidth
Network calculus
Traffic classes (incl. self-similar, heavy-tailed)
Limited
Broad
Broad (but loose)
Scheduling No
Yes
QoS (bounds on loss, throughput delay)
Very limited
Loss, throughput
Deterministic
Statistical Multiplexing
Some