1. 1 Congestion Control and Traffic Management in High Speed Networks Carey Williamson University of Calgary

2. 2 Introduction l The goal of congestion control is to regulate traffic flow in the network in order to avoid saturating or overloading intermediate nodes in the network

3. 3 Congestion: Effects l Congestion is undesirable because it can cause: Increased delay, due to queueing within the network Packet loss, due to buffer overflow Reduced throughput, due to packet loss and retransmission l Analogy: “rush hour” traffic

4. 4 Congestion: Causes l The basic cause of congestion is that the input traffic demands exceed the capacity of the network l In typical packet switching networks, this can occur quite easily when: - output links are slower than inputs - multiple traffic sources competing for same output link at the same time

5. 5 Buffering: A Solution? l Buffering in switches can help alleviate short term or transient congestion problems, but... l Under sustained overload, buffers will still fill up, and packets will be lost –only defers the congestion problem l More buffers means more queuing delay –beyond a certain point, more buffering makes the congestion problem worse, because of increased delay and retransmission

6. 6 Motivation l The congestion control problem is even more acute in high speed networks l Faster link speeds mean that congestion can happen faster than before e.g., 64 kilobyte buffer @ 64 kbps: 8.2 seconds @ 10 Mbps: 52 milliseconds @ 1 Gbps: 0.52 milliseconds

7. 7 Motivation (Cont’d) l Buffer requirements increase with link speeds e.g., to store 1 second worth of traffic @ 64 kbps: 8 kilobytes @ 10 Mbps: 1.25 Mbytes @ 1 Gbps: 125 Mbytes

8. 8 Motivation (Cont’d) l Heterogeneity of link speeds - just because you add new high speed links to a network doesn’t mean that the old low speed links go away - interconnecting high speed and lower speed networks creates congestion problems at the point of interconnect

9. 9 Motivation (Cont’d) l Traffic is bursty - high peak-to-mean ratio, peak rates - e.g., data traffic: 10-to-1, 1-10 Mbps - e.g., video traffic: 20-to-1, 5-100 Mbps - can statistically multiplex several channels, but if too many are active at the same time, congestion is inevitable

10. 10 Motivation (Cont’d) l Reaction time is bounded by the propagation delay - in a high-speed wide-area network, the delay x bandwidth product is HUGE!!! - d x b tells you how many bits fit in the “pipe” between you and the receiver - by the time you realize that network is congested, you may have already sent another Mbit or more of data!!!

11. 11 Reactive versus Preventive l There are two fundamental approaches to congestion control: reactive approaches and preventive approaches l Reactive: feedback-based –attempt to detect congestion, or the onset of congestion, and take action to resolve the problem before things get worse l Preventive: reservation-based –prevent congestion from ever happening in the first place, by reserving resources

12. 12 Reactive versus Preventive (Cont’d) l Most of the Internet approaches are reactive schemes –TCP Slow Start –Random-Early-Detection (RED) Gateways –Source Quench l The large d x b product means that many of these approaches are not applicable to high speed networks l Most ATM congestion control strategies are preventive, reservation-based

13. 13 Congestion Control in ATM l When people discuss congestion control in the context of high speed ATM networks, they usually distinguish between call-level controls and cell-level controls

14. 14 Call-Level Control l An example of the call-level approach to congestion control is call admission control (to be discussed later this semester) l Tries to prevent congestion by not allowing new calls or connections into the network unless the network has sufficient capacity to support them

15. 15 Call-Level Control (Cont’d) l At time of call setup (connection establishment) you request the resources that you need for the duration of the call (e.g., bandwidth, buffers) l If available, your call proceeds l If not, your call is blocked l E.g., telephone network, busy signal

16. 16 Call-Level Control (Cont’d) l Tradeoff: aggressive vs conservative l Want to accept enough calls to have reasonably high network utilization, but don’t want to accept so many calls that you have a high probability of network congestion (which might compromise the QOS requirements that you are trying to meet)

17. 17 Call-Level Control (Cont’d) l Problems: Can be unfair - denial of service, long access delay Hard to specify resource requirements and QOS parameters precisely - may not know, or may cheat - congestion can still occur

18. 18 Cell-Level Control l Also called input rate control l Control the input rate of traffic sources to prevent, reduce, or control the level of congestion l Many possible mechanisms: Traffic shaping, traffic policing, UPC Leaky bucket (token bucket) Cell tagging (colouring), cell discarding Cell scheduling disciplines

19. 19 Congestion Control in ATM l There is actually a complete spectrum of traffic control functions, ranging from the very short-term (e.g., traffic shaping, cell discarding) to the very long-term (e.g., network provisioning) l See [Gilbert et al 1991]

20. 20 ATM Traffic Control Schemes Time Scale

21. 21 ATM Traffic Control Schemes Time Scale Short Term usec

22. 22 ATM Traffic Control Schemes Time Scale Short Term usec Long Term Months, years

23. 23 ATM Traffic Control Schemes Time Scale Cell Time Usage Parameter Control Priority Control Traffic Shaping Cell Discarding Propagation Delay Time Explicit Congestion Notification Fast Reservation Protocol Node to Node Flow Control Call Duration Admission Control Routing, Load Balancing Long Term Resource Provisioning

24. 24 ATM Traffic Control Schemes Time Scale Cell Time Usage Parameter Control Priority Control Traffic Shaping Cell Discarding

25. 25 ATM Traffic Control Schemes Time Scale Propagation Delay Time Explicit Congestion Notification Fast Reservation Protocol Node to Node Flow Control

26. 26 ATM Traffic Control Schemes Time Scale Call Duration Admission Control Routing, Load Balancing

27. 27 ATM Traffic Control Schemes Time Scale Long Term Resource Provisioning

28. 28 ATM Traffic Control Schemes Time Scale Cell Time Usage Parameter Control Priority Control Traffic Shaping Cell Discarding Propagation Delay Time Explicit Congestion Notification Fast Reservation Protocol Node to Node Flow Control Call Duration Admission Control Routing, Load Balancing Long Term Resource Provisioning

29. 29 ATM Traffic Control Schemes l Preventive controls: Resource provisioning Connection admission control Call routing and load balancing Usage parameter control Priority control Traffic shaping Fast reservation protocol

30. 30 ATM Traffic Control Schemes l Reactive controls: Adaptive admission control Call routing and load balancing Adaptive usage parameter control Explicit congestion notification (forward or backward) Node to node flow control Selective cell discarding

31. 31 Leaky Bucket l One of the cell-level control mechanisms that has been proposed is the leaky bucket (a.k.a. token bucket) l Has been proposed as a traffic policing mechanism for Usage Parameter Control (UPC), to check conformance of a source to its traffic descriptor l Can also be used as a traffic shaper

32. 32 Leaky Bucket (Cont’d) l Think of a bucket (pail) with a small hole in the bottom l You fill the bucket with water l Water drips out the bottom at a nice constant rate: drip, drip, drip...

33. 33 Leaky Bucket (Cont’d)

34. 34 Leaky Bucket (Cont’d) Bucket

35. 35 Leaky Bucket (Cont’d) Bucket Empty

36. 36 Leaky Bucket (Cont’d) Bucket Hole

37. 37 Leaky Bucket (Cont’d) Bucket Water Hole

38. 38 Leaky Bucket (Cont’d)

39. 39 Leaky Bucket (Cont’d) Drip

40. 40 Leaky Bucket (Cont’d)

41. 41 Leaky Bucket (Cont’d)

42. 42 Leaky Bucket (Cont’d)

43. 43 Leaky Bucket (Cont’d) Constant rate stream of drips, all nicely spaced, periodic

44. 44 Leaky Bucket (Cont’d) Constant rate stream of drips, all nicely spaced, periodic Storage area for drips waiting to go

45. 45 Leaky Bucket (Cont’d) l A leaky bucket flow control mechanism is then a software realization of this very simple idea l Packets (cells) waiting for transmission arrive according to some (perhaps unknown) arrival distribution l Tokens arrive periodically (deterministically) l Cell must have a token to enter network

46. 46 Leaky Bucket (Cont’d) Incoming Cells (generated by traffic source with rate X) Incoming Tokens at rate r tokens/sec + To Network

47. 47 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens at rate r tokens/sec + To Network 1 234 5

48. 48 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network 1234 5

49. 49 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network 1 2345

50. 50 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network 1 2 345

51. 51 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network 12345

52. 52 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network X 1 23 45

53. 53 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network XX 123 45

54. 54 Leaky Bucket (Cont’d) Incoming Cells Incoming Tokens + To Network XX 123

55. 55 Leaky Bucket (Cont’d) l Cells must obtain tokens in order to proceed into the network l If no token available, then cell is discarded l Constrains the rate at which cells enter the network to be the rate negotiated at the time of call setup l Shapes traffic, reduces burstiness

56. 56 Buffered Leaky Bucket l Arriving cells that find a token waiting can proceed directly into the network l Arriving cells that find no token ready must wait in queue for a token l Cells that arrive to a full queue are lost l Tokens that arrive to a full token pool are simply discarded

57. 57 Buffered Leaky Bucket Incoming Cells Incoming Tokens at rate r tokens/sec + To Network Queue of at most B waiting cells Pool of at most M waiting tokens

58. 58 Buffered Leaky Bucket (Cont’d) l Incoming cell rate: X l Token rate: r l If X > r, then cells wait in buffer until tokens are available Output traffic is r cells/sec, nicely paced l If X < r, then tokens always ready Output traffic is X (< r) l Use for traffic shaping or UPC

59. 59 Buffered Leaky Bucket (Cont’d) l A station can “save up” at most M tokens l Limits the maximum burst size in the network l Can send at most M cells back to back l B can be set to balance the tradeoff between cell loss and cell delay

60. 60 Leaky Bucket UPC l The token rate r is set based on the rate declared at the time of call setup l Makes sure that each source obeys rate that was used when the call admission decision was made (i.e., descriptor) l Can use “single leaky bucket” to police just the peak cell rate (PCR) l Can use “dual leaky bucket” to police both PCR and SCR

61. 61 Variations l There are several different variations of the basic leaky bucket concept described in the literature, such as the virtual leaky bucket, spacer, others l Basic idea: rather than strictly enforcing rates, allow senders to occasionally exceed their prescribed rate, as long as they mark or tag their extra cells

62. 62 Cell Marking Scheme l Uses leaky bucket to regulate cell transmissions as before, but rather than having cells wait for tokens when there are no tokens ready, the station can transmit the cell and mark it as a violation cell (i.e., cell colouring) l Green (CLP = 0) for cells that obey rate l Red (CLP = 1) for cells that don’t

63. 63 Cell Colouring (Cont’d) l If the network detects congestion at any point, then it does not hesitate to throw away red cells (CLP = 1), but always gives preference to green cells l Green cells must get through l Red cells get through only if there is spare capacity in the network l “No harm in trying!” principle

64. 64 Selective Cell Discard (SCD) l A cell-level control mechanism in ATM switches called selective cell discard can be implemented quite easily using a CLP threshold on each queue/buffer l Below the threshold, can accept both green and red cells l Beyond the threshold, can only accept green cells

65. 65 Selective Cell Discard (Cont’d) Buffer in an ATM switch

66. 66 Selective Cell Discard (Cont’d) Buffer in an ATM switch

67. 67 Selective Cell Discard (Cont’d) Buffer in an ATM switch Some cells waiting to go

68. 68 Selective Cell Discard (Cont’d) CLP Threshold Buffer in an ATM switch

69. 69 Selective Cell Discard (Cont’d) CLP Threshold

70. 70 Selective Cell Discard (Cont’d) CLP Threshold

71. 71 Selective Cell Discard (Cont’d) CLP Threshold

72. 72 Selective Cell Discard (Cont’d) CLP Threshold

73. 73 Selective Cell Discard (Cont’d) CLP Threshold

74. 74 Selective Cell Discard (Cont’d) CLP Threshold

75. 75 Selective Cell Discard (Cont’d) CLP Threshold

76. 76 Selective Cell Discard (Cont’d) CLP Threshold

77. 77 Selective Cell Discard (Cont’d) CLP Threshold

78. 78 Selective Cell Discard (Cont’d) CLP Threshold

79. 79 Selective Cell Discard (Cont’d) CLP Threshold

80. 80 Selective Cell Discard (Cont’d) CLP Threshold

81. 81 Selective Cell Discard (Cont’d) CLP Threshold

82. 82 Selective Cell Discard (Cont’d) CLP Threshold

83. 83 Selective Cell Discard (Cont’d) CLP Threshold

84. 84 Selective Cell Discard (Cont’d) CLP Threshold

85. 85 Selective Cell Discard (Cont’d) CLP Threshold

86. 86 Selective Cell Discard (Cont’d) CLP Threshold

87. 87 Selective Cell Discard (Cont’d) CLP Threshold

88. 88 Selective Cell Discard (Cont’d) CLP Threshold

89. 89 Explicit Congestion Notification l There are some proposals to use reactive congestion control approaches for end-to-end flow control in ATM l One of the mechanisms proposed is called Explicit Forward Congestion Notification (EFCN) (or EFCI, for Explicit Forward Congestion Indication)

90. 90 EFCI: Basic Operation l Switches can detect the onset of congestion (e.g., buffers filling up) l Switches set a control bit in cell headers to indicate this congestion condition l Sources react by reducing the volume of traffic that they are sending through that switch l Suitable for VBR or ABR traffic

91. 91 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch

92. 92 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch Buffer

93. 93 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch Occupied Unoccupied

94. 94 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

95. 95 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold Data Cell

96. 96 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

97. 97 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

98. 98 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

99. 99 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

100. 100 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

101. 101 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

102. 102 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

103. 103 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold !!!

104. 104 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold Ack Cell

105. 105 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

106. 106 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

107. 107 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

108. 108 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold !!!

109. 109 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

110. 110 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

111. 111 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

112. 112 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

113. 113 EFCI: Basic Operation (Cont’d) Traffic Source Traffic Sink Switch EFCI Threshold

114. 114 EFCI Issues l How to set EFCI threshold l What should sources do when EFCI signal is seen l What should sources do when no EFCI signal is seen l Forward versus backward notification l Effect of feedback delay l Delay x bandwidth product

115. 115 Summary l Congestion control in high speed ATM networks is a difficult problem l Lots of good ideas of how to do it, but no real standard (yet?) l Will likely require a combination of schemes at different time scales and for different classes of traffic l Lots more remains to be done