Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture P. J. García 1, J. Flich 2, J. Duato 2, I. Johnson 3, F. J. Quiles 1,

Similar presentations


Presentation on theme: "Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture P. J. García 1, J. Flich 2, J. Duato 2, I. Johnson 3, F. J. Quiles 1,"— Presentation transcript:

1 Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture P. J. García 1, J. Flich 2, J. Duato 2, I. Johnson 3, F. J. Quiles 1, F. Naven 3 2 Technical University of Valencia Valencia, Spain 3 Xyratex Havant, UK 1 University of Castilla-La Mancha Albacete, Spain HiPEAC November - 18 NovemberBarcelona, Spain

2 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 2 Outline Introduction Congestion trees and HOL blocking HOL blocking elimination techniques Traditional view of congestion trees Different dynamics of congestion trees RECN improvements Performance evaluation Conclusions

3 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 3 Introduction High-speed interconnection networks: Myrinet, Infiniband, Quadrics, Advanced Switching… Main features: High bandwidth, Low latencies Additional features: Lossless networks, Flexible topology Cost and power consumption considerations recommend working close to the saturation point Network performance may be affected by congestion

4 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 4 Contention: Several packets request the same output port One makes progress, the others wait Congestion: Persistent contention It is quickly propagated by flow control (lossless nets), forming congestion trees Network performance degrades dramatically!!! Congestion trees and HOL blocking Head of line (HOL) blocking: When the first packet in a queue is blocked, any other packet in the same queue is also blocked, even if it will request available resources WHY?

5 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 5 Congestion trees and HOL blocking Network contention

6 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 6 Congestion trees and HOL blocking Persistent network contention

7 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 7 Congestion trees and HOL blocking Persistent network contention Flow control

8 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 8 Congestion trees and HOL blocking Persistent network contention Congestion propagates

9 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 9 Congestion trees and HOL blocking Congestion tree root Congestion tree leaf Congestion tree leaf Congestion tree branch Congestion tree branch

10 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 10 Congestion trees and HOL blocking Congestion trees introduce HOL blocking, and this may degrade network performance dramatically 33% HOL 33% 33% 100% 33% 100%

11 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 11 HOL blocking elimination/reduction techniques DAMQs and Virtual Channels Different buffers for different flows VOQ (Virtual Output Queues) VOQ at switch level: A separate queue at every input port for every output port VOQ at network level: A separate queue at every input port for every destination Credit Flow Controlled ATM Handles congestion at network outputs only A separate queue at every output port for every destination In general, these techniques try to separate different flows of packets in order to avoid HOL blocking:

12 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 12 RECN: Regional Explicit Congestion Notification RECN is a new efficient and scalable congestion management technique Basic ideas: The real problem is not the congestion, but its negative effects (HOL blocking) By eliminating HOL blocking, congestion becomes harmless Non-congested flows do not introduce significant HOL blocking HOL blocking elimination: Packets belonging to congested flows are stored in specific Set Aside Queues (SAQs) Packets belonging to non-congested flows are stored in a “common” queue Implementation requirements: Deterministic source routing A reduced number of SAQs per port, controlled by a CAM

13 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 13 How RECN works RECN basic procedure: Congested points are detected in any egress switch port of the network The routes to detected congested points are progressively notified to ingress and egress ports crossed by congested flows After receiving a notification, a port allocates a SAQ for the detected congested point A packet arriving to a port will be stored in a SAQ if it will pass through the congested point associated to that SAQ A packet arriving at a port will be stored in the common (“cold”) queue if its route does not match any SAQ SAQs can be deallocated, and later allocated for other congested points

14 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 14 A congestion point forms How RECN Works

15 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 15 How RECN Works Cold queue fills over a threshold

16 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 16 How RECN Works

17 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 17 How RECN Works Internal notification to each input port sending packets to the output port

18 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 18 How RECN Works

19 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 19 How RECN Works Input ports allocate a new SAQ for packets addressed to the congested output port

20 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 20 How RECN Works

21 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 21 How RECN Works Notification sent when the SAQ fills over a threshold

22 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 22 How RECN Works

23 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 23 How RECN Works A new SAQ allocated for the congested port at each output port

24 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 24 How RECN Works Internal notification when the SAQ fills over A threshold

25 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 25 How RECN Works The input port allocates A new SAQ

26 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 26 How RECN Works At the end, the congestion tree builds and is mapped entirely onto SAQs

27 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 27 Traditional view of congestion trees Traditional ideas about congestion trees growth: Congestion propagates from the root to the leaves Congestion first appears at egress sides This is not always true: Congestion trees may evolve in several ways The effectiveness of HOL blocking elimination techniques may drop if they do not consider congestion tree dynamics

28 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 28 Different dynamics of congestion trees Effect of switch architecture (I): Switch speedup may vary for different technologies Depending on switch speedup, congestion may appear at ingress or egress sides No speedup switch Full rate injection Congestion Switch speedup: 2 Congestion Full rate injection Congestion Switch speedup: 2 Full rate injection

29 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 29 Switch speedup: 2 All the sources start injection simultaneously Different dynamics of congestion trees Effect of switch architecture (II): Several congested points may appear both at ingress or egress sides along the branches of a congestion tree

30 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 30 Different dynamics of congestion trees Impact of traffic patterns (I): Depending on traffic patterns, the congestion tree root may “move” downstream Switch speedup: 2 Solid flows appear first, dashed ones later

31 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 31 Different dynamics of congestion trees Impact of traffic patterns (II): Different congestion trees may merge, even when the involved packets have different destinations Switch speedup: 2 Solid flows appear first, dashed ones later

32 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 32 Different dynamics of congestion trees Impact of traffic patterns (III): Different congestion trees may overlap without merging Switch speedup: 2 Solid flows appear first, dashed ones later

33 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 33 Different dynamics of congestion trees Impact of traffic patterns (IV): A congestion tree root may also move upstream Switch speedup: 2 dashed flow disappears first

34 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 34 Impact of congestion dynamics on RECN Original (“Basic”) RECN: Congestion is detected only at egress ports In order to keep in-order delivery of packets, no SAQ is allocated if it would be more specific than an existing one Wrong and delayed detection when congestion appears at ingress sides The downstream movement of the root is not detected

35 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 35 Impact of congestion dynamics on RECN Basic RECN tree detection: Unique tree detected as several independent trees R R R R R R R R Switch speedup: 1.5 All the sources start injection simultaneously : tree root

36 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 36 RECN improvements Modified (“Enhanced”) RECN: Congestion is detected at ingress or egress ports –Ingress cold queues are replaced by small “detection queues”, one per output port –If a detection queue fills over a threshold, congestion is detected for the corresponding output port It is allowed the allocation of more-specific SAQs –In order to keep in-order delivery of packets, a new allocated and more-specific SAQ is blocked until all the packets on the less-specific SAQ are forwarded –A pointer to the new SAQ is placed on the less- specific SAQ in order to control the blocking

37 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 37 RECN improvements Enhanced RECN tree detection: Unique tree detected as unique tree (a single root) R R Switch speedup: 1.5 All the sources start injection simultaneously : tree root

38 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 38 Performance Evaluation Objective: Evaluation of RECN improvements Comparative evaluation based on simulation results Evaluation metric: Network throughput when using: –Basic RECN –Enhanced RECN –VOQ at switch level (VOQsw)

39 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 39 Simulation Model Network configurations evaluated: 64 hosts connected by a 64x64 BMIN 512 hosts connected by a 512x512 BMIN 2048 hosts connected by a 2048x2048 BMIN Simulation assumptions: BMINs based on perfect shuffle scheme Deterministic routing 32 KB memories at ingress/egress ports Multiplexed crossbar (BW=8 or12 Gbps) Serial full-duplex pipelined links (BW=8 Gbps) 64-byte packets Credit-based and Xon-Xoff (for SAQs) flow control Maximum of 8 SAQs at ingress/egress ports (RECN)

40 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 40 Traffic Load Six different synthetic traffic patterns: Traces: From I/O activity at cello system disk interface A compression factor applied Normal trafficCongestion tree Traffic case Endnodes#SourcesDest. Injection rate #SourcesDest. Injection rate Congestion type #164x6475%Rand.50%25% Single hot-spot 100%Incremental #264x6475%Rand.100%25% Single hot-spot 100%Incremental #364x6475%Rand.50%25% Single hot-spot 100%sudden #464x6475%Rand.100%25% Single hot-spot 100%sudden #5512x51275%Rand.100%25% Four hot-spot 100%sudden #62048x204875%Rand.100%25% Four hot-spots 100%sudden

41 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 41 Simulation Results Network throughput: Traffic cases 1 and 2 (single hot-spot incremental traffic) 64-endnodes networks Speedup: 1.5 Traffic case 1 (Uniform traffic injection rate 50%) Traffic case 2 (Uniform traffic injection rate 100%)

42 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 42 Simulation Results Network throughput: SAN traffic (traces) 64-endnodes networks Traces compression factor: 40 Speedup 1.5 No Speedup

43 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 43 Simulation Results Network throughput: Traffic cases 3 and 4 (single hot-spot sudden traffic) 64-endnodes networks No Speedup Traffic case 3 (Uniform traffic injection rate 50%) Traffic case 4 (Uniform traffic injection rate 100%)

44 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 44 Simulation Results Network throughput: Traffic cases 3 and 4 (single hot-spot sudden traffic) 64-endnodes networks Speedup: 1.5 Traffic case 3 (Uniform traffic injection rate 50%) Traffic case 4 (Uniform traffic injection rate 100%)

45 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 45 Simulation Results Network throughput: Traffic cases 5 and 6 (four hot-spots sudden traffic) Uniform traffic injection rate 100% Speedup: 1.5 Traffic case 5 (512-endnodes network) Traffic case 6 (2048-endnodes network)

46 Title: Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture Conference: HiPEAC NovemberBarcelona, Spain 46 Conclusions Congestion trees producing HOL blocking may affect network performance We have shown that congestion trees may form and evolve in different ways We have analyzed the importance of considering congestion trees dynamics on the design of HOL blocking elimination techniques We have proposed some improvements for RECN, in order to manage HOL blocking independently of the way congestion trees form From the results of our experiments, these improvements were necessary

47 Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture P. J. García 1, J. Flich 2, J. Duato 2, I. Johnson 3, F. J. Quiles 1, F. Naven 3 2 Technical University of Valencia Valencia, Spain 3 Xyratex Havant, UK 1 University of Castilla-La Mancha Albacete, Spain HiPEAC November - 18 NovemberBarcelona, Spain


Download ppt "Dynamic Evolution of Congestion Trees: Analysis and Impact on Switch Architecture P. J. García 1, J. Flich 2, J. Duato 2, I. Johnson 3, F. J. Quiles 1,"

Similar presentations


Ads by Google