1. INTERCONNECTION NETWORKS Work done as part of Parallel Architecture Under the guidance of Dr. Edwin Sha By Gomathy Gowri Narayanan Karthik Alagu Dynamic Interconnection Networks - Gomathy Interconnection Networks -Karthik

2. ORGANISATION OF THE PRESENTATION Study Interconnection Networks Requirements Network Design Considerations Static Interconnection Networks Dynamic Interconnection Networks Concentrate more on the DIN and on MIN Comparisons and Tradeoffs

3. What are Interconnection Networks? Connect processors and memory banks Connect processors and memory banks Determines the overall performance Determines the overall performance Consists of Processing Elements Consists of Processing Elements Route packets based on application Route packets based on application

4. Interconnection Networks Processing element to processing element Processing element to processing element N processing elements connected by network. N processing elements connected by network. Processing element(PE)  own processor and elements. Processing element(PE)  own processor and elements. Unidirectional network. Unidirectional network. Processor to memory Processor to memory Position network between processors and memories Position network between processors and memories Bidirectional Bidirectional Connects each processor to all or some subset of memories Connects each processor to all or some subset of memories Move data from processor to processor  thro memories Move data from processor to processor  thro memories

5. Requirements Transfer a maximum number of messages in minimum time with minimum cost and maximal reliability. Transfer a maximum number of messages in minimum time with minimum cost and maximal reliability. Small diameter and small average distance Small diameter and small average distance Small and fixed vertex degree Small and fixed vertex degree Large bisection width Large bisection width High connectivity and fault tolerance High connectivity and fault tolerance Small fault average distance and diameter Small fault average distance and diameter Hamiltonianity Hamiltonianity Hierarchical recursivity Hierarchical recursivity Incremental extendibility and incremental scalability Incremental extendibility and incremental scalability Symmetry Symmetry Support for routing and collective communication Support for routing and collective communication

6. Design Considerations Performance requirements Performance requirements Message latency Message latency network may saturate network may saturate throughput throughput Scalability Scalability Incremental expandability Incremental expandability Distance span Distance span Physical constraints Physical constraints Reliability Reliability Expected workloads Expected workloads Control strategy Control strategy Switching methodology Switching methodology Cost constraints Cost constraints

7. Metrics Framework to compare and evaluate interconnection networks. Framework to compare and evaluate interconnection networks. Network connectivity  measures resilience and fault tolerance Network connectivity  measures resilience and fault tolerance Network diameter  maximum inter-node distance Network diameter  maximum inter-node distance Narrowness  congestion in a network Narrowness  congestion in a network Network expansion increments  expandability Network expansion increments  expandability

8. Classification Topology  what type of network Topology  what type of network Direct/indirect Direct/indirect Switching strategy  how data in a message traverses a route. Switching strategy  how data in a message traverses a route. Circuit switching/packet switching Circuit switching/packet switching Routing algorithm  which path the data follows Routing algorithm  which path the data follows Deterministic Deterministic Adaptive Adaptive Store and forward Store and forward Cut-through switching – Worm-hole routing Cut-through switching – Worm-hole routing Flow control mechanism  when a message traverses a route Flow control mechanism  when a message traverses a route Ethernet, FDDI/token ring, TCP/WAN Ethernet, FDDI/token ring, TCP/WAN

9. Static interconnection networks Topology remains same Topology remains same Connected statically via Point-Point communication links Connected statically via Point-Point communication links Used in message-passing architectures Used in message-passing architectures

10. Static interconnection networks Completely-connected network. Completely-connected network. Star-connected network. Star-connected network. Linear array or ring of processors. Linear array or ring of processors. Mesh network (in 2- or 3D). Mesh network (in 2- or 3D). Intel Paragon XP/S, Cray T3D/E Intel Paragon XP/S, Cray T3D/E Tree network of processors. Tree network of processors. TMC CM5 TMC CM5 Hypercube network. Hypercube network. SGI/Cray Origin 2000 SGI/Cray Origin 2000

11. Dynamic interconnection networks Connected dynamically via switches or buses Connected dynamically via switches or buses Allow reconfiguration during operation Allow reconfiguration during operation Used in shared address space architectures Used in shared address space architectures Cost: number of switch boxes required Cost: number of switch boxes required

12. Types Bus-based networks Bus-based networks PE-M  Common data path PE-M  Common data path Crossbar switching networks Crossbar switching networks grid of switching elements grid of switching elements Multistage interconnection networks Multistage interconnection networks In multiprocessor systems In multiprocessor systems Compromise between cost and performance Compromise between cost and performance Non-blocking Non-blocking Rearrangeable Rearrangeable Blocking: Blocking: Multilevel interconnection network Multilevel interconnection network two or more levels of connections two or more levels of connections SGI/Cray Origin 2000 SGI/Cray Origin 2000

13. Detailed overview Bus Architecture Bus Architecture Used over a wide range of cost and performance Used over a wide range of cost and performance Limitations Limitations capacity bottleneck capacity bottleneck single point of failure single point of failure Limited Fan out capability Limited Fan out capability Solution: use multiple shared buses. Solution: use multiple shared buses. Sequent symmetry and Alliant(double) Sequent symmetry and Alliant(double)

14. Cross Bar Networks Cross Bar Networks non-blocking access from any input port to any output port non-blocking access from any input port to any output port Low latency Low latency Problems Problems requires O (N 2 ) requires O (N 2 ) number of pins number of pins Ex C - YMP and Fujitsu VPP 500 Ex C - YMP and Fujitsu VPP 500

15. Multistage Interconnection Networks Sets of switches in parallel Sets of switches in parallel Nodes are switches Nodes are switches Types of MIN Types of MIN Omega Omega Generalized cube Generalized cube Clos Clos Benes Benes Banyan Banyan Butterfly Butterfly Gemini Gemini Honeycomb Honeycomb Delta Delta Bi-delta Bi-delta

16. Omega network Multistage implementation of single-stage shuffle-exchange network. Multistage implementation of single-stage shuffle-exchange network. 4 operations  4 operations  Blocking Blocking routes to different memory banks share a link - message blocked routes to different memory banks share a link - message blocked log p stages log p stages Broadcast data to multiple destinations Broadcast data to multiple destinations

17. Generalized-cube networks Multistage cube-type network topology Multistage cube-type network topology log 2 N stages. log 2 N stages.

18. Clos network Rearrangable non-blocking Rearrangable non-blocking Implements low latency, high- bandwidth, connection-oriented ATM switching. Implements low latency, high- bandwidth, connection-oriented ATM switching. Multiple routes between hosts Multiple routes between hosts  deadlock-free – hot-spots No. of middle switches = (2n-1) i No. of middle switches = (2n-1) i

19. Benes Networks Special case of Clos networks made of 2 x 2 switches Special case of Clos networks made of 2 x 2 switches Recursively constructed from Clos network Recursively constructed from Clos network Exhibits a symmetric topological structure. Exhibits a symmetric topological structure. Minimal number of cross points Minimal number of cross points For N x N switches For N x N switches N/2 alternative paths N/2 alternative paths 2 log N-1 stages 2 log N-1 stages cost = N 2 cost = N 2

20. Banyan network Unique path from each input to each output Unique path from each input to each output For k× k switches For k× k switches logk N stages logk N stages N/k log k N switches N/k log k N switches Advantages Advantages self-routing self-routing regularity and interconnection patterns regularity and interconnection patterns makes very attractive for VLSI implementation makes very attractive for VLSI implementation Blocking - buffers Blocking - buffers

21. Butterfly networks Built using crossbar switches Built using crossbar switches closely related to shuffle-exchange networks closely related to shuffle-exchange networks no broadcast connections no broadcast connections Unique path of length log N Unique path of length log N Source Oblivious path selection Source Oblivious path selection NYU Ultra computer, IBM RP3, BBN Butterfly, NEC's Cenju NYU Ultra computer, IBM RP3, BBN Butterfly, NEC's Cenju

23. Gemini Networks Dual technology interconnection network Dual technology interconnection network Used in tightly coupled multicomputer systems. Used in tightly coupled multicomputer systems. circuit-switched optical data path in parallel with a packet- switched electrical control/data path. circuit-switched optical data path in parallel with a packet- switched electrical control/data path. optical path  transmission of long data messages optical path  transmission of long data messages electrical path  switch control and transmission of short messages. electrical path  switch control and transmission of short messages. O (N log N) switching elements O (N log N) switching elements

25. Honey Comb network Based on hexagonal plane tessellation Based on hexagonal plane tessellation n nodes n nodes degree 3 degree 3 Diameter O ( sqrt (n)) Diameter O ( sqrt (n)) Network cost = degree * diameter Network cost = degree * diameter Easy physical layout Easy physical layout

26. Delta Networks Digit controlled or baseline routing Digit controlled or baseline routing recursive structure recursive structure Path descriptor Path descriptor Exactly one path from each input node to each output node. Exactly one path from each input node to each output node.

27. Comparison of DIN Bus based Bus based Increasing processor, performance deteriorate Increasing processor, performance deteriorate Crossbar Crossbar Good data performance - expensive Good data performance - expensive Multistage Multistage Compensate between cost and performance Compensate between cost and performance

28. Tradeoffs

29. Comparison of MIN

30. Real machines

31. Summary Interconnection Overview Interconnection Overview Static Interconnection Networks Static Interconnection Networks Dynamic Interconnection Networks Dynamic Interconnection Networks Multistage Interconnection Networks Multistage Interconnection Networks Tradeoffs Tradeoffs Comparisons Comparisons Comment: choice of IN depends on the application Comment: choice of IN depends on the application

32. Questions ??

33. Our thanks to Dr. Edwin Sha for his support and guidance.