1. Azeddien M. Sllame, Amani Hasan Abdelkader
A COMPARATIVE STUDY BETWEEN FAT TREE AND MESH NETWORK-ON-CHIP INTERCONNECTION ARCHITECTURES
Azeddien M. Sllame, Amani Hasan Abdelkader
Tripoli University
Tripoli, Libya

2. SOC building systems using billions of transistors on single chip
SOC contains many modules with different signals digital, analog, mixed-signal
Often includes
radio frequency (RF) functions
buses, interconnection networks
memory elements
image processing blocks (e.g. MPEG core)
digital signal processing (DSP) cores
CPUs, FPGA blocks

3. SOC examples of SOC systems: mobile phones, portable media devices
cable and satellite TV set-top-boxes

4. Definitions
Core (node): defined as any reusable design block, i.e. can be used as building block within chip designs in hardware or a sub-component in software programs.
IP (Intellectual Property) refer to copyrights.
Switch: is responsible for forwarding (switching and routing) packets from sender to the intended receiver using suitable techniques to guarantee this function with proper flow control and reasonable quality of services.

5. Definitions
Packet: is the smallest unit of communication containing routing information (e.g., destination address) and the sequencing information in its header. Its size is of order of hundreds or thousands of bytes or words. It consists of header flit and data flits

6. Definitions
Flit: it is the smallest unite of information at link layer and it is of size of one of several words. Flits can be several types and flit exchange protocol typically requires several cycles.
Phit: It is the smallest unite of information at the physical layer, which is transferred across one physical channel in one cycle.

7. SOC In the past: point-to-point communication links were used
Problems: power dissipation, cross talk delays due to routing wires inside the chip
Now :interconnection networks used to route packets between IP cores
Advantages: modular, well-structured, flexible, and has efficient performance
when one IP core is idle, other IP cores continue to make use of the network resources
interconnection networks are already used in many super-computers for many years

8. NOC The most distinguishing characteristic of SOC
Structure and connectivity complexity
In practice, most of SOCs are MPSOCs
“Network-On-Chip” (NOC) is a communication subsystem on CHIP, typically made between IP cores composing of SOC
Interconnection network is adopted from computer architecture best practice, to implement NOC for SOC cores
Hence:“route packets, not wires”

9. Interconnection Networks
mesh
hypercube
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
0000
Multistage interconnection networks: Butterfly Network
Fat tree

11. Interconnection Networks
Topology
Static (or direct)
point to point links interconnect the network nodes in some fixed topology (regular)
mesh and hypercube
Dynamic (or indirect)
allows the interconnection pattern between the network nodes to be varied dynamically (using SWITCHING)
Fat trees and multistage networks

12. Switching, Routing
Routing and switching are the two main factors that control network latency and throughput, and realize the overall network performance
The most commonly used switching techniques are including: circuit switching, packet switching, virtual cut-through, and wormhole routing
Wormhole routing is the most common switching technique used in commercial machines because it allows simple, small, cheap, and fast routers
Wormhole supported by virtual channels

13. Wormhole Switching
Physical channel may support several virtual channels multiplexed (time-multiplex) across physical channel
They will all have their own buffers, but they will share one single physical channel medium.
Each unidirectional virtual channel can hold, for example, four flits of the same packet, mixing flits from different packets is not allowed (see figure)

14. Virtual Channels
Packets can share the physical channel on a flit-by-flit basis
The physical channel protocol must be able to distinguish between the virtual channels
Keeping adding virtual channels to further reduce the blocking; will result in increased network throughput in flits/second, due to increased physical channel utilization
Increasing channel multiplexing reduces the data rate of individual message and increasing message latency. Nevertheless, general network throughput will be increased,

15. Typical switch It consists of: -Switching -Routing -Arbitration
-Input link controller
-Output link controller

16. IP Switch Performs the functions of routing and switching
Ensures the storing of packets (flits) to be transferred to other intermediate switches of the fat network
Each switch is bidirectional; every port is associated with a pair of opposite unidirectional channels, one for inputs and one for outputs
It consists of:
Routing unit
Arbitration unit
Input link controller unit
Output link controller unit

17. IP Switch: Input Link Controller Unit
Responsible for receiving incoming flits from different IP’s (and switches) and forwarding them to the associated units, with the help of using virtual channel technique.
It checks the availability of free input virtual channel, if exists then it returns free virtual channel number;
It manages sending out the flit that is available in input virtual channel buffer;
It helps do routing function by setting the outgoing physical link number that the flit occupying in the virtual channel must follow to reach the destination;
It keeps track of the outgoing physical link number that is used by flit occupying the virtual channel now;
It does path setting up using the outgoing virtual channel number for the flit occupying the virtual channel ;
It sends flit by passing the front flit in the specified input virtual channel on the corresponding input physical link, and.
It makes buffer management.

18. IP Switch: Output Link Controller Unit
Responsible for receiving the incoming flits from the input controlling unit (after determining the appropriate output link controller number)
Then, it forwards them to destination or to other intermediate switches, with the help of using virtual channel technique
Buffers at a specified virtual channel are used to help output link in performing its functions

19. Procedure of moving a flit from switch output buffer to NEXT switch input buffer

20. Fat Tree Simulator Structure
Packet communication flow for the proposed fat tree based

21. IP Node: Traffic Generator
Traffic generator unit is in charge of creating messages in random lengths and it works in IP node level to generate the random data to pass through the fat tree NOC model
Each message has random data and generated at different random time stamps and has random message lengths (different packet sizes)

22. IP Node
IP nodes are placed at the leaves in the level zero and connected with parent switches with two unidirectional physical links
Each IP node generates its own messages that are required to be sent to certain destinations

23. IP Node Each node has: its address,
its generated message list to hold the generated messages,
received message list to hold the received messages from different nodes

25. :::::::Results::::::
Example: IP node#3 sends Msg to IP node#6

27. Fat Tree Simulator Structure
Packet communication flow for the proposed fat tree based

28. Criteria of comparison
Topology
Out of order reception of packets
Network traffic balance
Deadlock, livelock, starvation
Routing
Fault tolerance
Congestion control
Latency and throughput
Network utilization
Scalability
Energy dissipation
Physical realization

29. MESH vs. FAT tree
Mesh networks belongs to direct interconnection networks;
point-to-point interconnects the network nodes in fixed regular topology
Fat tree is the typical example of the indirect interconnection network;
allows changing of the interconnection arrangement among the network nodes dynamically through the use of network’s switches

30. MESH vs. FAT tree
Fat tree designs employ adaptive routing in which there is a possibility of livelock and starvation. Hence, a special care should be taken during switch design process in order to avoid deadlock, livelock and starvation
it can adapt to network congestion conditions (can do re-routing, out-of-order transmission)
Mesh uses XY deterministic routing which is considered as deadlock and livelock free
Deterministic routing in 2D mesh has in-order packet delivery which makes it simple to implement

31. The simulator: (gpNoCsim)
General Purpose Simulator for Network-on-Chip Architectures simulator
gpNoCsim is an open-source tool developed in Java, component based simulation framework for NOC architectures. Version 1.0 of gpNoCsim contains the implementation of mesh, torus, butterfly fat tree, extended butterfly fat tree networks
Described in :(Hemayet et al. 2007)
Fat tree simulator: Described in (Sllame et al 2012)

32. [Flits leaving Switch]
Mesh
Number
of flits / Buffer
Throughput
[Flits leaving Switch]
Avg Packet Delay 2
0.1605 4
0.2135 8
fat tree
1.1795

33. Throughput and Number of Virtual Channels for 16 IP Cores

34. Average Throughput per Switch (Flits Leaving Switch) and Number of Virtual Channel for 16 IP Cores

35. Average Packet Delays (ns) with Average Message Length (bytes) for 64 IP Cores

36. The Relation Between Average Packet Delay and Number of Virtual Channels for 64 IP Cores

37. Different Values of Buffer Size with Different Virtual Channels for Fat Tree with 64 IP Cores

38. Different Values of Buffer Size with Different Virtual Channels for Mesh with 64 IP Cores

39. Conclusions
The main goal of this paper was to analyze the 2D-mesh and fat-tree architectures as a NOC interconnection networks candidates.
The evaluation process has been done using available open-source simulators. The comparison includes: routing, switching methods used in the switches, effect of buffering, effect of virtual channel technique, effect of packet length.
We believe that the scalability and higher bandwidth of the fat tree network makes it the preferred NOC for future massively parallel NOC systems.

40. Thank you