1. NETWORK-ON-CHIP (NOC): A New SoC Paradigm
Dr. Konstantinos Tatas

2. PRESENTATION OUTLINE Introduction Part A Part B Summary
Motivation – SoC Communication
Current Solutions
NoC Concept
Part B
Summary

3. THE MANY CORES ERA Source:
International Roadmap for Semiconductors 2007 edition (

4. THE GROWING GAP: COMPUTATION VS. COMMUNICATION
2:1
9:1
Taken From ITRS, 2001

5. GROWING CHIP DENSITY Future? Design complexity - high IP reuse
1998
ASIC mm
2012
SoC - 22nm
Memory, I/O P
Future?
Design complexity - high IP reuse
Efficient high performance interconnect
Scalability of communication architecture

6. Traditional SoC Nightmare
The architecture is tightly coupled
DMA
CPU
DSP
Mem
Ctrl.
Bridge
MPEG I o
The “Board-on-a-Chip” Approach C
System Bus
Control Wires
Peripheral Bus
Variety of dedicated interfaces
Poor separation between computation and communication.
Design Complexity
Unpredictable performance

7. Computational demands of future multimedia applications - Memory bandwidth scales proportional
K. Uchiyama., “Power-Efficient Heterogeneous Parallelism for Digital Convergence”, VLSI Circuit Digest of Technical Papers, IEEE p 6-9, June 2008
Jian Li, “3D Integration opportunities and challenges”, ISCAS 2008 tutorial on 3D

8. Shared address space communications

9. System bus

10. Cross-bar

11. Multi-stages network on chip

12. An NoC example
Source: ossum, MPSoC’07

13. NOC Topologies
Regular topologies: general-purposed on-chip multiprocessors
Custom topologies:

14. NoC vs. “Off-Chip” Networks
What is Different?
Routers on Planar Grid Topology
Short Point-To-Point Links between routers
Unique VLSI Cost Sensitivity:
Area-Routers and Links
Power

15. NoC vs. “Off-Chip” Networks
No legacy protocols to be compliant with …
No software  simple and hardware efficient protocols
Different operating env. (no dynamic changes and failures)

16. Custom Network Design – You design what you need!
NoC vs. “Off-Chip” Networks
No legacy protocols to be compliant with …
No software  simple and hardware efficient protocols
Different operating env. (no dynamic changes and failures)
Custom Network Design – You design what you need!

17. Custom Network Design – You design what you need!
NoC vs. “Off-Chip” Networks
No legacy protocols to be compliant with …
No software  simple and hardware efficient protocols
Different operating env. (no dynamic changes and failures)
Custom Network Design – You design what you need!
Example1: Replace modules
Replace

18. Custom Network Design – You design what you need!
NoC vs. “Off-Chip” Networks
No legacy protocols to be compliant with …
No software  simple and hardware efficient protocols
Different operating env. (no dynamic changes and failures)
Custom Network Design – You design what you need!
Example2: Adapt Links
Adapt Links

19. Non-Segmented Bus (NS-Bus) Segmented Bus (S-Bus) Point-To-Point (PTP)
NoC Cost Scalability vs. Alternatives
Compare the cost of:
NoC
Non-Segmented Bus (NS-Bus)
Segmented Bus (S-Bus)
Point-To-Point (PTP)

20. Why noc? Bus NoC Longer connections  higher parasitic capacitance
Performance does not downgrade with network scaling
Arbitration grows and becomes a bottleneck
Arbitration and routing are distributed
Bandwidth is limited and shared by all cores
Aggregated bandwidth scales with network size
Latency is wire-speed once arbitration granted control
Multiple hops increase latency
Well-known and simple concepts
Further study needed

21. Which are the main challenges?
Communication infrastructure
Communication paradigm selection
Application mapping optimization
Programming model
Physical design
Design automation/tool-flow integration

22. Basic Switching Techniques
Circuit Switching A real or virtual circuit establishes a direct connection between source and destination.
Packet Switching Each packet of a message is routed independently. The destination address has to be provided with each packet.
Store and Forward Packet Switching The entire packet is stored and then forwarded at each switch.
Cut Through Packet Switching The flits of a packet are pipelined through the network. The packet is not completely buffered in each switch.
Virtual Cut Through Packet Switching The entire packet is stored in a switch only when the header flit is blocked due to congestion.
Wormhole Switching is cut through switching and all flits are blocked on the spot when the header flit is blocked.

23. Circuit Switching (are they noc?)
Phases:
Circuit Setup
Transmission
Tear Down
Disadvantages:
Exclusive allocation of resources
Long setup phase
Advantages:
High performance - throughput and latency
Low power consumption
Low overhead during transmission phase
Predictable transmission

24. Packet Switching vs Circuit Switching

25. NoC Router

26. NoC-based MPSoC nodes Routers Links Network Interfaces (NIs)
Processing Elements (PEs), such as CPUs, custom IPs, DSPs, etc.
storage elements (embedded memory blocks),
Routers
Links
Network Interfaces (NIs)
Often a switch together with its host node memory is referred to as a tile.

27. NoC Topologies Regular/irregular Direct/indirect
each node has a direct point-to-point link to a subset of other nodes in the system, called neighboring nodes

28. 2D Mesh simplest and most popular topology for NoCs.
Every switch, except those at the edges, is connected to four neighboring switches and one node.

29. 2D Torus
layout of a regular mesh except that nodes at the edges are connected to switches at the opposite edge via wrap-around routing channels.
Every switch has five ports
The limitation of this topology affects the long end-around connections

30. Octagon well-established direct topology found in NoCs.
ring of 8 nodes connected by 12 bi-directional links.
links provide two-hop communication between any pair of nodes in the ring
simple algorithms for fast yet efficient shortest-path routing.
In case a platform consists of more than eight nodes, the octagon is extended to multidimensional space

31. Fat-tree and butterfly fat-tree
nodes are connected to an architecture's external switch
switches have point-to-point links to other switches.
processing units and memory modules are assigned to the leafs of the trees,
switches are placed at the vertices,
communication involves climbing up and down some part of the tree.
A pair of coordinates is used to label each node, ($l$, $p$), where $l$ denotes a node's level and $p$ gives its position within this level.

32. Polygon widely accepted topology
packets travel in a loop from one router to the next.
We can add chords to the circle
if chords are inserted only between opposite routers, the topology is called a spidergon.

33. Star central router in the middle of the star,
computational resources, or subnetworks, in the spikes of the star.
The capacity requirements of the central router are quite large,
significant possibility of congestion in the middle of the star

34. Flow Control intra-switch switch-to-switch end-to-end Buffered
Bufferless
end-to-end

35. ACK/NACK handshaking protocol
When a sender puts data on the link, it activates a VALID signal.
When the receiver is ready to consume the valid data, it activates the corresponding ACK signal.
If the data is corrupt or there is no buffer space to store them, a NACK signal is activated instead.
Upon receipt of a NACK, the sender starts resending flits starting from the not acknowledged one
inherently supports fault tolerance,
additional buffer space required to keep sent flits in case retransmission is required.

36. Stall/go requires just two control wires
one going forward, signifying data availability,
one going backward and signaling either a condition of buffers filled ("STALL") or of buffers free ("GO")

37. Credit-based transmitter has a "credit" counter
initialized to the value of empty buffer slots of the receiver
decrements it every time a flit is sent.
The credit counter must be updated in case the receiver consumes or forwards a flit and therefore increases its buffer space.
a credit value that is sent back to the transmitter to be added to the current value of the credit counter.
transmitter stalls when the credit value is zero and
resumes when its value increases again.

38. NI Design logic required to connect the nodes to the NoC.
NIs can differ significantly depending on the nature of the node
Using a NI allows IPs and communication infrastructure to be designed independently
One end of a NI is connected to a router using the selected flow control protocol
the other to the node IP
Since most IPs are designed to communicate through a bus, the NI uses a bus interface
NI is not simply a protocol adapter from a processor bus to a router port.
Ideally, the NI must offer the processing cores the view of a shared memory system, and the network itself should be transparent.

39. NI services adaptation services transaction reordering services,
packetization/depacketization
protocol conversion and clock domain crossing.
absolute minimum services required of the NI so that data can be sent and received on the NoC
transaction reordering services,
error and flow control services
error detection and/or correction
request retransmission when required
route computation services
Source routing
upper layer services
Cache coherence

40. Typical NoC Packet Format
Header
routing and network control information.
In the case of distributed routing the information required is the destination and source addresses
in the case of source routing the complete routing information is written
In the case of variable packet size a length field is required
Payload
Tail
sequence number
error control fields such as hamming code or CRC fields

41. Source vs Distributed Routing
In source routing the entire routing path is computed at the source and appended to the packet.
The routers do not make any routing decisions,
in distributed routing, the routing path is decided in a hop-by-hop basis at each router even for deterministic routing algorithms.
The only information required to be found in the packet is the destination address.
The advantage of source routing is that it requires simple routers and can easily support irregular architectures. Its disadvantage is that it does not provide adaptiveness and requires more complex NIs and packets.

42. Source vs Distributed Routing