Introduction to Interconnection Networks

Introduction to Interconnection network Digital systems(DS) are pervasive in modern society. Digital computers - simulating physical systems, managing large databases, preparing documents and etc Audio and video entertainment is increasingly being delivered and processed in digital form. Finally, almost all products from automobiles to home appliances are digitally controlled.

Digital Systems Communication (Moves Data) Communication (Moves Data) Memory (Stores Data) Memory (Stores Data) Logic (Transfers and combines Data) Logic (Transfers and combines Data)

Digital Systems Logic (Transfers and combines Data) Logic (Transfers and combines Data) Memory (Stores Data) Memory (Stores Data) Communication (Moves Data) Communication (Moves Data) Interconnection Network

Performance – limited by communication/interconnection not logic/memory Power - most of the clock cycle spent on wire delays not gate delay. Technology improves, memories and processors become small, fast, and inexpensive. Also, the frequency of communication between components is lagging far beyond the clock rates of modern processors. These factors combine to make interconnection the key factor in the success of future digital systems.

Interconnection Network ApplicationAlgorithmProgramming LanguageOperating System Instruction Set Architecture Micro-architecture Register Transfer LevelCircuitsDevicesTechnology Computer Architecture LogicMem Logic Mem LogicMem Interconnection Network

Network Scenario “On-chip networks may deliver data between memory arrays, registers, and arithmetic units within a single processor “

Why Networks ? Computing demands Processors requirements What is the Solution ?

Three question about Interconnection Networks What is an Interconnection Network ? An Interconnection Network is a programmable system that transports data between terminal.

Where do you find Interconnection Network ? They are used in almost all digital systems that are large enough to have two components to connect. The most common applications computer systems and communication switches Computer System - they connect processors to memories and input/output (I/O) devices to I/O controllers Communication Switches - They connect input ports to output ports

Why are interconnection networks important ? Because they are a limiting factor in the performance of many systems The interconnection network between processor and memory largely determines the memory latency and memory bandwidth, two key performance factors, in a computer system of many systems The performance of the interconnection network in a communication switch largely determines the capacity (data rate and number of ports) of the switch

Why on-chip networks ? They provide external connectivity from system to outside world Also, connectivity within a single computer system at many levels I/O units, chips, modules and blocks inside chips Trends: high demand on communication bandwidth Increased computing power and storage capacity Switched networks are replacing buses Integral part of many-core architectures Energy consumed by communication will exceed that of computation in future systems

NoC: A paradigm Shift in VLSI s s s s ss s s Module s From: Dedicated signal wiresTo: Shared network Network switch Computing Module Point to Point Link

Processing element Network Interface Router Input buffers Unidirectional links NoC Architecture

Perspective 1: NoC vs. Bus Aggregate bandwidth grows Link speed unaffected by N Concurrent spatial reuse Pipelining is built-in Distributed arbitration However: No performance guarantee Extra delay in routers Area and power overhead? Modules need NI Unfamiliar methodology  Bandwidth is shared  Speed goes down as N grows  No concurrency  Pipelining is tough  Central arbitration However :  Fairly simple and familiar NoC Bus AB E C D Shared Bus

Perspective 2: NoC vs. Off-chip Networks Cost is in the links Latency is tolerable Traffic/applications unknown Changes at runtime Adherence to networking standards  Sensitive to cost: area power  Wires are relatively cheap  Latency is critical  Traffic may be known a-priori  Design time specialization  Custom NoCs are possible  Example: Off-Chip NetworksNoC

Performance Metrics: Packet Latency Effective Bandwidth

Terms and Definitions: Bandwidth: Maximum rate at which information can be transferred (including packet header, payload and trailer) Unit: bits per second (bps) or bytes per second (Bps) Time of flight: Time for first bit of a packet to arrive at the receiver Includes the time for a packet to pass through the network, not including the transmission time Unit: Picoseconds (OCNs), nanoseconds (SANs), microseconds (LANs), milliseconds (WANs)

Transmission time: The time for a packet to pass through the network, not including the time of flight Equal to the packet size divided by the data bandwidth of the link Transport latency: Sum of the time of flight and the transmission time Measures the time that a packet spends in the network Sending overhead (latency): Time to prepare a packet for injection, including hardware/software A constant term (packet size) plus a variable term (buffer copies) Receiving overhead (latency): Time to process an incoming packet at the end node A constant term plus a variable term Includes cost of interrupt, packet reorder and message reassembly

Receiver Sender Sending overhead Transmission time (bytes/bandwidth) Time of flight Transmission time (bytes/bandwidth) Receiving overhead Transport latency Total latency Time Latency = Sending Overhead + Time of flight + + Receiving Overhead packet size Bandwidth

Example (latency): calculate the total packet latency for interconnect distances of 0.5 cm, 5 m, 5,000 m, and 5,000 km Assume a dedicated-link network with 8 Gbps (raw) data bandwidth per link Device A sends 100-byte packets (header included) Overheads Sending overhead: x ns/byte Receiving overhead: 4/3(x) ns/byte x is 0 μs for OCN, 0.3 μs for SAN, 3 μs for LAN, 30 μs for WAN Assume time of flight consists only of link propagation delay (no other sources of delay) int. network Device A Device B 8 Gbps raw data bandwidth per link Cr

int. network Device A Device B 8 Gbps raw data bandwidth per link Cr Latency OCN = 5 ns ns ns + 5 ns = ns Latency SAN = μs ns μs μs = μs Latency LAN = μs + 25 μs μs μs = μs Latency WAN = μs + 25 μs μs μs = ms Latency = Sending overhead + Time of flight + Packet size Bandwidth + Receiving overhead