1. NETWORK ON CHIP ROUTER Students : Itzik Ben - shushan Jonathan Silber Instructor : Isaschar Walter Final presentation part B Spring 2006

2.  NoC concept  NoC’s Router design architecture & implementation (Short review of part-A)  NoC design  NoC demonstration and waveforms Agenda

4. Problem: Power, size and performance not practical for multi-processor chips using a single bus interconnection Solution: Network on Chip, based interconnection: fast, reliable data and low power consumption

5.  Implement a router for NoC in VHDL based upon research made by faculty members  Design and implement interface unit between NoC routers and process units  Design and implement a NoC based system connected to several processing units Project Goals

6. Network Scheme Processing Unit NoC ROUTER Interface

7. Network problems  Transmitting large data (buffer size, latency)  Loss of data due to full buffer  Data priority  Data from several inputs to one output  Blocked path prevents data transmission The NoC/Router should deal with these issues

8. Data in network  Packet based data flow  Single-flit or multi-flit packets (wormhole)  Service level support (packet priority)  Virtual channel support (router feature)

9. Packets in Wormhole architecture Packet bodyTarget Address Command A whole Packet  Each Packet is divided to Several smaller segments (flits)  Each flit is several bit width

10. Interface Process Unit Process Unit Process Unit Process Unit Process Unit Processing Unit Interface Between Bus & Router Packets in Wormhole architecture From Bus to Packets To Router Local Bus “network language” “local bus protocol”

12. Router 5x5 Processing Unit Data Control Data Control ROUTER Crossbar Input port West East South North Processing Unit Interface Module output port next router

13. Flit structure one flit Type Service Level Data out x 2bit x 12bit Or 8bit – first flit of packet CRT (first flit of packet) x 4bit (network size factor) Lsb bit Msb bit

14. 4 msb bits of every flit Service Level (SL) : Block Transfer = “00” (unimplemented yet) Real-Time = “01” (unimplemented yet) RD/WR = “10” (implemented as SL2) Signaling = “11” (implemented as SL1) Data Type : Idle = “00” – no signal Body = “01” – middle/header flit of a packet EP = “10” – end of packet FP = “11” – full packet (single flit packet)

15. Router design architecture guidelines  Input port serves as register-buffer  Crossbar designed in a full-connectivity (all inputs to all output)  Output port recognizes data waiting in input port buffer to be delivered, and extracts it (read command from output port goes to one or more inputs)

16. Routing Control & Terminology  Each Service Level (SL) has an independent path from input port until output port buffer  According to Current Routing Table (CRT) data is channeled to the appropriate output port, via the crossbar  In each output port the Current Serviced Input Port (CSIP) is determined by a Round-Robin logic which arbitrates between input ports waiting for transmission  Next Buffer’s State (NBS) determines if next router is ready to receive data

17. Router Input Port FIFO ROUTER InputPort CRT Switching SL Data In flits PREVIOUS ROUTER IN PATH / PROCESSING UNIT Buffer credits read/write Control Crossbar Data Per Service- Level Current Routing Table SL Read lines From output- ports

18. To Output Port From Input Port Routing DATA according to CRT From Input Port To Output port Crossbar Example for 2 Service Levels DATA + CRT DATA DATA + CRT CSIP SL1 SL2 SL1 SL2 CRT

19. Router Output Port ROUTER OutputPort CSIP Buffer credits NBS CSIP NBS CSIP Buffer credits Round-Robin & control NEXT ROUTER IN PATH / PROCESSING UNIT Data In flits Switching SL Crossbar Currently Serviced Input Port Next Buffer State Data Per Service- Level Read lines To input- ports

20. Part A – Final Router Design  Modular 5x5 Router, 16 bit data width  2 Service Levels (SL1, SL2)  Receiving/Transmitting Multi-flit packets (not only FP*) * FP = Full-Packets

21. NoC’s Router top view : Input port output port crossbar

22. Simulation conclusions : (Part A – Router only)  Latency = 3 clock cycles for a flit  Throughput = outputs 1 flit on each clock cycle (no wasted cycles)  No latency in output transition when between SL or in switching between inputs  Router can handle receiving & transmitting of multi-flit packets of different SL, from and to different inputs & outputs simultaneously

23. Part B Goals  Implement Router On Virtex II Pro  Validating & Extracting design parameters (area, timing) – Done  Ramp up understanding of PPC architecture or Micro-Blaze for connecting user-logic in order to build a QNoC based application on the Virtex II Pro – Done  Design, implement & simulate interface – Done  Implement a QNoC based application on a FPGA for validation of NoC – Done  Add virtual channel to existing router – Not implemented

25. Router – synthesis parameters

26. System Implementation System Components: 2x2 NoC, each router on the network is connected to a processing unit in order to validate NoCs activity PPC connected to NoC IP through OPB IPIF Module Chipscope ILA’s for signal check and analysis

27. NoC based Multi-core scheme Xilinx XUP Virtex™-II Pro Development System Router (part-A) PPC & peripherals (just as another module) Processing Module Processing Module Processing Module Router (part-A) Router (part-A) Router (part-A) PC – User Interface Through UART and Chipscope Analyzer NoC

28. Embedded system diagram (SOPC) : PLB_BRAM DPLB IPLB PPC PLB2 OPB Bridge OPB_ UART Custom Peripheral PLBOPB Custom Peripheral IPIF (IPIC)

29. Custom peripheral connection Write and read FIFO for data transaction between IP CORE and IPIC Reset IP CORE through IPIC – through code command

30. IP CORE - NoC Design 2x2 NoC Interface to PPC as another Module connected to the NoC Modules connected to the NoC

31. NoC – synthesis parameters Tight timing parameters and over-resources  Lets try to make the NoC a bit “slim” by reducing SL Tight timing Chip out of resources

32. NoC 1SL – synthesis parameters Good timing parameters, and have the resources for it Tight timing, but in PPC limits

33. IPIF2NoC Interface Considerations:  WR/RDFIFO communication protocol  Buffer credit calculation  Wormhole architecture

34. WRFIFO read protocol (Implemented in vhdl text) WRFIFO write is done by IPIC

35. RDFIFO write protocol (Implemented in vhdl text) RDFIFO read is done by IPIC We need to WrReq only for valid ≠0 data

36. IPIF RDFIFO protocol WrReq only for existing valid data ( ≠0 according NoC/Router protocol)

37. IPIF RDFIFO Buffer state (Initialize Router to n-1 places left on RDFIFO) Calculates if RDFIFO is not full so Output-port can transmit

38. Calculates Input-port NBS, if it’s not full so WRFIFO can transmit Router Input Buffer State (calculate Full state to WRFIFO)

39. CRT FIFO Adding Virtual Channel ROUTER InputPort Control routing by CRT Buffer credits Input Port Control Crossbar Data Per Service- Level CRT Switching SL Input Buffer PREVIOUS ROUTER IN PATH / PROCESSING UNIT Read command from output

40. Adding Virtual Channel – not as simple as it sounds PROBLEM: Multi flit packets might split to different virtual channels and may be confused as other packets

41. Development of interface Process Unit Process Unit Process Unit Process Unit Process Unit PPC Interface Packets in Wormhole architecture To Router PLBorFSL User-logic translating data from address to flits in SL with appropriate CRT IPIF

42. Future optional Enhancements  Debug router design  Adding Virtual-Channel to Router architecture  Adding wormhole architecture to the Interface architecture

43. NoC Implementation conclusions and summary NoC based upon developed router from part A simulated, synthesized and works well Latency and throughput as expected form router design multi-flit packets Unexpected result in switching between ports and different SL, as opposed to excellent behavior in simulations. The implementation of the part-A designed Router took more logic-area than expected.

44. Project conclusions  Learned a lot form the NoC project, on the NoC and Router concepts and architectures  Learned a lot on designing a system and not only looking at 1 object at a time  Improved our design and VHDL capabilities, and experienced in the “real world” hardware.  Debug tools aren’t so friendly for the first-time user, and design references should be more available.

46. QNoC based multi-core system Router 00 (address 0000) Interface to PPC through opb-ipif XOR With ‘1’ Calculator Inverter (Buffer) Calcul ating PC – User Interface Through UART and Chipscope Analyzer Xilinx XUP Virtex™-II Pro Development System PLB PPC IPIF IPIC PPC - Running pre-defined code Chipscope Control+ILA’s OPB On-Board memory UART Router 01 (address 0100) Router 10 (address 0001) Router 11 (address 1010

47. Calculator waveform - Full * Press on shadowed boxes for zoom

48. Calculator waveform part 1 Operator-1 enters through Interface to 00 router Operator-1 Exits through output East of router 00 and enters input West of router 10 Operator-1 Exits through output South of router 10 and enters input north of router 11 Operator-1 Exits through output Module of router 10 and enters IP calculator

49. Calculator waveform part 2 Operand enters through Interface to 00 router Operand Exits through output East of router 00 and enters input West of router 10 Operand Exits through output South of router 10 and enters input north of router 11 Operand Exits through output Module of router 10 and enters IP calculator

50. Calculator waveform part 3 Result Exits IP calculator and arrives back to input Module Operand-2 Exits output Module and through same path in the router enters IP calculator Operand-2 enters IP calculator and proper Result Exits IP calculator