Module R R RRR R RRRRR RR R R R R Access Regulation to Hot-Modules in Wormhole NoCs Isask’har (Zigi) Walter Supervised by: Israel Cidon, Ran Ginosar and Avinoam Kolodny Or: Hot-Modules, Cool NoCs Technion – Israel Institute of Technology

May 2007Hot-Modules in Wormhole NoCs2 Hot-Modules NoC is designed and dimensioned to meet QoS requirements -Buffer sizing, routing, router arbitration, link capacities, … NoC designers cannot tune everything -Modules typically have limited capacity High-demanded, bandwidth limited modules create edge bottlenecks -In SoC, often known in advance Off-chip DRAM, on-chip special purpose processor System performance is strongly affected -Even if the NoC has infinite bandwidth

May 2007Hot-Modules in Wormhole NoCs3 Hot Module (HM) in NoC Wormhole, BE NoC At high Hot Module utilization, multiple worms “ get stuck ” in the network Two problems arise: -System Performance -Source Fairness IP (HM) Interface

May 2007Hot-Modules in Wormhole NoCs4 IP3 Interface IP2 Interface IP1 (HM) Interface HM is not a local problem. Traffic not destined at the HM suffers too! Hot Module Affects the System Problem#1

May 2007Hot-Modules in Wormhole NoCs5 Multiple locally fair decisions Global fairness HM Interface The limited, expensive HM resource isn ’ t fairly shared Source Fairness Problem Problem#2

May 2007Hot-Modules in Wormhole NoCs6 Our Approach Problem is not caused by the NoC -But rather by a congested end-point Solution should address the root cause -Not the symptoms Utilize existing NoC infrastructure Solve both problems -Simple and efficient

May 2007Hot-Modules in Wormhole NoCs7 Hot Module Congestion During congested periods, sources should not inject packets towards the HM -Will experience increased delay anyway -Better wait at the source, not in the network Keep routers unmodified!

May 2007Hot-Modules in Wormhole NoCs8 IP1 Control IP4 NoC Interface IP3 IP2 (HM) HM Allocation Control Basics Interface Allocation Controller Interface

May 2007Hot-Modules in Wormhole NoCs9 IP1 IP4 NoC Interface IP3 IP2 (HM) Interface Control HM Allocation Control Basics Allocation Controller Interface

May 2007Hot-Modules in Wormhole NoCs10 IP1 Control NoC IP2 (HM) Allocation Controller Interface IP3 IP4 Interface HM Allocation Control Basics Interface

May 2007Hot-Modules in Wormhole NoCs11 HM Control Packets The HM Controller receives all requests and can employ any scheduling policy Control packets are sent using a high service level -Bypassing (blocked) data packets! Dest. Req. Credit Source Dest. Credit Source Credit request packetCredit reply packet

May 2007Hot-Modules in Wormhole NoCs12 Multiple Priority Router Control packets

May 2007Hot-Modules in Wormhole NoCs13 Enhanced Request packet The request may include additional data as needed -payload ’ s priority, deadline, expiration time, etc. Dest. Deadline Expiration Priority Req. Credit Source …… Optional fields Credit request packet

May 2007Hot-Modules in Wormhole NoCs14 …… ExpirationdeadlinePrioritySizeSRC The HM Allocation Controller is customized according to system ’ s requirements HM Allocation Controller Pending Requests Table Local Arbiter Credit Requests Credit Replies Requests Decoder Reply Encoder Optional HM Access Controller

May 2007Hot-Modules in Wormhole NoCs15 Short packets are not negotiated Source ’ s quota is slowly self-refreshing The mechanism is turned-off when the network is not congested Crediting modules ahead of time hides request-grant latency -For light-load periods Further Enhancements

May 2007Hot-Modules in Wormhole NoCs16 Not Classic Flow-Control Flow-control protects destination ’ s buffer -A pair-wise protocol HM access regulation protects the system -Many-to-one protocol

May 2007Hot-Modules in Wormhole NoCs17 Results – Synthetic scenario Hotspot traffic -All-to-one traffic with all-to-all background traffic High network capacity Limited hot module bandwidth HM controller arbitration: Round-robin Module HM Module R R R R RR R RRRR RRRR R

May 2007Hot-Modules in Wormhole NoCs18 System Performance Without regulation With Regulation X30 X10 Average Packet Latency

May 2007Hot-Modules in Wormhole NoCs19 Hot vs. non-Hot Module Traffic HM Traffic without regulation Background Traffic Without regulation HM Traffic with regulation Background Traffic With regulation Using regulation, non-HM traffic latency is drastically reduced X40 Average Packet Latency

May 2007Hot-Modules in Wormhole NoCs20 Source Fairness Source#16 no regulation Source#5 no regulation Source#5 with regulation Source#16 with regulation R R R R RR R RRRR RRRR R

May 2007Hot-Modules in Wormhole NoCs21 Fairness in Saturated Network Hot-Module Utilization: 99.99% Regulated Hot-Module Utilization: 98.32% Simulation results for a 4-by-4 system, Data packet length: 200 flits Control packet length: 2 flits No allocation control With allocation control

May 2007Hot-Modules in Wormhole NoCs22 MPEG-4 Decoder Real SoC Over provisioned NoC Two hot-modules VU AU MED CPU RAST SDRAM SRAM1 SRAM2 IDCT ADSP UP SAMP BAB RISC 25% of all traffic 22% of all traffic SDRAMSRAM2

May 2007Hot-Modules in Wormhole NoCs23 Results – MPEG-4 load: X2 load: X8 reduction All traffic HM/non-HM traffic breakdown X2 X8

May 2007Hot-Modules in Wormhole NoCs24 The HMs are better utilized Without regulation, the hot-modules are only 60% utilized -Traffic to one HM blocks the traffic to the other! No allocation control With allocation control 1  HM1 2  HM1 3  HM1 4  HM1 9  HM1 10  HM1 11  HM1 8  HM2 10  HM2 11  HM2 12  HM2 Total Flows destined at HM1 Significant differences in BW! Flows destined at HM2

May 2007Hot-Modules in Wormhole NoCs25 Hot-Module Placement

May 2007Hot-Modules in Wormhole NoCs26 Summary Hot-modules are common in real SoCs Hot-modules ruin system performance and are not fairly shared -Even in NoCs with infinite capacity -The network intensifies the problem -But can also provide tools for resolving it Simple mechanism achieves dramatic improvement -Completely eliminating the HM effects Hot-Modules, Cool NoCs!

May 2007Hot-Modules in Wormhole NoCs27 Thank you! Questions? Hot-Modules, Cool NoCs! QNoC Research Group Group Research QNoC