© Sudhakar Yalamanchili, Georgia Institute of Technology (except as indicated) Switch Microarchitecture Basics

ECE 8813a (2) Reading 1.L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, L. S. Peh and W. J. Dally, “A Delay Model for Router Microarchitectures,” IEEE Micro, January-February Text: Sections 7.2.1, 7.2.2, and (pages )

ECE 8813a (3) Organization Routing, arbitration, switching Buffer management and flow control Concurrent, pipelined, speculative implementation of basic switch functions

ECE 8813a (4) Major Switch Components Microarchitecture level operation Buffers/Virtual Channels: how? Routing: where? Arbitration: who? Scheduling/Allocation: when?

ECE 8813a (5) A Virtual Channel Switch Data plane Control plane From L. S. Peh and W. J. Dally, “A Delay Model for Router Microarchitectures,” IEEE Micro, January- February 2001

ECE 8813a (6) Routing Decisions Formally represented as a routing function  Mapping from input ports (channels) to output ports (channels)  Distinct for oblivious vs. adaptive routing Common implementation forms  Finite state machine  Table look up Centralized vs. distributed  Across input ports (virtual channels)

ECE 8813a (7) Some Operational Principles Data and control planes operate a three rates  Phit, flit, packet  Resources are allocated and de-allocated at these rates Fixed clock cycle model State Management  Of resources – allocation  Of data – mapping to resources Granularity of allocation/management key to deadlock freedom in pipelined switches

ECE 8813a (8) Pipelined Switch Microarchitecture CrossBar Stage 1Stage 2Stage 3Stage 4Stage 5 VC Allocation IB (Input Buffering) RC VCA SA ST & Output Buffering Input buffers DEMUX Physical channel Link Control Link Control Physical channel MUX DEMUX MUX Output buffers Link Control Output buffers Link Control Physical channel Physical channel DEMUX MUX DEMUX MUX Routing Computation Switch Allocation L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (9) Buffer States Input buffers  free, routing, VCA, transmitting, stalled (flow control)  Output port and output virtual channel  Flow control information: stop/go, credits Output Buffers:  transmitting, stalled (flow control), free  Input port and input virtual channel  Flow control information: stop/go, credits

ECE 8813a (10) Virtual Channel Allocation How are candidates for arbitration created  routing function  Alternatives depend on routing flexibility This is the point at which dependencies are create  when deadlock is avoided L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001 Deterministic routing Adaptive/Link Fully adaptive

ECE 8813a (11) Switch Allocator Separable allocator Separate allocator for speculative and non-speculative requests Output port allocated for packet duration Low state update rate Non-VC AllocatorVC Allocator L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (12) Switch Allocation Flits bid on a cycle basis for cross-bar slots  Possible to increase the granularity of bids or the duration which a crossbar port can be held  SA cannot create deadlock since ports are not held indefinitely Success in SA is accompanied by flow control updates  For example, transmitting credits Traversal of tail flit reinitializes, input channel, resets input/output buffer allocations L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (13) The Crossbar There are many alternative cross-bar designs L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (14) Speculation What can be speculated?  Cross bar pending VC allocation  More complex for adaptive routing protocols Speculative flits vs. non-speculative  Header flits vs. body & tail flits Overhead of speculation  High traffic loads masks failures in speculation  Low traffic loads increase probability of success

ECE 8813a (15) Pipeline Disruptions Resource availability disruptions  VC availability  Downstream buffer space not available  Inter-packet gap is a function of deadlock freedom (later) Allocated flow disruptions  Switch not available  Downstream buffer space not available Disruptions (pipeline bubbles) propagate to the destination through intermediate routers

ECE 8813a (16) A Look at Channel Dependencies Issue: creating structural dependencies  Dependencies between messages due to concurrent use of VC buffers  Such dependencies must be globally managed to avoid deadlock Architectural decision: when is a VC freed?  When the tail flit releases an input virtual channel  When the tail releases the output virtual channel oRemember a VC traverses a link!

ECE 8813a (17) Base Performance L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (18) Buffer Occupancy Deeper pipelining increases the buffer turnaround time and decreases occupancy L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (19) Impact of Flow Control L. S. Peh and W. J. Dally, “A Delay Model and Speculative Architecture for Pipelined Routers,” Proc. of the 7th Int’l Symposium on High Performance Computer Architecture, Monterrey, January, 2001

ECE 8813a (20) What Next? Buffer organization Arbitration Switching scheduling On-chip vs. off-chip implementations