1. IP Lookup: Some subtle concurrency issues Arvind Computer Science & Artificial Intelligence Lab Massachusetts Institute of Technology February 22, 2011L06-1 http://csg.csail.mit.edu/6.375

2. IP Lookup block in a router Queue Manager Packet Processor Exit functions Control Processor Line Card (LC) IP Lookup SRAM (lookup table) Arbitration Switch LC A packet is routed based on the “Longest Prefix Match” (LPM) of it’s IP address with entries in a routing table Line rate and the order of arrival must be maintained line rate  15Mpps for 10GE February 22, 2011 L06-2http://csg.csail.mit.edu/6.375

3. 18 2 3 IP addressResultM Ref 7.13.7.3F 10.18.201.5F 7.14.7.2 5.13.7.2E 10.18.200.7C Sparse tree representation 3 A … A … B C … C … 5 D F … F … 14 A … A … 7 F … F … 200 F … F … F* E5.*.*.* D10.18.200.5 C10.18.200.* B7.14.7.3 A7.14.*.* F … F … F F … E 5 7 10 255 0 1 4 4A In this lecture: Level 1: 16 bits Level 2: 8 bits Level 3: 8 bits  1 to 3 memory accesses February 22, 2011 L06-3http://csg.csail.mit.edu/6.375

4. “C” version of LPM int lpm (IPA ipa) /* 3 memory lookups */ { int p; /* Level 1: 16 bits */ p = RAM [ipa[31:16]]; if (isLeaf(p)) return value(p); /* Level 2: 8 bits */ p = RAM [ptr(p) + ipa [15:8]]; if (isLeaf(p)) return value(p); /* Level 3: 8 bits */ p = RAM [ptr(p) + ipa [7:0]]; return value(p); /* must be a leaf */ } Not obvious from the C code how to deal with - memory latency - pipelining … 2 16 -1 0 … … 2 8 -1 0 … 0 Must process a packet every 1/15 s or 67 ns Must sustain 3 memory dependent lookups in 67 ns Memory latency ~30ns to 40ns February 22, 2011 L06-4http://csg.csail.mit.edu/6.375

5. Longest Prefix Match for IP lookup: 3 possible implementation architectures Rigid pipeline Inefficient memory usage but simple design Linear pipeline Efficient memory usage through memory port replicator Circular pipeline Efficient memory with most complex control Designer’s Ranking: 123 Which is “best”? February 22, 2011 L06-5http://csg.csail.mit.edu/6.375 Arvind, Nikhil, Rosenband & Dave [ICCAD 2004]

6. Circular pipeline The fifo holds the request while the memory access is in progress The architecture has been simplified for the sake of the lecture. Otherwise, a “completion buffer” has to be added at the exit to make sure that packets leave in order. enter? done? RAM yes inQ fifo no outQ February 22, 2011 L06-6http://csg.csail.mit.edu/6.375

7. interface FIFO#(type t); method Action enq(t x); // enqueue an item method Action deq(); // remove oldest entry method t first(); // inspect oldest item endinterface FIFO n = # of bits needed to represent a value of type t not full not empty rdy enab n n rdy enab rdy enq deq first FIFO module February 22, 2011 L06-7http://csg.csail.mit.edu/6.375

8. Addr Ready ctr (ctr > 0) ctr++ ctr-- deq Enable enq Request-Response Interface for Synchronous Memory Synch Mem Latency N interface Mem#(type addrT, type dataT); method Action req(addrT x); method Action deq(); method dataT peek(); endinterface Data Ack Data Ready req deq peek Making a synchronous component latency- insensitive February 22, 2011 L06-8http://csg.csail.mit.edu/6.375

9. rule recirculate (True); TableEntry p = ram.peek(); ram.deq(); IP rip = fifo.first(); if (isLeaf(p)) outQ.enq(p); else begin fifo.enq(rip << 8); ram.req(p + rip[15:8]); end fifo.deq(); endrule Circular Pipeline Code rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(ip[15:0]); inQ.deq(); endrule enter? done? RAM inQ fifo When can enter fire? inQ has an element and ram & fifo each has space done? Is the same as isLeaf February 22, 2011 L06-9http://csg.csail.mit.edu/6.375

10. rule recirculate (True); TableEntry p = ram.peek(); ram.deq(); IP rip = fifo.first(); if (isLeaf(p)) outQ.enq(p); else begin fifo.enq(rip << 8); ram.req(p + rip[15:8]); end fifo.deq(); endrule Circular Pipeline Code: discussion rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(ip[15:0]); inQ.deq(); endrule enter? done? RAM inQ fifo When can recirculate fire? ram & fifo each has an element and ram, fifo & outQ each has space Is this possible? February 22, 2011 L06-10http://csg.csail.mit.edu/6.375

11. Ordinary FIFO won’t work but a pipeline FIFO would February 22, 2011L06-11 http://csg.csail.mit.edu/6.375

12. Problem solved! rule recirculate (True); TableEntry p = ram.peek(); ram.deq(); IP rip = fifo.first(); if (isLeaf(p)) outQ.enq(p); else begin fifo.enq(rip << 8); ram.req(p + rip[15:8]); end fifo.deq(); endrule PipelineFIFO fifo <- mkPipelineFIFO; // use a Pipeline fifo RWire has been safely encapsulated inside the Pipeline FIFO – users of the fifo need not be aware of RWires February 22, 2011 L06-12http://csg.csail.mit.edu/6.375

13. Dead cycles rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(ip[15:0]); inQ.deq(); endrule enter? done? RAM inQ fifo rule recirculate (True); TableEntry p = ram.peek(); ram.deq(); IP rip = fifo.first(); if (isLeaf(p)) outQ.enq(p); else begin fifo.enq(rip << 8); ram.req(p + rip[15:8]); end fifo.deq(); endrule Can a new request enter the system when an old one is leaving? Is this worth worrying about? assume simultaneous enq & deq is allowed February 22, 2011 L06-13http://csg.csail.mit.edu/6.375

14. The Effect of Dead Cycles enter done? RAM yes in fifo no What is the performance loss if “exit” and “enter” don’t ever happen in the same cycle? >33% slowdown!Unacceptable Circular Pipeline RAM takes several cycles to respond to a request Each IP request generates 1-3 RAM requests FIFO entries hold base pointer for next lookup and unprocessed part of the IP address February 22, 2011 L06-14http://csg.csail.mit.edu/6.375

15. Scheduling conflicting rules When two rules conflict on a shared resource, they cannot both execute in the same clock The compiler produces logic that ensures that, when both rules are applicable, only one will fire Which one? source annotations (* descending_urgency = “recirculate, enter” *) February 22, 2011 L06-15http://csg.csail.mit.edu/6.375

16. So is there a dead cycle? rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(ip[15:0]); inQ.deq(); endrule enter? done? RAM inQ fifo rule recirculate (True); TableEntry p = ram.peek(); ram.deq(); IP rip = fifo.first(); if (isLeaf(p)) outQ.enq(p); else begin fifo.enq(rip << 8); ram.req(p + rip[15:8]); end fifo.deq(); endrule In general these two rules conflict but when isLeaf(p) is true there is no apparent conflict! February 22, 2011 L06-16http://csg.csail.mit.edu/6.375

17. Rule Spliting rule foo (True); if (p) r1 <= 5; else r2 <= 7; endrule rule fooT (p); r1 <= 5; endrule rule fooF (!p); r2 <= 7; endrule  rule fooT and fooF can be scheduled independently with some other rule February 22, 2011 L06-17http://csg.csail.mit.edu/6.375

18. Spliting the recirculate rule rule recirculate (!isLeaf(ram.peek())); IP rip = fifo.first(); fifo.enq(rip << 8); ram.req(ram.peek() + rip[15:8]); fifo.deq(); ram.deq(); endrule rule exit (isLeaf(ram.peek())); outQ.enq(ram.peek()); fifo.deq(); ram.deq(); endrule rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(ip[15:0]); inQ.deq(); endrule Now rules enter and exit can be scheduled simultaneously, assuming fifo.enq and fifo.deq can be done simultaneously February 22, 2011 L06-18http://csg.csail.mit.edu/6.375

19. Packaging a module: Turning a rule into a method inQ enter? done? RAM fifo rule enter (True); IP ip = inQ.first(); ram.req(ip[31:16]); fifo.enq(p[15:0]); inQ.deq(); endrule method Action enter (IP ip); ram.req(ip[31:16]); fifo.enq(ip[15:0]); endmethod outQ Similarly a method can be written to extract elements from the outQ February 22, 2011 L06-19http://csg.csail.mit.edu/6.375

20. IP-Lookup module with the completion buffer Completion buffer ensures that departures take place in order even if lookups complete out-of-order Since cbuf has finite capacity it gives out tokens to control the entry into the circular pipeline The fifo now must also hold the “token” while the memory access is in progress: Tuple2#(Token,Bit#(16)) done? RAM fifo enter getResult outQ cbuf yes no getToken remainingIP The packets may come out of order February 22, 2011 L06-20http://csg.csail.mit.edu/6.375

21. Completion buffer: Interface interface CBuffer#(type t); method ActionValue#(Token) getToken(); method Action put(Token tok, t d); method ActionValue#(t) getResult(); endinterface typedef Bit#(TLog#(n)) TokenN#(numeric type n); typedef TokenN#(16) Token; cbuf getResult getToken put (result & token) February 22, 2011 L06-21http://csg.csail.mit.edu/6.375

22. IP-Lookup module with the completion buffer module mkIPLookup(IPLookup); rule recirculate… ; rule exit …; method Action enter (IP ip); Token tok <- cbuf.getToken(); ram.req(ip[31:16]); fifo.enq(tuple2(tok,ip[15:0])); endmethod method ActionValue#(Msg) getResult(); let result <- cbuf.getResult(); return result; endmethod endmodule done? RAM fifo enter getResult cbuf yes no getToken for enter and getResult to execute simultaneously, cbuf.getToken and cbuf.getResult must execute simultaneously February 22, 2011 L06-22http://csg.csail.mit.edu/6.375

23. IP Lookup rules with completion buffer rule recirculate (!isLeaf(ram.peek())); match{.tok,.rip} = fifo.first(); fifo.enq(tuple2(tok,(rip << 8))); ram.req(ram.peek() + rip[15:8]); fifo.deq(); ram.deq(); endrule rule exit (isLeaf(ram.peek())); cbuf.put(ram.peek()); fifo.deq(); ram.deq(); endrule For rule exit and method enter to execute simultaneously, cbuf.put and cbuf.getToken must execute simultaneously  For no dead cycles cbuf.getToken and cbuf.put and cbuf.getResult must be able to execute simultaneously February 22, 2011 L06-23http://csg.csail.mit.edu/6.375

24. Completion buffer: Interface Requirements cbuf getResult getToken put (result & token) Rules and methods concurrency requirement to avoid dead-cycles: exit < getResult < enter  cbuf methods’ concurency: cbuf.getResult < cbuf.put < cbuf.getToken February 22, 2011 L06-24http://csg.csail.mit.edu/6.375

25. Completion buffer: Implementation module mkCBuffer (CBuffer#(t)) provisos (Bits#(t,sz)); RegFile#(Token, Maybe#(t)) buf <- mkRegFileFull(); Reg#(Token) i <- mkReg(0); //input index Reg#(Token) o <- mkReg(0); //output index Reg#(Int#(32)) cnt <- mkReg(0); //number of filled slots … IIVIVIIIVIVI cnt i o buf Elements must be representable as bits A circular buffer with two pointers i and o, and a counter cnt Elements are of Maybe type February 22, 2011 L06-25http://csg.csail.mit.edu/6.375

26. Completion buffer: Implementation Problem 1 IIVIVIIIVIVI cnt i o buf A circular buffer with two pointers i and o, and a counter cnt Elements are of Maybe type buf must allow two simultaneous updates and one read Needs a register file with one read and two write ports Since the updates are always to different addresses there is no data hazard and concurrent operations should be permitted No compiler can detect that without full program analysis (i.e., understanding the use pattern) February 22, 2011 L06-26http://csg.csail.mit.edu/6.375

27. Completion buffer: Implementation Problem 2 // state elements // buf, i, o, cnt... method ActionValue#(t) getToken() if (cnt < maxToken); cnt <= cnt + 1; i <= (i==maxToken) ? 0 : i+1; buf.upd(i, Invalid); return i; endmethod method Action put(Token tok, t data); buf.upd(tok, Valid data); endmethod method ActionValue#(t) getResult() if (cnt > 0) &&& (buf.sub(o) matches tagged (Valid.x)); o <= (o==maxToken) ? 0 : o + 1; cnt <= cnt - 1; return x; endmethod IIVIVIIIVIVI cnt i o Concurrent updates to cnt A good implementation is quite difficult February 22, 2011 L06-27http://csg.csail.mit.edu/6.375

28. Longest Prefix Match for IP lookup: 3 possible implementation architectures Rigid pipeline Inefficient memory usage but simple design Linear pipeline Efficient memory usage through memory port replicator Circular pipeline Efficient memory with most complex control Which is “best”? February 22, 2011 L06-28http://csg.csail.mit.edu/6.375 Arvind, Nikhil, Rosenband & Dave [ICCAD 2004]

29. Implementations of Static pipelines Two designers, two results LPM versions Best Area (gates) Best Speed (ns) Static V (Replicated FSMs) 88983.60 Static V (Single FSM) 22713.56 February 22, 2011 L06-29http://csg.csail.mit.edu/6.375 Replicated: RAM FSM MUX / De-MUX FSM Counter MUX / De-MUX result IP addr FSM RAM MUX result IP addr BEST: Each packet is processed by one FSM Shared FSM

30. Synthesis results LPM versions Code size (lines) Best Area (gates) Best Speed (ns) Mem. util. (random workload) Static V 22022713.5663.5% Static BSV 1792391 (5% larger)3.32 (7% faster)63.5% Linear V 410147594.799.9% Linear BSV 16815910 (8% larger)4.7 (same)99.9% Circular V 36481033.6299.9% Circular BSV 2578170 (1% larger)3.67 (2% slower)99.9% Synthesis: TSMC 0.18 µm lib - Bluespec results can match carefully coded Verilog - Micro-architecture has a dramatic impact on performance - Architecture differences are much more important than language differences in determining QoR February 22, 2011 L06-30http://csg.csail.mit.edu/6.375 V = Verilog; BSV = Bluespec System Verilog