1. Design of a Diversified Router: Dedicated CRF plus IPv4 Metarouter
John DeHart

2. Revision History 5/22/06 (JDD): 6/1/06 (JDD): 6/2/06 (JDD):
Created
Buffer descriptor stuff probably needs updating.
6/1/06 (JDD):
Updating data going between blocks, still in progress.
6/2/06 (JDD):
More cleanup of data going between blocks.
Buffer descriptor details still need updating.
6/5/06 (JDD):
Slight change to format for Lookup Key and defining what goes in each word in the NN ring.
Add IP Pkt Length to data Demux passes to Parse
6/6/06 (JDD):
Reorganized the Lookup Result given to Hdr Format to distinguish between MR portion and Substrate portion.
Clean up labeling of data to Parse (MN vs. IP Pkt)
Output from Parse is still IP Pkt Offset and Length.
Data from Parse to Lookup needs update to reflect case where lookup is just for Substrate mapping of MI to LC.
6/7/06 (JDD):
Updated notes about Parse block’s input/output and functionality
6/15/06 (JDD):
Removed CRC from Rx to Demux data.
MSF does not pass us a CRC like we thought so we will skip the CRC checking.
Updated data going from Demux to Parse, Parse to Lookup and Lookup to Hdr Format

3. Revision History 6/19/06 (BDH):
Split Header Format into MR Header Format and Substrate Encap
Demux is now Substrate Decap
Reorganization of all slides into logical and physical formats, coloring scheme
IPv4 MR now has own section, integrated JL’s internal format slides

4. Dedicated CRF Slide Organization
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Metarouter
Block
Input Data
Output Data
Substrate
In the “at-a-glance” format, all blocks are logical
Logical inputs and outputs
High-level overview of processing
Each logical block is like an Intel microblock not necessarily an ME.
In the detailed format, all blocks are physical
Physical inputs and outputs
Specific functionality and implementation notes
Color scheme
Blue = Substrate, should not change!
Green = Metarouter, different for each MR

5. Logical Formats

6. Receive Rx Coordinate transfer of packets from RBUF to DRAM Lookup RxTx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3 Rx
Buffer Handle
RBUF
Ethernet Frame Len
Port
Coordinate transfer of packets from RBUF to DRAM

7. Substrate Decapsulate
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle
Block
Buffer Handle
Substrate Type
Ethernet Frame Len
Src ID
Port
MN Frame Length
MN Frame Offset
Read and validate Ethernet header from DRAM
Read and validate substrate header from DRAM
Extract Rx MI or Source MPE
Calculate MN frame length and offset

8. Metarouter Parse MR Parse
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle
Buffer Handle
MR Parse
Lookup Flags
Parse Flags
to Lookup
Lookup Key
Src ID (RxMI/SrcMPE)
Src ID
MN Frame Length
MN Pkt Length
MN Frame Offset
to MR Hdr Format
MR Data
Read and align MN header (includes IPv4 Hdr) from DRAM
MR-specific
Consume internal header (if packet from other MPE of MR)
Header validation
Header modification
Exception checks
Extract lookup key and set lookup flags
Write aligned modified IPv4 header back to DRAM

9. Lookup Lookup Perform lookup in TCAMRx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle
Buffer Handle
Lookup
Lookup Result Flags
Lookup Input Flags
Dest Addr
Lookup Key
Output Port
Src ID
QID
MN Pkt Length
MR Lookup Result
Perform lookup in TCAM
Increment counters based on Stats Index
Priority resolution of results from multiple databases, if needed

10. Metarouter Header Format
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle
Buffer Handle
Lookup Result Flags
MR Hdr Format
MN Frame Length
from Lookup
Dest Addr
MN Frame Offset
Output Port
Dest Addr
QID
Output Port
MR Lookup Result
QID
Substrate Type
from MR Parse
MR Data
Substr. Type-dep. Data
Process Lookup result
For exceptions, generate internal header
Decide substrate type

11. Substrate Encapsulation
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle
MN Frame Length
Substr Encap
Buffer Handle
MN Frame Offset
Output Port
Dest Addr
QID
Output Port
MN Frame Length
QID
Substrate Type
Substr. Type-dep. Data
Write substrate and ethernet headers

12. Queue Manager CRF queue management for Meta Interface queues WRR?
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Buffer Handle QM
Output Port
Buffer Handle
QID
Output Port
MN Frame Length
CRF queue management for Meta Interface queues
WRR?
Details

13. Transmit Tx Coordinate transfer of packets from DRAM to TBUFs
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3 Tx
Buffer Handle
TBUF
Output Port
Coordinate transfer of packets from DRAM to TBUFs
Recycle buffer handle

14. Physical Formats

15. Receive RBUF format details here Buf Handle details here Notes:
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
RBUF
Buf Handle(32b)
Port
(8b)
Reserved
Eth. Frame
Len (16b)
RBUF format details here
Buf Handle details here
Notes:
We’ll pass the Buffer Handle which contains the SRAM address of the buffer descriptor.
From the SRAM address of the descriptor we can calculate the DRAM address of the buffer data.

16. Substrate Decapsulate
Buf Handle(32b)
Port
(8b)
Reserved
Eth. Frame
Len (16b)
Buf Handle(32b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size P
Flags
(4b)
MR ID (VLAN) (12b)
Rx MI(16b)
MN Frm Length(16b)
MN Frm Offset (16b)
P Flags: indicate RxMI vs. SrcMPE
bit0: 0: RxMI , 1: ScrMPE

17. Lookup Key[143-112] MR/MI (32b)
Metarouter Parse
Buf Handle(32b)
Buf Handle(32b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size L
Flags
(4b)
MR Passthrough (28b) P
Flags
(4b)
MR ID (VLAN) (12b)
RxMI/SrcMPE
(16b)
MR Passthrough (32b)
MN Pkt Length (16b)
MN Pkt Offset (16b)
Lookup Key[ ] MR/MI (32b)
Lookup Key[111-80] (32b)
Lookup Key[ 79-48] (32b)
Lookup Key[ 47-16] (32b)
Lookup Key
[15- 0] (16b)
Reserved (16b)
L Flags:
bit 0: 0: Normal, 1: Substrate Lookup
bit 1: 0: Normal, 1: NH MN Address present in Key Word[1]
Key Word[0] = MR/MI
Bit 1 should never be set without bit 0 also being set.
Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer.
Can Parse adjust the buffer/packet size and offset?
Can Parse do something like, terminate a tunnel and strip off an outer header?
Rx MI needs to be passed to Header Format (through Lookup) so that Header Format can include it in the shim of packets that end up on the slow path. This will allow the Control Processor know what interface the exception packets arrived on.

18. Lookup Key[143-112] MR/MI (32b)
Buf Handle(32b)
Buf Handle(32b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size L
Flags
(4b)
MR Passthrough (28b) H
Flags
(4b)
MR Passthrough (28b)
MR Passthrough (32b)
MR Passthrough (32b)
Lookup Key[ ] MR/MI (32b)
MR Lookup Result (32b)
Lookup Key[111-80] (32b)
Lookup Key[ 79-48] (32b)
MR Lookup Result (32b)
Lookup Key[ 47-16] (32b)
DA(8b)
Port
(4b)
QID(20b)
Lookup Key
[15- 0] (16b)
Reserved (16b)
L Flags:
bit 0: 0: Normal, 1: Substrate Lookup
bit 1: 0: Normal, 1: NH MN Address present in Key Word[1]
Key Word[0] = MR/MI
Bit 1 should never be set without bit 0 also being set.

19. Metarouter Header Format
Buf Handle(32b)
Buffer Handle(32b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size H
Flags
(4b)
MR Passthrough (28b)
Port(8b)
Rsv
(4b)
QID(20b)
MR Passthrough (32b)
MN Pkt Offset (8b)
MN Pkt Length (16b)
MR Lookup Result (32b)
SH Type
(8b)
SH Len
(8b)
Rsv(16b)
MR Lookup Result (32b)
DA(8b)
Port
(4b)
QID(20b)
Substrate Header Data (LWO)
Substrate Header Data (LW1)
Substrate Header Data (LW2)
Text
Substrate Header Data (LW3)
Substrate Header Data (LW4)

20. Substrate Encapsulation
Buffer Handle(32b)
QID(20b)
MN Pkt Length (16b)
Buffer Handle(32b)
Rsv
(4b)
Reserved (16b)
Port(8b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Port(8b)
Rsv
(4b)
QID(20b)
MN Pkt Offset (8b)
MN Pkt Length (16b)
SH Type
(8b)
SH Len
(8b)
Rsv(16b)
Substrate Header Data (LWO)
Substrate Header Data (LW1)
Substrate Header Data (LW2)
Substrate Header Data (LW3)
Substrate Header Data (LW4)
Substrate header types/formats here?

21. Queue Manager Text QID(20b) Buffer Handle(32b) Buffer Handle(32b) VLAN
MN Pkt Length (16b)
Buffer Handle(32b)
Rsv
(4b)
Reserved (16b)
Port(8b)
Buffer Handle(32b)
Port(8b)
Reserved (24b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Text

22. Transmit Text VLAN Packet_Next MR_ID TxMI Free_List Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Buffer Handle(32b)
Port(8b)
Reserved (24b)
TBUF
Text

23. MR Parse and MR Header Format Logical and Physical Data Formats
IPv4 Metarouter
MR Parse and MR Header Format
Logical and Physical Data Formats

24. IPv4 Logical Formats MR Parse Lookup MR Hdr Format Exception Bits
Rx MI / Source MPE
MN Frame Length
MN Frame Offset
MR Parse
MR Hdr Format
Lookup
Buffer Handle
Buffer Handle
Buffer Handle
Buffer Handle
Parse Flags
Lookup Flags
Lookup Result Flags
MN Frame Length
Rx MI / Source MPE
Lookup Key
Dest Addr
MN Frame Offset
MN Frame Length
Rx MI / Tx MI
Output Port
Dest Addr
MN Frame Offset
QID
Output Port
IPv4 Lookup Flags
QID
TxMI
Substrate Type
IPv4 Lookup Result
Substr. Type-dep. Data

25. Lookup Key[ 47-16] Ports (32b) Lookup Key[143-112] MR/MI (32b)
IPv4 Physical Formats
RxMI/SrcMPE
(16b)
Buf Handle(32b)
MR ID (VLAN) (12b)
MN Pkt Offset (16b)
MN Pkt Length (16b) P
Flags
(4b)
IP Pkt Length (16b)
Buffer Handle(32b)
Reserved (16b)
QID(20b)
Rsv
(4b)
Port(8b)
MR Parse
MR Hdr Format
Lookup
Lookup Key[111-80] DA (32b)
Rx MI/SrcMPE
(16b)
Buf Handle(32b)
IP Pkt Length (16b)
IP Pkt Offset (16b)
Lookup Key[ 79-48] SA (32b)
Lookup Key[ 47-16] Ports (32b)
Lookup Key
Proto/TCP_Flags
[15- 0] (16b)
Exception
Bits (12b)
Reserved (16b)
Lookup Key[ ] MR/MI (32b) L
Flags
(4b)
Port
(4b)
TxMI(16b)
QID(20b)
DA(8b)
MrBits[31:0](32b)
Buf Handle(32b)
IP Pkt Length (16b)
IP Pkt Offset (16b)
Rx MI/SrcMPE
(16b)
Reserved
(11b) H
(1b) D
Exception
Bits (12b)
Flags N M A C L
H: Hit
D: Drop

26. IPv4 Parse and Header Format
Consume internal header
Verify IP header (per RFC )
Decrement TTL
Recalc IP checksum
Write updated checksum to DRAM
Store drop/exception/pass statistics
Extract lookup key, set lookup flags
Header Format

27. IPv4 Flag Formats IPv4 Lookup Flags IPv4 Exception Bits H: Hit D: Drop
NH: NH MN Address present
MAC: MAC Address needed
LD: Local Delivery
IPv4 Exception Bits
Bit 0: TTL = 0 or 1
Bit 1: Options
Bit 2?: RxMI or SrcMPE

28. IPv4 Internal Header Formats
For packets going from IPv4 MPE to IPv4 MPE
Packets from Ingress LC or to Egress LC don’t have Metanet Internal Header
Type (1B)
Length (1B)
Type Dependent Data (2-6B)
RxMI (2B)

29. IPv4 Internal Header Types
Packets entering routing MPE
Routing MPE: MPE that does routing lookup
FwdKey = [TxMI + MnNhAddr if multi-access link] (**)
Source
Category
Type Value
Reason
Incoming MR Internal Hdr
RMPE Action
Ingress LC
Normal Fwd
No MR Int Hdr
Classify and fwd
MPE
No Classify (w/ FwdKey**)
0x00
If from Control MPE, it is an original pkt that is reinjected back to data path
RxMI + FwdKey
Perform substrate lookup to resolve LCAddr, port and QID
Classify (w/o FwdKey)
0x01
New pkt generated by MPE. If from Control MPE, it is ICMP or local traffic
RxMI

30. IPv4 Internal Header Types
Packets exiting Routing MPE
If FwdKey in MR Internal hdr is invalid, lookup raises error flag and hdr_fmt sends pkt to CMPE for debug (***)
Destination
Category
Type Value
Reason
Outgoing MR Internal Hdr
Egress LC
Fast path
No MR Int Hdr
Regular MPE
0x02
RxMI + FwdKey
Control MPE
Exception
0x03
No route
RxMI
0x04
TTL expire
0x05
IP w/ options
0x06
Redirect due to RxMI = TxMI
Control
0x07
Inspect
0x08
Local delivery
Debug
0x09
Monitor
0x0a
Log due to error in pkts***

31. Extra
The next set of slides are for templates or extra information if needed

32. Text Slide Template

33. Image Slide Template

34. At-a-glance Block Template
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Block
Buffer Handle
RBUF
Ethernet Frame Len
Port
Text

35. Detailed Block Template
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
RBUF
Buf Handle(32b)
Port
(8b)
Reserved
Eth. Frame
Len (16b)
Text

36. QM/Scheduler on Multiple MEs
Header Format
Input
Hlpr
(1 ME)
QM/Schd
(1 ME) Tx
MR-1
MR-n
. . .
QM/Schd
(1 ME) Tx
QID(20b)
IP Pkt Length (16b)
Buffer Handle(32b)
Rsv
(4b)
Reserved (16b)
Port(8b)
NN/Scratch Rings
Buffer Handle(32b)
Port(8b)
Reserved (24b)
NN Ring
QID(32b):
Reserved (8b)
QM ID (3b)
QID(17b): 1M queues per QM
Input Hlpr would use QM ID to select Scratch ring on which to put request.
QM/Sched then sends on its output NN/scratch ring to its associated Tx
With 64 entries in Q-Array and 16 entries in CAM, max number of QM/Schds is probably 4 (2 bits).
We’ll set aside 3 bits to give us flexibility in the future.

37. Packet Buffer Descriptor Tradeoffs
Why use a Buffer Descriptor at all?
QM needs something to link packets/buffers in queues
ME-to-ME communications costs vs. SRAM access costs
Specific to Radisys, from dl_meta.u

38. Packet Buffer Descriptor def
Meta Data structure of Packet Buffers (LSB to MSB)
buffer_next 32 bits Next Buffer Pointer (in a chain of buffers)
offset bits Offset to start of data in bytes
BufferSize 16 bits Length of data in the current buffer in bytes
header_type bits type of header at offset bytes in to the buffer
rx_stat bits Receive status flags
free_list bits Freelist ID
packet_size 16 bits (Total packet size across multiple buffers)
output_port 16 bits Output Port on the egress processor
input_port 16 bits Input Port on the ingress processor
nhid_type bits Nexthop ID type.
reserved bits Reserved
fabric_port bits Output port for fabric indicating blade ID.
nexthop_id 16 bits NextHop IP ID
color bits Qos Color
flow_id 24 bits QOS flow ID or MPLS label/flow id
reserved 16 bits Reserved
class_id 16 bits Class ID
packet_next 32 bits pointer to next packet (unused in cell mode)
Specific to Radisys, from dl_meta.u

39. Packet Buffer Descriptor Gets
buffer_next: tx
Offset: rx, tx, fwd
BufferSize: tx, fwd
header_type: tx, fwd
rx_stat: NONE
free_listpacket_size: NONE
output_port: qm(?), tx
input_port: rx, fwd
nhid_type: NONE
fabric_port: qm(?), tx
nexthop_id
color
flow_id
class_id
packet_next
Specific to Radisys, from dl_meta.u

40. Meta Data Caching Meta Data can be cached in one of three places:
SRAM Xfer Registers
DRAM Xfer Registers
GPR Registers
Size of Meta Data Cache is controlled by #define META_CACHE_SIZE
Macro dl_meta_load_cache[] loads meta data cache
buffer_handle: buffer handle for which meta data is to be fetched
dl_meta: read transfer register prefix
Xbuf_alloc[] should be used to allocate the needed registers
signal_number:
START_LW: starting long word for fetch
NUM_LW: number of long words to fetch
Each microengine (microblock?) can use Meta Data Caching differently.
Specific to Radisys, from dl_meta.u

41. Meta Data Caching Specific to Radisys, from dl_meta.u
In the ipv4_v6_forwarder sample app,
dl_meta_load_cache() used in:
Egress
ethernet_arp.uc
pkt_tx_16p.uc
statistics_util.uc
tx_helper.uc
Ingress
dl_meta_get_*[] used in:
Ether.uc
Ipv4_fwder.uc
Ipv4_fwder_util.uc
Ipv6_fwder.uc
V6v4_tunnel_decap.uc
V6v4_tunnel_encap.uc
dl_meta_set_*[] used in:
pkt_rx_init.uc
pkt_rx_two_me_util.uc
Specific to Radisys, from dl_meta.u

42. Buffer Handle

43. Buffer Descriptor Usage
Is there a different Buffer Descriptor defn for LC and PE?
Will we support Multi-Buffer Packets?
If not, we do not need buffer_next(32b) or buffer_size(16b)
QM uses packet_next for its packet chaining in qarray.
Output Port and Input Port probably translate to TxMI and RxMI
Next Hop fields (nhid_type(4b) and nexthop_id(16b)) probably can go away.
QOS fields (color(8b) and flow_id(24b)) probably can go away.
Two reserved fields 4b and 16b can go away.
class_id(16b) (virtual queue id?) can probably go away.
fabric_port can probably go away.

44. Buffer Descriptor Usage
PE Buffer Descriptor:
MR_ID (16b)
TxMI (16b)
VLAN (16b)
buffer_next 32 bits Next Buffer Pointer (in a chain of buffers)
offset bits Offset to start of data in bytes
BufferSize 16 bits Length of data in the current buffer in bytes
header_type bits type of header at offset bytes in to the buffer
rx_stat bits Receive status flags
free_list bits Freelist ID
packet_size 16 bits (Total packet size across multiple buffers)
output_port 16 bits Output Port on the egress processor
input_port 16 bits Input Port on the ingress processor
nhid_type bits Nexthop ID type.
reserved bits Reserved
fabric_port bits Output port for fabric indicating blade ID.
nexthop_id 16 bits NextHop IP ID
color bits Qos Color
flow_id 24 bits QOS flow ID or MPLS label/flow id
reserved 16 bits Reserved
class_id 16 bits Class ID
packet_next 32 bits pointer to next packet (unused in cell mode)

45. Buffer Descriptor Usage
PE Buffer Descriptor:
LW0: buffer_next 32 bits Next Buffer Pointer (in a chain of buffers)
LW1: offset bits Offset to start of data in bytes
LW1: BufferSize 16 bits Length of data in the current buffer in bytes
LW2: reserved bits reserved/unused
LW2: reserved bits reserved/unused
LW2: free_list bits Freelist ID
LW2: packet_size 16 bits (Total packet size across multiple buffers)
LW3: MR_ID bits Meta Router ID
LW3: TxMI bits Transmit Meta Interface
LW4: VLAN bits VLAN
LW4: reserved 16 bits reserved/unused
LW5: reserved 32 bits reserved/unused
LW6: reserved 32 bits reserved/unused
LW7: packet_next 32 bits pointer to next packet (unused in cell mode)
Leave multi-buffer fields there as a template for the dedicated blade implementation of a jumbo-frame MR.
Also reduces changes to Rx, Tx, and QM and reduces potential problems.

46. Multicast Alternatives
At least Three Options
Force MRs that need Multicast to be Dedicated Blade MRs and do their own Multicast
For our short term goals this is probably sufficient and the best course.
Perhaps longer term we can look at adding it to the CRF
Treat as exception and send to Xscale
Provide support in CRF for Multicast
Use Multi-Hit Lookup capability of the TCAM
MI Bit mask defined in Lookup Result
Will put a bound on the number of MIs that can be supported on an MR because of the size of the lookup result.
Has issues of mapping bits in the bit mask to actual MIs.
Lookup Result contains an index into a table containing MI bit masks
Allow but do not force MRs to provide code to interpret Lookup Result.
This would also allow other possible extensions on an MR-specific basis
This carries with it the problem of bounding the execution time of the MR-specific code in the Lookup block. For general multicast, this could be a serious issue.
There are also issues with generating a QID based on an MI when the QID is not included in the Lookup Result.
Other options?

47. CRF Support for Multicast
Default/Unicast path MR
Interp
Parse
Header Format
MR-Specific
Path
Post Process
Lookup
MR-1
MR-1
MR-n
. . .
. . .
MR-n
DRAM Buf Ptr
MR Id
Input MI
MR Ctrl Blk Ptr
MR Mem Ptr
DRAM Buf Ptr
MR Id
MR Lookup Key
MR Ctrl Blk Ptr
MR Mem Ptr
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
DRAM Buf Ptr
MR Id
Output MI
Buffer Offset
QID

48. CRF Support for Multicast
Default path MR
Interp
MR-Specific
Path
Post Process
Lookup
DRAM Buf Ptr
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
Copy Cnt
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
Copy Cnt=1
MR Id
MR Lookup Key
MR Ctrl Blk Ptr
MR Mem Ptr
We will need some kind of copy count or multicast bit and last copy bit to let TX know when it can release the DRAM buffer that holds the packet.

49. CRF Support for Multicast
Default path MR
Interp
MR-Specific
Path
Post Process
Lookup
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
Copy Cnt
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
Copy Cnt
DRAM Buf Ptr
MR Id
Output MI
QID
MR Specific
Lookup Result
Stats Index
MR Ctrl Blk Ptr
MR Mem Ptr
Copy Cnt
DRAM Buf Ptr
DRAM Buf Ptr
Output MI
Copy Cnt
Output MI
Copy Cnt
MR Id
MR Lookup Key
Output MI
Copy Cnt
MR Lookup Key
MR Specific
Lookup Result
MR Ctrl Blk Ptr
MR Ctrl Blk Ptr
MR Mem Ptr
MR Mem Ptr
We will need some kind of copy count or multicast bit and last copy bit to let TX know when it can release the DRAM buffer that holds the packet.

50. OLD
The rest of these are old slides that should be deleted at some point.

51. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
RBUF
Buf Handle(32b)
Rx:
Function
Coordinate transfer of packets from RBUF to DRAM
Notes:
We’ll pass the Buffer Handle which contains the SRAM address of the buffer descriptor.
From the SRAM address of the descriptor we can calculate the DRAM address of the buffer data.

52. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
Buf Handle(32b)
MR Id(16b)
Input MI(16b)
MR Mem Ptr(32b)
Buf Handle(32b)
DRAM Buf Ptr(32b)
Buffer Offset(16b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
DeMux
Function
Read Pkt Header from DRAM
Use VLAN from Ethernet header to determine destination MR in order to locate:
MR Parse code
MR specific memory pointers
Write MR Id to Buffer Descriptor
Write VLAN to Buffer Descriptor

53. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
MR Id(16b)
Input MI(16b)
MR Mem Ptr(32b)
Buf Handle(32b)
DRAM Buf Ptr(32b)
Buffer Offset(16b)
Buf Handle(32b)
DRAM Buf Ptr(32b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Buffer Offset(16b)
MR Id(16b)
Input MI(16b)
MR Mem Ptr(32b)
MR Lookup Key(16B)
Parse
Function
MR-specific header processing
Generate MR-specific lookup key (16 Bytes) from packet
Need CRF functionality to managed multiple MRs in shared PE.
Notes:
Can Parse adjust the buffer/packet size and offset?
Can Parse do something like, terminate a tunnel and strip off an outer header?

54. CRF Wrapper Around Parse
MR-1
MR-n
. . . MR
Selector
MR Id
Input MI
MR Mem Ptr
Buf Handle(32b)
DRAM Buf Ptr
Buffer Offset
MR Lookup Key
MR Id
Input MI
MR Mem Ptr
Buf Handle(32b)
DRAM Buf Ptr
Buffer Offset
DRAM Buf Ptr
Input MI
MR Mem Ptr
Buffer Offset
MR Lookup Key
Buffer Offset

55. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
MR Lookup Key(16B)
MR Id(16b)
Input MI(16b)
MR Mem Ptr(32b)
Buf Handle(32b)
DRAM Buf Ptr(32b)
Buffer Offset(16b)
Buffer Handle(32b)
MR Id(16b)
Lookup Result(Nb)
MR Mem Ptr(32b)
DRAM Buf Ptr(32b)
Buffer Offset(16b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Lookup
Function
Perform lookup in TCAM based on MR Id and lookup key
Result:
Output MI
QID
Stats index
MR-specific Lookup Result (flags, etc. ?)
How wide can/should this be?

56. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
Buffer Handle(32b)
Buffer Handle(32b)
QID(16b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
DRAM Buf Ptr(32b)
Header Format
Function
MR specific packet header formatting
MR specific Lookup Result processing
Drop and Miss bits
Need CRF functionality to managed multiple MRs in shared PE.
Pulls out QID, Length and Port from MR Result, etc.
Checks for Drop and Miss bits and deals with those actions.
Buffer Offset(16b)
Size (16b)
Port(8b)
MR Id(16b)
MR Mem Ptr(32b)
Lookup Result(Nb)
Includes drop
and miss bits

57. CRF Wrapper Around Header Format
MR-1
MR-n
. . .
Buffer Handle MR
Selector
Buffer Handle
QID
Size
Port
DRAM Buf Ptr(32b)
Buffer Offset
MR Id
DRAM Buf Ptr
Output MI
MR Specific
Lookup Result
MR Mem Ptr
Buffer Offset
MR Mem Ptr
Buffer Offset
Gets written to Buffer Descriptor
May also cause size(s) in
Descriptor to be updated.
(what about trimming data,
What if it is a buffer’s worth
Which would change the chaining,
Can they add/trim at either end?
Lookup Result

58. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
Buffer Handle(32b)
Buf Handle(32b) QM
Function
CRF queue management for Meta Interface queues
For performance reasons, QM may actually be implemented as multiple instances
Each instance on a separate ME would support a separate set of Meta Interfaces.
See next slide for more details…
QID(16b)
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size
Size (16b)
Port(8b)

59. QM/Scheduler on Multiple MEs
Header Format
Input
Hlpr
(1 ME)
QM/Schd
(1 ME)
Output
Hlpr
(1 ME) Tx
MR-1
MR-n
. . .
. . .
QM/Schd
(1 ME)
Buffer Handle(32b)
QID(32b)
Buf Handle(32b)
Scratch Rings
Size (16b)
NN Ring
NN Ring
Port(8b)
QID(32b):
Reserved (8b)
QM ID (4b)
QID(20b): 1M queues per QM
Input Hlpr would use QM ID to select Scratch ring on which to put request.
Output Hlpr would process all Scratch rings coming from QM/Schd MEs and multiplex onto one NN ring to TX
With 64 entries in Q-Array and 16 entries in CAM, max number of QM/Schds is probably 4 (2 bits).
We’ll set aside 4 bits to give us flexibility in the future.

60. Common Router Framework (CRF) Functional Blocks
Parse
Header Format Rx
DeMux
Lookup QM Tx
MR-1
. . .
MR-1
MR-n
. . .
MR-n
Buffer Handle(32b)
TBUF
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size Tx
Function
Coordinate transfer of packets from DRAM to TBUF

61. Old Template Tx Function Rx DeMux Parse Lookup Header Format QM Tx
Buffer Handle(32b)
Port(8b)
Reserved (24b)
TBUF
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size Tx
Function
Coordinate transfer of packets from DRAM to TBUF

62. Old Rejected Overly Busy Slide
Lookup Rx Tx QM MR
Parse
MR Hdr
Format
Substr
Decap
Encap L1 L2 L3
Block
 Logical interface
Data passing between layers
Notes here
 Physical format
Actual Format of data
Shows type of communication
Scratch Ring
NN Ring
SRAM Rings
 Buf Descriptor
shows fields read/written
Input Data
Output Data
Input Data
Output Data
VLAN
Packet_Next
MR_ID
TxMI
Free_List
Packet_Size
Buffer_Next
Offset
Buffer
Descriptor
Buffer_Size