1. Design of a Diversified Router: Packet Formats
John DeHart

2. Revision History 6/2/06 (JDD): 6/6/06 (JDD): 6/13/06 (JDD): Created
Modified Internal Frame formats
Added some notes about handling of frames with IPv4 MR Internal Headers.
6/13/06 (JDD):
Updated notes on handling frames with IPv4 MR Internal Headers.

3. External Frame Formats: with VLAN
DstAddr (6B)
Type=IP (2B)
PAD (nB)
CRC (4B)
IP Payload
Dst Addr (4B)
Src Addr (4B)
Ver/HLen/Tos/Len (4B)
ID/Flags/FragOff (4B)
TTL (1B)
Protocol (1B)
Hdr Cksum (2B)
Type=802.1Q (2B)
TCI ≠ VLAN0 (2B)
DstAddr (6B)
SrcAddr (6B)
Legacy
SrcAddr (6B)
Type=802.1Q (2B)
TCI ≠ VLAN0 (2B)
Type=802.1Q (2B)
MLI (2B)
LEN (2B)
Meta Frame
TCI (2B)
Type=Substrate (2B)
PAD (nB)
CRC (4B)
DstAddr (6B)
SrcAddr (6B)
Type=IP (2B)
Type=802.1Q (2B)
MLI (2B)
LEN (2B)
Meta Frame
TCI=VLAN0 (2B)
Type=Substrate (2B)
PAD (nB)
CRC (4B)
DstAddr (6B)
SrcAddr (6B)
P2P-VLAN0
Multi-Access
Type=802.1Q (2B)
MLI (2B)
LEN (2B)
Meta Frame
TCI≠VLAN0 (2B)
Type=Substrate (2B)
PAD (nB)
CRC (4B)
DstAddr (6B)
SrcAddr (6B)
Ver/HLen/Tos/Len (4B)
ID/Flags/FragOff (4B)
TTL (1B)
Protocol=Substrate (1B)
Hdr Cksum (2B)
Src Addr (4B)
Dst Addr (4B)
MLI (2B)
LEN (2B)
Meta Frame
PAD (nB)
CRC (4B)
P2P-DC
(Configured)
P2P-Tunnel
Blue Shading: Determine SL Type
Black Outline: Key Fields from pkt

4. External Frame Formats: w/o VLAN
DstAddr (6B)
DstAddr (6B)
SrcAddr (6B)
SrcAddr (6B)
Type=IP (2B)
Type=IP (2B)
Ver/HLen/Tos/Len (4B)
Ver/HLen/Tos/Len (4B)
DstAddr (6B)
DstAddr (6B)
ID/Flags/FragOff (4B)
ID/Flags/FragOff (4B)
TTL (1B)
SrcAddr (6B)
TTL (1B)
Protocol=Substrate (1B)
SrcAddr (6B)
Protocol (1B)
Hdr Cksum (2B)
Type=Substrate (2B)
Hdr Cksum (2B)
Src Addr (4B)
Type=Substrate (2B)
MLI (2B)
Src Addr (4B)
MLI (2B)
LEN (2B)
Dst Addr (4B)
Dst Addr (4B)
LEN (2B)
Meta Frame
MLI (2B)
IP Payload
Meta Frame
LEN (2B)
Meta Frame
PAD (nB)
PAD (nB)
PAD (nB)
PAD (nB)
CRC (4B)
CRC (4B)
CRC (4B)
CRC (4B)
P2P-DC
(Configured)
P2P-Tunnel
Legacy
Multi-Access
Blue Shading: Determine SL Type
Black Outline: Key Fields from pkt

5. External Frame Formats: GRE
We are considering GRE tunnels as another SL Type
DstAddr (6B)
SrcAddr (6B)
DstAddr (6B)
Type=802.1Q (2B)
SrcAddr (6B)
TCI ≠ VLAN0 (2B)
Type=IP (2B)
Type=IP (2B)
Ver/HLen/Tos/Len (4B)
Ver/HLen/Tos/Len (4B)
ID/Flags/FragOff (4B)
ID/Flags/FragOff (4B)
TTL (1B)
TTL (1B)
Protocol=GRE (1B)
Protocol=GRE (1B)
Hdr Cksum (2B)
Hdr Cksum (2B)
Src Addr (4B) `
Src Addr (4B)
Dst Addr (4B)
Dst Addr (4B)
Flags/recur/Ver (2B)
Flags/recur/Ver (2B) `
Type=Substrate (2B)
Type=Substrate (2B) `
Optional Fields (nB)
Optional Fields (nB)
MLI (2B) `
MLI (2B)
LEN (2B)
LEN (2B)
Meta Frame
Meta Frame
MLI (2B)
PAD (nB)
PAD (nB)
CRC (4B)
CRC (4B)
Blue Shading: Determine SL Type
Black Outline: Key Fields from pkt
VLAN GRE
GRE

6. Internal Frame Format DAddr (6B) SAddr (6B) Ethernet Header 18 Bytes
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
Ethernet Header
18 Bytes
Substrate Header
Metanet Frame
(Metanet Frame Len Bytes)
Type (4b)
Metanet Frame Len (12b)
Ethernet Trailer (Pad+CRC)
Destination MPE/MI (2B)
6-70 Bytes
Source MPE/MI (2B)
Type Dependent Data (0-64B)
Type:Defines what type of frame we have
See next slides for details on types
MPE: Meta Processing Element
MI: Meta Interface
MetaNet Frame Length: Length of Metanet Frame in Bytes

7. Substrate Header Ingress: LC  MPE Transit: LC  LC
Destination MPE (2B)
RxMI (2B)
Metanet Frame Len (12b)
0000
Ingress: LC  MPE
Transit: LC  LC
TxMI and RxMI are defined by the Substrate on the LC
They are not actual MIs on a MR
TxMI (2B)
RxMI (2B)
Metanet Frame Len (12b)
0001
Destination MPE (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0010
Internal: MPE  MPE
MNs can define their on MN Internal header to be put in the MetaNet Frame
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0100
Egress Simple: MPE  LC
No extra information needed

8. Substrate Header (continued)
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0101
Length (1B)
NH MN Addr (0-63B)
Egress NH_MN_Addr: MPE  LC
MPE provides a next hop MN address for Substrate to perform ARP for it.
(Length + Nh MN Addr) field padded to be even number of bytes long
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0110
MAC Address (6B)
Egress MAC_ADDR: MPE  LC
MPE provides MAC address for Substrate to use when sending out on Egress link

9. Type Dependent Data (0-???B)
IPv4 MR Internal Header
Length (1B)
Exception Type (1B)
MPE to CP
Slow path handling of exception packets
Some exceptions may require additional data to be
communicated to CP
RxMI (2B)
Type Dependent Data (0-???B)
CP to MPE !Reclassify
Return packets to MPE for re-injection into fast path
Also used by CP to inject new packets into data stream.
For example, ICMP
MR does not need to reclassify
Substrate would need a lookup on TxMI to get the information needed to get packet to LC.
Length = 4 Bytes
Type = 0x0
No Type Dependent Data
TxMI (2B)
Length (1B)
Type (1B)
Type Dependent Data (0-???B)
RxMI (2B)
Length (1B)
Type (1B)
Type Dependent Data (0-???B)
CP to MPE Reclassify
Return packets to MPE for re-injection into fast path
Ingress Lookup: Redo the normal lookup with provide RxMI in Key.
Length = 4 Bytes
Type = 0x1
No Type Dependent Data

10. IPv4 MR Internal Header Notes
Input data to Parse is MN Frame Offset and Length
Output data from Parse is IP Pkt Offset and Length
If there is a MN Internal Header, Parse basically consumes it.
Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer.
If Parse receives a frame/pkt from the CP,
It should not decrement the TTL.
Assume a pkt coming from the CP has the correct TTL for going out on the link
It should validate the header and check the header checksum
It should not generate an exception for IP options.
Assume the CP has dealt with all the exceptions.
What about the case where the TxMI is on a Multi-Access Link and needs a Next Hop MetaNet Address to resolve the next hop destination?
For Frame/Pkt from CP, what does Demux put in RxMI data field passed to Parse?

11. LC  MPE (Ingress) Internal Format
DAddr (6B)
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
Destination MPE (2B)
RxMI (2B)
Metanet Frame Len (12b)
0000
Metanet Frame
(Metanet Frame Len Bytes)
MPE
Ethernet Trailer (Pad+CRC)
Destination MPE: ID of receiving MPE
RxMI: Receive Meta Interface

12. LC  LC (Transit) Internal Format
DAddr (6B) LC
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
TxMI (2B)
RxMI (2B)
Metanet Frame Len (12b)
0001
Metanet Frame
(Metanet Frame Len Bytes) LC
Ethernet Trailer (Pad+CRC)
TxMI: Substrate allocated MI for translation to Egress MLI
RxMI: Substrate allocated MI
Probably not really needed, present for symmetry

13. MPE  MPE Internal Format
DAddr (6B)
MPE
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
Destination MPE (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0010
Metanet Frame
(Metanet Frame Len Bytes)
MPE
Ethernet Trailer (Pad+CRC)
Destination MPE: ID of receiving MPE
Source MPE: ID of sending MPE
MR specific information, things like exception packet indicators, etc. go in a MetaNet specific header in the MetaNet Frame.

14. MPE  LC (Egress Simple) Internal Format
DAddr (6B)
MPE
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0100
Metanet Frame
(Metanet Frame Len Bytes) LC
Ethernet Trailer (Pad+CRC)
Source MPE: ID of sending MPE
TxMI: Transmit Meta Interface

15. MPE  LC (Egress NH MN) Internal Format
DAddr (6B)
MPE
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0101
Length (1B)
NH MN Addr (0-63B) LC
Metanet Frame
(Metanet Frame Len Bytes)
Ethernet Trailer (Pad+CRC)
Source MPE: ID of sending MPE
TxMI: Transmit Meta Interface
NH MN Addr: Padded to so that Length + NH MN Addr is an even number of Bytes

16. MPE  LC (Egress MAC) Internal Format
DAddr (6B)
MPE
SAddr (6B)
Type=802.1Q (2B)
VLAN (2B)
Type=Substrate (2B)
TxMI (2B)
Source MPE (2B)
Metanet Frame Len (12b)
0110
MAC Address (6B) LC
Metanet Frame
(Metanet Frame Len Bytes)
Ethernet Trailer (Pad+CRC)
Source MPE: ID of sending MPE
TxMI: Transmit Meta Interface
MAC: We are assuming Ethernet addresses here. Should we?

17. Data Delivered by Substrate to MPE
Metanet Frame Len (16b)
Destination MPE (2B)
RxMI (2B)
Metanet Frame
(Metanet Frame Len Bytes)
In a shared NPU case:
MR will not receive the Ethernet header or trailer
Type field will be zeroed out and look like it is part of the Frame Length.
Destination MPE should be id of MR MPE receiving packet

18. Extra
The next set of slides are for templates or extra information if needed

19. Text Slide Template

20. Image Slide Template

21. IPv4 MR Internal Header Notes
Input data to Parse is MN Frame Offset and Length
Output data from Parse is IP Pkt Offset and Length
If there is a MN Internal Header, Parse basically consumes it.
Hdr Format needs to start at the beginning of the IP Header and re-write headers upward in the Buffer.
If Parse receives a !Reclassify frame/pkt from the CP,
it should not decrement the TTL.
Assume this was done on the first trip through the MR.
it should not generate exceptions again.
For Frame/Pkt from CP, what does Demux put in RxMI data field passed to Parse?