1. Addition of Virtual Interfaces in NetFlow Probe for the NetFPGA Muhammad Shahbaz Zaheer Ahmed Habibullah Jamal Asrar Ashraf Nadeem Yousaf Raania Naeem Khan

2. Presentation Organization
NetFlow Overview
Virtual Interfaces in NetFlow
Hardware Architecture of NetFlow probe
Software Architecture of NetFlow probe
Sample Netflow Record
Extended Applications
Conclusion
Demonstration Setup
Questions / Answers

3. NetFlow Overview
Network Protocol developed by Cisco for Collecting IP traffic information
Cisco proprietary but supported by other platforms like Juniper, Linux etc.
Netflow enabled routers/probes generate netflow records
Exported via UDP or SCTP to data-collectors
Netflow record identified traditionally by 7-Tuple keys formed by combining
Source IP
Destination IP
Source port for UDP or TCP and 0 for other protocols
Destination port for UDP or TCP and 0 for other protocols
IP protocol
Ingress interface
IP Type of Service(TOS)
Netflow Records contain extensive information regarding traffic flow including Version, Sequence number, ingress interface, timestamp and other data statistics of particular data flow.
A netflow is a network analysis protocol designed by the Cisco systems for collecting IP traffic information. Though initially developed by Cisico, netflow is now supported by various platforms like Juniper and Linux etc. Netflow enabled routers and probes generate netflow records that are exported to different local and remote data-collectors over UDP or SCTP protocol. A netflow record is normally identified as a 7-tuple key formed by a combination of …. . These netflow records contain extensive information regarding the IP traffic flowing through the network. Information includes IP version, TCP packet sequence number, input interface, timestamp and various other information regarding the data flow. Currently the implemented design only provides support for IPv4 packets.

4. NetFlow Overview (contd. ) Net Flow Applications
Network Planning
RMON Probe
RMON Application
Accounting/Billing
NetFlow FlowCollector:
Data Collection
Data Filtering
Data Aggregation
Data Storage
NetFlow Data Export:
Data Switching
Data Aggregation
Data Export
Network Data Analyzer:
Data Presentation
Flow Control and Configuration
Partner Applications
Image From NetFlow PPT by Michael Lin, Cisco Systems

5. NetFlow Overview (contd.) Standalone NetFlow Architecture
In order to avoid the limitations laid down by the router-based approach for netflow analysis a new approach using dedicated probes is becoming very popular. The implementation of NetFlow on NetFPGA is an example of such probe. In the standalone netflow architecture these probes are connected to the monitored links as a totally passive entity using TAPs. These probes export netflow information over a dedicated link established between the collectors and the probes themselves, making them completely invisible to the monitored links.
Image From Wikipedia, NetFlow

6. Virtual Interfaces in NetFlow
Virtual interfaces are usually found in technologies like
Layer 2 Tunneling Protocol (L2TP)
Generic Routing Encapsulation (GRE) tunnels
Multiprotocol Label Switching over Virtual Private Network (MPLS-VPN)
Collect network flow information from L2TP, GRE and MPLS enabled networks
Virtual interfaces are usually found in technologies like L2TP, GRE, and MPLS-VPN. … . … . Our purpose is to demonstrate the scalability and flexibility of the implemented architecture to support any new protocol in the existing netflow design for flow analysis.

7. Hardware Architecture of NetFlow Probe
Here is the hardware architecture of our NetFlow probe design. The IO Queues for Ethernet and PCI, the input arbiter and Output Queues are taken from the netFPGA platform. We have used the user data path of the original netFlow probe design as reference. The major modification is the addition of multi-layer protocol extraction block in the output port lookup. The record wrapper module in refer has been replaced with the OPL processor. Another modification is the placement of Timestamp module outside the output port lookup, just after the input arbiter. This is done to make the timestamp block independent of the main pipeline i.e. output port lookup.

8. Multi-Layer Protocol Extraction Block
Composed of following protocols:
original L3/L4 block provided with the reference NetFlow design
MPLS block for the extraction of multi protocol label switched packets with support for only two labels,
GRE block for the parsing of GRE encapsulated protocol packets
L2TP block for mining layer 2 tunneled PPP packets
Future protocols.
Architecture of Multi-Layer Protocol Extraction consists of
Packet Monitor that tracks the state of the packet during the extraction process
a configurable stack of protocol combinations
a Multi-stage Priority Multiplexer.
The multi-layer protocol extraction block is the heart of our design. The extraction of exact and accurate flow records from different combination of layers is a crucial task. Currently the protocols supported by the extraction block are: (1) the L3L4 protocol (2) multi-protocol label switching MPLS protocol with support for only two labels (3) Generic Routing Encapsulation protocol and (4) layer 2 tunneled PPP packet L2TP protocol.
The architecture of the Multi-layer protocol extraction block consists of (1) a Packet monitor that tracks the state of the packet during the extraction process (2) a configurable stack of protocol combinations and (3) a multi-stage priority MUX.

9. Multi-Layer Protocol Extraction Block (contd.)
Multi-Stage Protocol Extraction Pipeline
Packet Monitor broadcasts packet words to all components with a latency of 2 cycles
Header Information extracted after n+2 cycles delay
n is either total number of words taken by the protocol combination with largest header or size of incoming packet which ever is smaller.
Total latency for the example shown below is be n+7 cycles
Here you can see the complete pipeline of the protocol extraction block. The Packet Monitor broadcasts the received packets to all the protocol combination blocks with a latency of 2 cycles. These blocks extract the flow information from the packet header in n+2 cycles. Here n is equal to the total number of words taken by the largest protocol combination i.e. supported packet with largest header however, if the incoming packet is smaller than the size of the largest protocol combination then n is equal to the size of the incoming packet. The total latency observed for the configuration shown in the diagram is n+7 cycles, including the 3 cycles latency for the Multiplexer.

10. Multi-Layer Protocol Extraction Block (contd.)
MPLS Decoding and Extraction
Multiple Protocol Label Switching (MPLS) tunnels are detected based on lower layer protocol type field as 0x8847
Detection of Upper Layer Protocol is not defined in MPLS Standard Documents
Upper Layer Protocols are detected based on byte pattern detection and verification.
Currently IP Protocol Detection is supported as MPLS upper layer protocol.
Flow for IP Detection
Check for final MPLS header from ‘Bottom of Label Stack’ field
Check top nibble of first byte after MPLS header (0x4 for IPv4 and 0x6 for IPv6).
Check Lower Nibble as Header Length
Treat it as IPv4 or IPv6 Packet and verify Length of remaining packet from expected Total Length field of IP header
If verified, Upper Layer Protocol is IPv4 or IPv6
Else It is treated as Ethernet packet
Support for Any other Protocol above MPLS can be very easily added due to the scalable architecture of the design.
MPLS tunnels are detected on the basis of lower layer protocol type field with value 0x8847. The detection of upper layer protocol is not defined in the MPLS standard documents. So in order to find out the type of protocol we have adopted a byte pattern and verification model. Currently the system supports only IP protocol detection as MPLS upper layer protocol.

11. Multi-Layer Protocol Extraction Block (contd.)
MPLS with IP as upper layer protocol
MPLS Detection
Bottom of Label Stack as 1
Values extracted from 1st Byte
IPv4, Header length=20
Verification from Total length field confirms whether IP packet or not.
Here is an example of MPLS packet with IP as an upper layer protocol. From the Ethernet type field we see that the next protocol in the stack is MPLS. Next we look for the MPLS bottom of the label stack with value 1, indicating that it’s the last MPLS header. The next nibble just after the last MPLS header is checked for IP version (4 for IPv4). If result is true, as in our case it is, we calculate the sum of all the header lengths and the total length from the IP header field and compare it with the byte count calculated by the receive queue at the reception of the packet. If its verified the upper layer is marked as IP.

12. Multi-Layer Protocol Extraction Block (contd.)
GRE Decoding and Extraction
Generic Routing Encapsulation (GRE) tunnels detected based on lower layer protocol type field as 0x2f
Sample GRE packet
GRE Detection
GRE Upper Layer protocol
The decoding of GRE packets is very straight forward. The GRE tunnels are detected on the basis of lower layer protocol type field. The upper layer protocol is detected using the protocol type field in the GRE header.

13. Multi-Layer Protocol Extraction Block (contd.)
L2TP Decoding and Extraction
L2TP Decoding Performed only on UDP Port Number 1701
Sample L2TP Packet
L2TP Detection
L2TP tunneled packets are always encapsulated in UDP. The L2TP decoding is performed only on the basis of UDP port number as Normally L2TP carries Point to point protocol as its upper layer protocol. The presence of the IP layer is then detected by the protocol type field of the PPP header.

14. Software Architecture of NetFlow probe
The host software provided with the original netflow design has been upgraded to provide support for virtual interfaces (GRE, L2TP, and MPLS). The other basic functionalities like configuration, statistical interfaces and information collection remain the same.

15. Resource Utilization
Only about 3% extra resources were used to incorporate the support for GRE, MPLS and L2TP Protocols
Table 1: Resource utilization of the Current Architecture with Virtual Interfaces (MPLS, L2TP, and GRE) plus l3l4 protocol
Table 2: Resource utilization of the Original NetFlow probe Architecture with only l3l4 protocol
Resources
XC2VP50
Utilization
Utilization Percentage
Slices
out of 23616
77%
4 - Input LUTS
25165 out of 47232
53%
Flip Flops
21244 out of 47232
44%
Block RAMs
200 out of 232
86%
Resources
XC2VP50
Utilization
Utilization Percentage
Slices
out of 23616
74%
4 - Input LUTS
23319 out of 47232
49%
Flip Flops
19504 out of 47232
41%
Block RAMs
200 out of 232
86%
The resource utilization of the original NetFlow probe architecture and the current architecture is shown in Table 1 and Table 2 respectively. Even though three different technologies of virtual interfaces (GRE, L2TP and MPLS) have been incorporated in the current design, yet there is only an average increase of 3% in the overall utilization of the resources.

16. Sample Netflow (Cflow) Record
NetFlow (CFlow) Packets
Using UDP as Export Transport
Top Header of NetFlow Record
This is a sample output generated from our system. As Shown in Packet, UDP is used for tranporting netflow records. Main Netflow Header contains basic fields like version, count, timestamp and other standard fields. At the end, there is pdu containing statistics regarding a particular flow.
Data PDU NetFlow Record

17. Extended Applications
Deep Packet Inspection (DPI) based VoIP Monitoring
Telecom Regulatory Authorities Perspective
VoIP Header and RTP Monitoring for illegalV oIP Identification and Mitigation
Scalability Tested for upto 40G Data Rates using HighTech Global Cards
Flexible Protocol addition

18. Conclusion
Presented a generic protocol extraction layer for Netflow probe architecture
Primary focus on extraction mechanism for technologies supporting virtual interfaces i.e. MPLS, L2TP and GRE
The architecture finds applications in
Deep Packet Inspection (DPI)
Voice over IP (VoIP) monitoring
Accounting /Billing
In this paper we presented an architecture of a generic protocol extraction layer for NetFlow Probe with primary focus on the extraction of technologies supporting virtual interfaces like MPLS, L2TP and GRE. The architecture finds applications in Deep packet inspection, Voice over IP monitoring and other applications like accounting and billing.

19. System Demonstration Setup for Multi-Gigabit networks
NetFPGA
(Traffic Generator)
NetFPGA
(Netflow Probe)
Remote Collector
This is the demonstration setup for our netFPGA netFlow probe with virtual interfaces. We have a netfgpa card mounted on a linux system acting as a standalone netflow probe. The remote collector is connected to the probe through an ethernet interface.
We have used another netfpga card for traffic generation in order to simulate traffic for L2TP, GRE and MPLS. The traffic generator communicates with the probe using an ethernet interface.

20. System Demonstration Setup for Multi-10Gigabit networks
Avnet PCIe Card
(10G Traffic Generator)
Remote Collector
I would like to mention here that at CASE, we have also implemented the similar architecture of netFlow probe using a hitech global PCIE card with 4 10g interfaces. The configuration of the board is similar to the netFPGA excpet that now we have 10g interfaces instead of 1G and a PCIe interface instead of PCI for host connectivity.
For simulating network traffic for MPLS, L2TP and GRE we have used an Avnet PCIe card as a 10G traffic generator.
Hitech Pcie 40G Card
(Netflow Probe)

21. Questions / Answers