1. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (1) 5. Passive Monitoring Techniques

2. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (2) 5. Passive Monitoring - Packet Capturing  Packets can be captured using Port Mirroring or Network Splitter (Tap) Mirroring Probe system Splitting Probe system Port MirroringNetwork Splitter How it works- Copies all packets passing on a port to another port - Splits the signal and send a signal to original path and another to probe Advantage- No extra hardware required - No processing overhead on router/switch Disadvantage- Processing overhead on router/switch - Splitter hardware required

3. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (3) 5. Passive Monitoring - Packet Capturing  Difficulties in packet capturing  Massive amount of data  How much packet data is generated from 100 Mbps network in an hour?  Port speed ⅹ In&Out ⅹ Link Utilization ⅹ sec/hour = throughput 100 Mbps ⅹ 2 ⅹ 0.5 ⅹ 3600 = 360 Gbps  Throughput / avg. packet length ⅹ bytes of packet data = data size 360 Gbps / (1500 ⅹ 8) ⅹ 30 = 1 Gbyte  Processing of high-speed packets  Processing time for 100 Mbps network  Port speed ⅹ In&Out ⅹ Link Utilization / average packet length = 8333 packets/sec => 0.12 msec/packet 100 Mbps1 Gbps1 Tbps Data size per hour (assume 0.5 link util)1 Gbyte10 Gbyte10 Tbyte Processing Time per packet0.12 msec0.012 msec0.012 μsec

4. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (4) 5. Passive Monitoring - Sampling  If the rate is too high to capture all packets reliably, there is no alternative but to sample the packets  Sampling algorithms: every Nth packet or fixed time interval 1 2 3 4 5 6 7 8 9 10 11 (a) 2:1 sampling (b) 1 msec sampling 0 msec1 msec2 msec3 msec4 msec

5. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (5) 5. Passive Monitoring - Flow Generation  Flow is a collection of packets with the same {SRC and DST IP address, SRC and DST port number, protocol number, TOS}  Flow data can be collected from routers directly, or standalone flow generator having packet capturing capability  Popular flow formats  NetFlow (Cisco), sFlow (sFlow.org), IPFIX (IETF)  Issues in flow generation  What information should be included in a flow data?  How to generate flow data from raw packet information efficiently?  How to save bulk flow data into DB or binary file in a collector?  How long should the data be preserved? flow 4flow 1flow 2flow 3

6. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (6) 5. Passive Monitoring - Flow Technology: NetFlow  Cisco NetFlow  is an option configurable in Cisco routers that exports data on each IP flow passed through an interface  Cisco IOS NetFlow technology  is an integral part of Cisco IOS software that collects and measures data as it enters specific routers or switch interfaces  enables to perform IP traffic flow analysis without custom probes  3 key components in a NetFlow system  Flow Exporter  Flow Collector  Network Data Analyzer (Flow Analyzer)

7. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (7) 5. Passive Monitoring - Flow Technology: NetFlow  NetFlow Export Datagram  Version 1, Version 5, Version 7, Version 8  Version 1: original format supported in the initial Cisco IOS software releases.  Version 5: Source IP Address Destination IP Address Source IP Address Destination IP Address Next Hop Address Source AS Number Dest. AS Number Source Prefix Mask Dest. Prefix Mask Next Hop Address Source AS Number Dest. AS Number Source Prefix Mask Dest. Prefix Mask Input Interface Port Output Interface Port Input Interface Port Output Interface Port Type of Service TCP Flags Protocol Type of Service TCP Flags Protocol Packet Count Byte Count Packet Count Byte Count Start Timestamp End Timestamp Start Timestamp End Timestamp Source TCP/UDP Port Destination TCP/UDP Port Source TCP/UDP Port Destination TCP/UDP Port Usage QoS Time of Day Application Routing and Peering Port Utilization From/To Header · Sequence number · Record count · Version number Flow Record Flow Record Flow Record Flow Record Flow Record

8. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (8) 5. Passive Monitoring - Flow Technology: NetFlow  Version 7  Enhancement that supports Cisco Catalyst 5000 Series switches equipped with NetFlow Feature Card (NFFC).  Version 8  developed mainly to MINIMIZE output size from exporter by adding Router-Based Aggregation schemes type UDP datagram records/datagram max udp pktsize ASMatrix 51 1456 ProtocolPortMatrix 51 1456 SourcePrefixMatrix 44 1436 DestPrefixMatrix 44 1436 PrefixMatrix 35 1428  available on Cisco routers from IOS release 12.0(3)T

9. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (9) 5. Passive Monitoring - Flow Technology: sFlow  sFlow is described in RFC 3176: “InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks”  sFlow is a monitoring technology that gives visibility into the use of networks, enabling performance optimization, accounting/billing for usage, and defense against security threats  sFlow provides an effective means of embedding traffic monitoring in high-speed switches and routers  sFlow samples packets using statistical sampling theory

10. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (10) 5. Passive Monitoring - Flow Technology: sFlow  sFlow Datagram Format  is specified using the XDR standard  XDR is a standard for the description and encoding of data. (eXternal Data Representation Standard, RFC1014)  version 4  Packet Header Data  Header Protocol (Format of sampled header)  Frame_length  Header bytes  Packet IP v4 Data  Length  Protocol (IP Protocol Type)  src_ip / dst_ip  src_port / dst_port  TCP flags  tos  Packet IP v6 Data  Length  IP next Header  src_ip / dst_ip  src_port / dst_port  TCP flags  IP priority

11. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (11) 5. Passive Monitoring - Flow Technology: sFlow  Equipment Supporting sFlow  Foundry Networks  BigIron, FastIron, NetIron Series  InMon’s sFlow Probe  By attaching to a monitor/SPAN port  Gathers mirrored or tapped (using a splitter) traffic data  The resulting data is forwarded in sFlow datagrams to a central sFlow collector (for example InMon Traffic Server) for analysis. Source: InMon

12. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (12) 5. Passive Monitoring - Flow Technology: IPFIX  IPFIX (IP Flow Information eXport) Working Group  http://www.ietf.org/html.charters/ipfix-charter.html  Background  There are a number of IP flow export systems in common use  These systems differ significantly, even though some have adopted a common transport mechanism  such differences make it difficult to develop generalized flow analysis tools  Goal  To produce a standard method for exporting flow info from network devices, as an eventual replacement for the various proprietary methods in use now

13. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (13) 5. Passive Monitoring - Flow Technology: IPFIX  IPFIX Internet Drafts  Requirements for IP Flow Information Export  J. Quittek et al., Jan 2003 (work in progress)  Architecture Model for IP Flow Information Export  K.C. Norseth, G. Sadasivan, June 2002 (work in progress)  Early stage of work….

14. POSTECH DP&NM Lab. Internet Traffic Monitoring and Analysis: Methods and Applications (14) 5. Passive Monitoring - Traffic Analysis  Spatial aspect  The patterns of traffic flow relative to the network topology  Important for proper network design and planning  Identification of bottleneck & avoidance of congestion  Example: Flow aggregation by src, dst IP address or AS number  Temporal aspect  The stochastic behavior of a traffic flow, usually described in statistical terms  Important for resource management and traffic control  Important for traffic shaping and caching policies  Example: Packet or byte per hour, day, week, month  Composition of traffic  A breakdown of traffic according to the contents, application, packet length, flow duration  Helps to explain its temporal and spatial characteristics  Example: game, streaming media traffic for a week from peer ISP