ICmyNet.Flow Network Traffic Analysis System If You Want to See Your Net

2 What is IP flow? IP flow is a unidirectional series of IP packets of a given protocol traveling between a source and destination IP address/port pair within a certain period of time IP flow parameters: Src & Dst IP address Src & Dst TCP/UDP port Protocol ToS field

3 What is IP flow accounting? IP flow accounting is a collection of statistical data for every single IP flow crossing a network device: Number of packets Number of bytes Timestamps

4 What is NetFlow? NetFlow is a network protocol developed by Cisco Systems for export of collected IP flow statistics

5 NetFlow Statistics Collection

6

7 ICmyNet.Flow system architecture Binary raw data files Flows_ Flows_ Flows_ ICmyNet.Flow Collector ICmyNet.Flow Aggregator Database ICmyNet.Flow Web Raw Data Files Archive

8 ICmyNet.Flow/Collector ICmyNet.Flow/Collector is a part of the system that collects flow records exported over Netflow protocol. Exported flow records have statistics about every data flow transported over network device: Src & Dst IP address Src & Dst TCP/UDP port Protocol ToS field In & Out Interfaces of the network device Statistics information contains timestamps and number of packets and bytes carried over the data flow Supported NetFlow protocol versions: Version 5 (supported on most of the network devices) Version 9 (flexible format with support for IPv6, MPLS, Multicast and MAC addresses) System can be easily extended to support different vendor protocols: J-Flow – Juniper protocol for statistics export NetStream – Huawei protocol for statistics export IPFIX – currently standardized protocol based on NetFlow v9

9 ICmyNet.Flow/Aggregator ICmyNet.Flow/Aggregator is performing analysis and aggregation over collected raw NetFlow records. This analysis is done according to the user configuration of the “Traffic Patterns” which is the basic element of the analysis Analyzed information is stored in the database and it is used for further search and view from the user interface System supports fast PostgreSQL database The level of aggregated data can be configured according to the user needs and the available server capabilities Different grains for keeping the data. For example: High grain – 5 min aggregation sample, 7 days keeping Medium grain – 60 min aggregation sample, 30 days keeping Low grain – 360 min aggregation sample, 356 days keeping

10 Traffic Pattern – basic element of analysis Local Network External Network The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed. “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

11 Traffic Pattern – basic element of analysis Local Network /8 Application Servers /24 The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed. “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

12 Traffic Pattern – basic element of analysis Local Network /8 Internet Exclude /8 The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed. “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

13 Traffic Pattern – basic element of analysis Local Network /8 Internet /0 The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed. “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

14 Traffic Pattern – basic element of analysis The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed. “Local” and “External” networks can be defined as IP address ranges or as single host IP address. Local Network /8

/ / /16 Traffic analysis based on Subnets Local Network /8 External Network IP address space is usually divided in hierarchical manner to represent a logical or sometimes physical topology of the network. Example: Universities have /16 address range Campuses have /21 address range Faculties have /24 address range

/ / /16 Traffic analysis for Hosts Local Network /8 External Network Within the scope of the Subnet, system is accounting network traffic of single hosts. Cut-off value can be configured for minimum traffic Universities have /16 address range Campuses have /21 address range Faculties have /24 address range

17 Parameters for traffic analysis Traffic analysis gives a detail information about following parameters at the Traffic Pattern level: IP subnets traffic Hosts traffic Network Services and applications based on TCP/UDP ports Network Protocols (TCP, UDP, ICMP, GRE...) QoS markers (ToS, IP precedence or DSCP) Autonomous System Numbers For every parameter of analysis there are following counters: Traffic Bandwidth (in bits/s, kbps, Mbps..) Traffic Volume (in MBytes, GB, TB...) Number of Packets, volume and time based diagrams (pps) Number of Flows, volume and time based diagrams (fps) Configurable cut-off percentage or data amount for negligible consumers

18 ICmyNet.Flow/Web Web application is chosen for the user interface De-facto standard for network management applications Accessibility, permanent development, flexibility Java application working under Tomcat JSF technologies

19 Settings Tab – Traffic Patterns Configuration of the NetFlow analysis is done from the Settings Tab User can configure following elements of analysis: Traffic Patterns Subnets Subnet Sets Services Protocols QoS markers AS Numbers Exporters Control Panel General Users Update My Account

20 Settings Tab – Traffic Patterns Advanced Traffic Patterns can be configured with flexible matching of any supported NetFlow field Examples: Local Network -> Facebook Local address /16, Src or Dst AS (Facebook) Router X Local & External address: /0, Exporter Potential attacks: Src or Dst port: 22, , 445, 1434,… “Weird” Protocols: Protocols: Exclude 6 (TCP) or 17 (UDP) Blocked Traffic: Out Interface: 0 (Null)

21

22 Subnets Each Subnet is defined with its Name and IP address range View tab, Address Space button: below Traffic Pattern element gives an IP address hierarchy in a tree structure

23 Subnet Set Subnet Set is user defined grouping of Subnets and other Subnets Sets. View tab, Custom Space button: below Traffic Pattern element gives user defined hierarchy of Subnet Sets and belonging Subnets Subnet Set can be any logical grouping of Subnets: Customer Institution Faculty University

24

25

26

27

28

29

30

31 Viewing the analyzed NetFlow data ICmyNet.Flow system tends to give a user the best insight into the network traffic structure Therefore, every parameter of the network traffic analysis is presented to the user in various useful ways: Top – Visual representation of the distribution of the “Top N Talkers” in the form of the pie chart. Gives a data for the network traffic volume. Chart – Time based diagram with a Top N consuming parameters presented in different colors. List – Tabular form for reviewing of all parameters and data with advanced options for sorting according to different criteria. For every view user can select arbitrary time scale for convenient view Number of Top Talkers is user configurable parameter

32 View Tab – Top N

33 View Tab – Chart

34 View Tab – List

35 Archived raw data review Raw NetFlow records collected from network devices are archived in the files created every 5 minutes. When Collector closes a current file and Aggregator finish with analysis, file is compressed and archived in separate folder. Every single flow is saved in these files and no data is wasted User can access, review and explore these files, searching for a single flow or event that traversed the network. Review of the raw data is done over User Interface and search is available for every supported NetFlow field.

36 Archived raw data review

37

38 Searching and grouping raw data

39 Whois and DNS functions

40 Monitoring system performance At the View mode System Tab user can access to relevant graphs monitoring system performance Processed flows - number of flows in a single raw data file (created on 5 minutes) Matched flows – number of flows that match criteria of any Traffic Pattern

41 Monitoring system performance At the View mode System Tab user can access to relevant graphs monitoring system performance Processing time for a single raw data file (created every 5 minutes) Required time to store aggregated data into database Required time for aggregation between grains and deleting data

ICmyNet.Flow Network Traffic Analysis System If You Want to See Your Net