Presentation is loading. Please wait.

Presentation is loading. Please wait.

Usefulness of the results - a forgotten evaluation metric of traffic identification tools Tomasz Bujlow Aalborg University.

Similar presentations


Presentation on theme: "Usefulness of the results - a forgotten evaluation metric of traffic identification tools Tomasz Bujlow Aalborg University."— Presentation transcript:

1 Usefulness of the results - a forgotten evaluation metric of traffic identification tools Tomasz Bujlow Aalborg University

2 Agenda Few words about myself Motivations for traffic monitoring Existing methods and tools for traffic monitoring & classification - and why they are far from being excellent A deep look into Deep Packet Inspection How to verify the accuracy of classification tools? Implementation and various applications of VBS

3 Tomasz Bujlow University of Southern Denmark (2007 – 2009) Bachelor of Computer Engineering, Computer Engineering The Silesian University of Technology (2003 – 2008) Master of Science in Engineering, Computer Engineering Aalborg University (2010 – 2014) Doctor of Philosophy (PhD): Classification and analysis of computer network traffic Universitat Politecnica de Catalunya (January 2013 – April 2013) Visiting PhD Student, CBA - Broadband Communications Research Group Cisco Certified Network Professional (2010)

4 Part I Motivations for traffic monitoring

5 Why to perform traffic monitoring? To obtain basic statistical information about different kinds of flows in the network and improve Quality of Service Our interests ● content (audio, video) ● application (P2P, FTP) ● service (YouTube, Facebook)

6 Why to perform traffic monitoring? To obtain the knowledge which applications are most frequently used in the network and enhance user experience by tuning some network parameters or setting up dedicated proxies or servers Our interests ● application (Skype, HTTP)

7 Why to perform traffic monitoring? To compare users located in the same network and group them into profiled sections Our interests ● application (Skype, BitTorrent) ● IP protocol (TCP / UDP)

8 Why to perform traffic monitoring? To create graphs of traffic flow between different networks and optimize amounts of bandwidth bought from different content providers Our interests ● service (YouTube, Facebook)

9 Why to perform traffic monitoring? To introduce smart logging of traffic. Logging is now required by law. The ability to recognize types of transmitted content can result in registering of only, for example, text content of websites, but not images or downloaded binary files. That will save resources, especially storage space. Our interests ● content (text, audio, video)

10 Why to perform traffic monitoring? To create a traffic generator, to imitate traffic generated by particular applications, or to imitate the real traffic in the network. That allows to test different solutions before implementing them in the real network, and therefore, to minimize the cost. Our interests ● IP protocol (TCP / UDP) ● application (HTTP, BitTorrent, Skype) ● content (audio, video) ● service (YouTube, Facebook)

11 Why to perform traffic monitoring? To obtain precise data needed to create fast and accurate traffic classifiers working in the network core, which are based on statistical informations (Machine Learning Algorithms). Our interests ● IP protocol (TCP / UDP) ● application (HTTP, BitTorrent, Skype) ● content (audio, video) ● service (YouTube, Facebook)

12 Why to perform traffic monitoring? To implement smart assessment of QoS in the network at the users' level and in the core of the network Our interests ● IP protocol (TCP / UDP) ● application (HTTP, BitTorrent, Skype) ● content (audio, video) ● service (YouTube, Facebook)

13 Why to perform traffic monitoring? To understand the behavior of different applications, services,... Skype Web browsing YouTube World of Warcraft Our interests ● IP protocol (TCP / UDP) ● application (HTTP, BitTorrent, Skype) ● content (audio, video) ● service (YouTube, Facebook)

14 Why to perform traffic monitoring? To detect malicious traffic, such as Botnet traffic Our interests ● application (bot?)

15 Why to perform traffic monitoring? To detect malicious traffic, such as DDoS attacks

16 Part II Existing methods and tools for traffic monitoring & classification - and why they are far from being excellent

17 Traffic classification – overview Classification by ports Deep Packet Inspection (DPI) QoS based (IP precedence, DSCP) Statistical classification

18 Port-based classification Very simple idea, widely used by network administrators to limit unwanted traffic (generated by worms, spam, etc.) Implemented on almost all layer-3 switches existing on the market Can classify only applications operating on fixed ports numbers Very easy to cheat, so unreliable What can we get? ● low application-layer protocol (HTTP, POP3) for some old, well- known cases

19 Deep Packet Inspection (DPI) Rely on inspecting the payload on the application layer Much more convenient to use than previously described methods Requires significant amounts of resources Numerous privacy and confidentiality issues Encryption makes DPI more difficult False positives and false negatives due to implemented statistical methods in DPI tools What can we get? ● Everything we want: IP protocol, application, content, service, etc ● But what kinds of results are really produced by the existing DPI tools?

20 DPI – is it really a consistent mean? Ipoque PACE – application level, content container level [FLASH, WINDOWSMEDIA, QUICKTIME] OpenDPI – an open-source fork of PACE, the same level of consistency nDPI – successor of OpenDPI, additionally: service provider level [FACEBOOK, GOOGLE, TWITTER] Libprotoident – L4 level [TCP / UDP] + the application [BitTorrent], content [Flash_Player], or service provider [YahooError] NBAR – consistent output on the application level L7-filter - consistent output on the application level Today accuracy != consistency ● accurate tools (PACE, OpenDPI, nDPI) – inconsistent ● consistent tools (NBAR, L7-filter) - inaccurate

21 DPI – results by PACE, OpenDPI, nDPI applications and application protocols: BITTORRENT, RDP, SMB, NTP, SSH, DNS, PANDO, NETBIOS, EDONKEY, SOPCAST,DIRECT_DOWNLOAD_LINK, FTP, ICMP, QUICKTIME, MAIL_SMTP, MAIL_IMAP, WINDOWSMEDIA, MAIL_POP, PPSTREAM, STUN, STEAM low-level application protocol: HTTP, SSL content: FLASH, MPEG Undetected traffic: UNKNOWN nDPI adds few services, as Facebook, YouTube, and Google

22 DPI - effects of the consistency aspect Even if the classification results are consistent on the application level, other levels are unknown (IP protocol, lower application protocol, content, service). So, the usefulness of such results is very limited. However, they can be used for the accounting purposes on the application level. Mixing the levels of the results makes the things even worse: a) it is not possible to account the traffic on any level, as always one chosen level is given and the rest is unknown b) as only one level is given, we do not know what is on any other level, so the usefulness of such results in almost NONE! Today accuracy != consistency ● accurate tools (PACE, OpenDPI, nDPI) – inconsistent ● consistent tools (NBAR, L7-filter) - inaccurate

23 DPI - reasons for the lack of consistency Most developers claim that “their tool provides the most detailed result, on whatever level it is” However, how to assess, which level is more precise? Content (MP4 video), content container (Flash), service (YouTube), or application protocol (HTTP)? Given that the obtain result is Flash, what is the real flow association? a) TCP → HTTP → Flash → MP4 video → YouTube (regular file download)? b) TCP → RTMP → Flash → Justin.tv (live TV streaming)? c) TCP → FTP → Flash → EXE (executable file inside Flash container transferred by FTP)? Today accuracy != consistency ● accurate tools (PACE, OpenDPI, nDPI) – inconsistent ● consistent tools (NBAR, L7-filter) - inaccurate

24 DPI – how to generate useful results? Structure the results, so all the relevant classification levels are evaluated: a) IP protocol level (TCP / UDP) b) lower application-level protocol, as HTTP, SSL, POP3, etc c) higher application-level protocol or application, as SMTPS, Skype, BitTorrent, Dropbox d) content, as MP4 video, FLV video, MP3 audio, JPG image e) service, as Facebook, YouTube, or Google Implemented by: a) new version of PACE (partly and in a very limited manner) b) new, development version of nDPI (full implementation)

25 DPI – results generated by new PACE BitTorrent:plain:not_detected RDP:no_subprotocols:not_detected unknown:no_subprotocols:not_yet_detected BitTorrent:uTP:not_detected SMB/CIFS:no_subprotocols:not_detected SSH:no_subprotocols:not_detected BitTorrent:encrypted:not_detected DNS:no_subprotocols:not_detected Pando:no_subprotocols:not_detected NETBIOS:no_subprotocols:not_detected Yahoo:webmail:not_detected eDonkey:plain:not_detected SSL:generic:not_detected Socks:socksv5:not_yet_detected PPLIVE:no_subprotocols:not_detected Skype:unknown:not_detected PPSTREAM:no_subprotocols:not_detected Google:encrypted:not_detected unknown:no_subprotocols:not_detected FLASH:no_subprotocols:not_detected

26 DPI – results generated by nDPI-ng proto: TCP->SSL_with_certificate->POP3S, service: Google → encrypted POP3 session with a Google mail server proto: TCP->SSL_with_certificate, service: Twitter" → encrypted connection to a Twitter server proto: TCP->FTP_Data, content: JPG → file-transfer FTP session, which carries a JPG image proto: TCP->SSL_with_certificate->Dropbox, service: Dropbox → encrypted Dropbox session (the application is Dropbox) with the Dropbox server proto: TCP->SSL_with_certificate, service: Dropbox → encrypted session with a Dropbox server, while the application is unknown (it can be a web browser connection) proto: TCP->HTTP, content: WebM, service: YouTube → a flow from YouTube, which transports WebM movie proto: UDP->DNS, service: Facebook → DNS query about a hostname belonging to Facebook

27 Using QoS markers Class of service (CoS): 3-bit field that is present in an Ethernet frame header when 802.1Q VLAN tagging is present Very easy to cheat – everyone can set it to any value Most Internet Service Providers do not trust incoming QoS markings from their customers What can we get? ● Nothing more than previously set by a user or an application

28 Using QoS markers IP packets contain the Type of Service field, which can be used for layer-3 QoS marking What can we get? ● Nothing more than previously set by a user or an application – limited to trusted devices in the network

29 Using QoS markers Valid values for IP Precedence: Valid values for DSCP: 0 – 63

30 Statistical classification Based on rules, which can be written manually (slow and inefficient) or derived automatically by the use of Machine Learning Algorithms (MLAs) Very broad choice of MLAs: K-Nearest Neighbors, K-Means, Naive Bayes Filter, C4.5, J48, Random Forest, etc Achievable detection rate is over 95% MLAs require significant amount of good quality training data But... the speed is the power! What can we get? ● application ● content (indirectly) ● service (indirectly)

31 So how can we use the statistical methods? What can we use to classify the traffic by the statistical methods? IP protocol level → Type field from the IP packets Application level → statistical classification by packet sizes, ports, TCP flags, flow durations, etc Content level → statistical classification by IP addresses Service provider level → statistical classification by IP addresses What is the real result? ● Pretty good accuracy for the cases, which were trained by MLA ● Poor accuracy for all the other cases

32 Identification of service providers Monitoring of DNS replies delivers the required information Problems: many service providers using the same IP address “tcpdump -v -K -n -N -t -i eth0 udp src port 53” IP (tos 0x0, ttl 46, id 30600, offset 0, flags [none], proto UDP (17), length 102) > : /0/0 CNAME star.c10r.facebook.com., star.c10r.facebook.com. A (74) IP (tos 0x0, ttl 46, id 26945, offset 0, flags [none], proto UDP (17), length 181) > : /0/0 fbstatic-a.akamaihd.net. CNAME fbstatic- a.akamaihd.net.edgesuite.net., fbstatic-a.akamaihd.net.edgesuite.net. CNAME a1168.dsw4.akamai.net., a1168.dsw4.akamai.net. A , a1168.dsw4.akamai.net. A (153)

33 Part III A deep look into Deep Packet Inspection

34 How much information is needed? It depends on the specific DPI tool Libprotoident requires only 4 bytes of packet payload in each direction to recognize the traffic. The price: only IP protocol and application levels can be determined. Other tools also process following bytes, looking for specific signatures of a content or a service. Some signatures can identify the traffic after receiving 1 first packet with payload (as DNS, NTP, or BitTorrent). Finding the web service or content in an HTTP flow usually requires 4 first packets. The most 10 packets in each direction should be sufficient to determine all the flow characteristics.

35 Which information is used by DPI? Libprotoident: comparison of the first 4 Bytes of payload + of the packet lengths + port numbers if (!match_str_either(data, "\x01\x00\x00\x00")) return false; if (!match_chars_either(data, 0x00, 0x00, 0x00, ANY)) return false; if (data->payload_len[0] == 4 && data->payload_len[1] == 1) return true; if (data->server_port != 53 && data->client_port != 53) return false;

36 Which information is used by DPI? In PACE / OpenDPI / nDPI, there are the same checks: if ((payload_len > 0) && match_first_bytes(packet->payload, "\xe9\x03\x41\x01")) NDPI_LOG(0, ndpi_struct, NDPI_LOG_DEBUG, "Found PPLIVE.\n"); if ((payload_len == 0) || ((payload_len == 2) && (packet->payload[0] == 0x05) && (packet- >payload[1] == 0x00))) NDPI_LOG(0, ndpi_struct, NDPI_LOG_DEBUG, "Found SOCKS5.\n"); if ((payload_len == 0) || (payload_len == 49) ||(payload_len == 94)) NDPI_LOG(0, ndpi_struct, NDPI_LOG_DEBUG, "Found PPLIVE.\n"); if ((packet->udp->dest == htons(5041) || packet->udp->source == htons(5041)) NDPI_LOG(0, ndpi_struct, 0 "Possible PPLIVE...\n");

37 Which information is used by DPI? But they are done for each packet separately (in the order how the packets arrive), so we do not have access to the payload of the previous packet The detection status is kept in state variables associated with the particular flow

38 Which information is used by DPI? However, they use a bunch of other methods, as IP check: /* Apple (FaceTime, iMessage,...) /8 */ if(((saddr & 0xFF /* */) == 0x /* */) || ((daddr & 0xFF /* */) == 0x /* */)) { flow->ndpi_result_service = NDPI_RESULT_SERVICE_APPLE; }

39 Which information is used by DPI? Or TCP flags: if (packet->tcp->psh != 0 && flow->rtmp_bytes == 1537) NDPI_LOG(0, ndpi_struct, NDPI_LOG_DEBUG, Or even the number of processed packets: if (flow->packet_counter > 20) NDPI_LOG(0, ndpi_struct, NDPI_LOG_DEBUG.....

40 Which information is used by DPI? In order to discover web services and types of HTTP content, nDPI parses HTTP headers to discover the “host” and “content- type” lines. The “host” field is compared against domain names associated with the particular service, as: "amazon.com" -> NDPI_RESULT_SERVICE_AMAZON "amazonaws.com" -> NDPI_RESULT_SERVICE_AMAZON "amazon-adsystem.com" -> NDPI_RESULT_SERVICE_AMAZON ".apple.com" -> NDPI_RESULT_SERVICE_APPLE ".mzstatic.com" -> NDPI_RESULT_SERVICE_APPLE

41 Which information is used by DPI? The “content-type” field is compared against predefined values associated with the particular types of the content: "video/mp4" -> NDPI_RESULT_CONTENT_MPEG "video/mpeg" -> NDPI_RESULT_CONTENT_MPEG "video/nsv" -> NDPI_RESULT_CONTENT_MPEG "misc/ultravox" -> NDPI_RESULT_CONTENT_MPEG "audio/ogg" -> NDPI_RESULT_CONTENT_OGG "video/ogg" -> NDPI_RESULT_CONTENT_OGG

42 How to deal with the encrypted traffic? Encrypted web traffic increased from 20% (in 2011) to 45% (2014) from the whole web traffic The content is always unknown The application protocol (HTTPS, POPS, SMTPS, etc) discovered based on ports (e.g., port 465 = HTTPS) The service discovered based on: a) inspection of the server field in certificates (nDPI) b) matching with services based on cached DNS replies (TSTAT)

43 Part IV How to verify the accuracy of classification tools?

44 The origin of the reference data The reference data (ground-truth) are usually obtained in one of previously described ways, what causes incompleteness and high misclassification rate Publicly available databases contain very often incomplete and inaccurate data So how to provide good quality data?

45 Monitoring on the user's level System sockets provide name of the application associated with each particular stream in the network Ability to split HTTP streams according to their content Fast, precise, avoid privacy issues Avoid unreliability of port-based or statistical tools

46 Volunteer-Based System (VBS) Collects data from clients Enhanced privacy Application names are taken from system sockets Recognizes different types of HTTP contents Open-source, GPL licensed Windows (32/64-bit) and Linux Can be downloaded free of charge from SourceForge:

47 Design of the system Volunteer-Based System consists of clients installed on users' computers, the server located at Aalborg University, and statistics generators. Each part of the software can be developed independently, and it collaborates with other by the use of database SQL interfaces

48 The concept of a flow Remote end-point: IP address, port The way of transport: transport protocol Local end-point: IP address, port

49 Information logged for each flow Identifier of the client Start timestamp Hashed local, global, and remote IP addresses Local and remote ports Transport layer protocol Name of the application Name of the network

50 Information logged for each packet Identifier of the flow Direction (inbound / outbound) Size State of all TCP flags (for TCP flows) Time elapsed from the previous packet in the flow Type of the content (for HTTP flows)

51 Privacy is guaranteed! Masked users' identities Masked IP addresses (local, global, remote) Collecting only general information about transferred HTTP content (as image/gif, video/flv, audio/mpeg) We do not perform deep inspection of application payloads, all the collected information is obtained only from headers

52 Performance tests The tests were made on 16 client machines located in Poland The clients analyzed GB of data The server stored 7.4 GB of statistical data Communication between the clients and the server was responsible for around 5% of the traffic

53 Performance of VBS CPU usage by our system does not exceed 5% in average The system has no impact on the performance of users' computers VBS is running in background as Windows service or Linux daemon, so it is completely transparent to the users

54 The official project website The official project website contains: Broad description of the project Screenshots Roadmap Binary packages for Windows and Linux Source code in Git repository Comprehensive documentation of the source code Bug tracking and feature request system

55 Part V Implementation and various applications of VBS - a host-based monitoring tool

56 Implementation of our VBS Developed in Java, using Eclipse environment Open source, GPL licensed, freely available to everyone VBS is split into client, server, and statistics generator Modular design, all parts can be developed independently Running in the background as a Windows service or Linux daemon thanks to Yet Another Java Service Wrapper (YAJSW) – an open source project that provides support for both 32-bit and 64-bit versions of Windows and Linux The auto-update mechanism of the client

57 The client The client is installed on users' computers and are responsible for collecting the data about the users' traffic and sending them to the server. The client consists of the following modules: Packet capturer Socket monitor Flows generator Data transmitter

58 Packet capturer Uses jNetPcap Java library to collect packets from the network interface. The library makes use of the installed WinPcap / libpcap Captures all traffic passing the network interface except the traffic from the local subnet (the traffic is filtered by Pcap on the interface level) JnetPcap offers detecting and stripping various headers (data- link, IP, TCP, UDP, HTTP, etc) Packets are collected using native Pcap function loopPacket, which saves the resources consumed by VBS

59 Socket monitor Calls the external socket monitoring tools every second to ensure that even quick openings of sockets are registered Uses Netstat (Linux) or TCPViewCon (Windows) to get the list of open sockets Monitors both TCP and UDP sockets and provides the information about the time of opening, time of closing, and name of the application associated with the socket

60 Flows generator Organizes the captured packets into flows Attaches the application name to the flow based on the information from the socket monitor TCP flows are closed based on the time when the corresponding socket is closed UDP flows are closed based on timeout (one UDP socket can be associated with many flows, since only the local point is defined) Closed flows are stored in the SQLite database

61 Data transmitter When the local SQLite database file exceeds 700 kB, it is sent to the server Raw sockets are used by the communication, but a simple password authentication mechanism exists The transmitted file also includes identifier of the client and information about the computer on which the client is installed: version of the operating system, information about RAM and CPU(s)

62 The server Responsible for registering and authenticating clients Receives SQLite database files from clients Obtained files are extracted into MySQL database installed on the server machine There are the following tables in the MySQL database: Clients, Flows, Packets, Applications, ContentTypes, Performance So far we collected information about 13,242,858 flows and 999,731,839 packets associated with them. The information takes 75.4 GiB disk space.

63 Risk assessment The threat assessment made by us proved that the system can handle majority of potential security problems

64 Examples of the statistics Data obtained from 4 users: - User 1 – private user in Denmark, joined December 28, User 2 – private user in Poland, joined December 28, User 3 – private user in Poland, joined December 31, User 4 – private user in Denmark, joined April 24, 2012 Statistics calculated for all users altogether and for each user separately Statistics obtained on the per-application basis and per- content-type basis

65 Number of flows vs amount of traffic

66 Top applications for all users

67 Torrent download vs upload

68 Top HTTP content-types for all users

69 Amounts of traffic vs content types

70 Characterizing the applications We tried to obtain characteristics of 5 different applications based on traffic originated by all our users It is interesting to observe that 60% of packets (and 71% of packets carrying data) for chrome are inbound For dropbox the number of inbound and outbound packets is almost the same, but there is a large difference in the size of the inbound and outbound packets

71 The End Thank you for your attention!


Download ppt "Usefulness of the results - a forgotten evaluation metric of traffic identification tools Tomasz Bujlow Aalborg University."

Similar presentations


Ads by Google