Slide title In CAPITALS 50 pt Slide subtitle 32 pt On the Validation of Traffic Classification Algorithms Géza Szabó, Dániel Orincsay, Szabolcs Malomsoky, István Szabó Traffic Lab, Ericsson Research Hungary

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt On the Validation of Traffic Classification Algorithms /17 Aim & Contents  Aim: –Introduce our novel validation method which makes it possible to measure the accuracy of traffic classification methods  Contents: –Requirements – How should validation be done? –Related work – How is it currently done? –Our proposal – What have we proposed? –Working mechanism – How does our proposal work? –Validation a state-of-the-art traffic classification method – What have we learnt from the validation? –Future work – What else can be done with the proposed method?

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt On the Validation of Traffic Classification Algorithms /17 Requirements – How should validation be done?  Objective of traffic classification: –Identify applications in passively observed traffic  Validation of classification method by active test -It should be independent from classification methods -About each packet the test should provide reference information -The test should be deterministic -Feasibility: create large tests in a highly automated way -Realistic environment

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt On the Validation of Traffic Classification Algorithms /17 Related work – How is it currently done? Traffic classification methods Port based classification Signature based classification Connection pattern based classification Statistics based classification Information theory based classification Combined classification method Validation methods Manual validation Use of other traffic classification method Measurement data Manually created / Active measurement Public availableNon public availableOnline measurement Header traces → port based method Impossible to validate by others Impossible to repeat with same conditions Non- realistic environment Dynamically allocated ports Proprietary protocols Encryption Be up2date Proprietary protocols Encryption Be up2date Lot of flows Simultaneous applications Lot of flows Simultaneous applications Previously well-classified traces Just hint S. Sen and J. Wang: Analyzing Peer-to- peer Traffic Across Large Networks T. Karagiannis, K. Papagiannaki and M. Faloutsos : BLINC: Multilevel Traffic Classification in the Dark J. Erman, M. Arlitt and A. Mahanti : Traffic Classification Using Clustering Algorithms L. Bernaille et al: Traffic Classification On The Fly CURRENTLY Weak and ad hoc validation No reliable and widely accepted validation technique No reference packet trace with well-defined content is available CURRENTLY Weak and ad hoc validation No reliable and widely accepted validation technique No reference packet trace with well-defined content is available

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms OUR PROPOSAL

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms The proposed method for validation  Principle: –Packets are collected into flows at the traffic generating terminal –Flows are marked with the identifier of the application that generated the packets of the flow  The main requirements on the realization of the method: –It should not deteriorate the performance of the terminal –The byte overhead of marking should be negligible  The preferred realization is a driver that can be easily installed on terminals The position of the proposed driver within the terminal

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Working mechanism 1.The packet is examined whether it is an incoming or outgoing packet 2.In case of an outgoing packet, the size of the packet is examined  Continues with only those packets which are smaller than the MTU decreased with the size of marking 3.The process continues with only TCP or UDP packets 4.According to the five-tuple identifier of the packet, it is checked whether there is already available information about which application the flow belongs to 5.Query operation system 6.Need marking:  Randomly  Only first  Leave the first  No mark The working mechanism of the introduced driver

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Place of marking  Extending the original IP packet with one option field –Router Alert option field  Transparent for both the routers on the path and also for the receiver host (according to RFC 2113 [3]).  The first two characters of the corresponding executable file name are added –Increasing the size of the packet with 4 bytes –The packet size field in the IP header is also increased with 4 bytes –Header checksum is recalculated A marked packet of the BitTorrent protocol

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms PROOF-OF-CONCEPT

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Reference measurement  Available at  In a separated access network  Our driver has been installed onto all computers on this network  Duration of the measurement: 43 hours  Captured data volume: 6 Gbytes, containing 12 million packets  The measurement contains the traffic of the most popular –P2P protocols:  BitTorrent  eDonkey  Gnutella  DirectConnect –VoIP and chat applications:  Skype  MSN Live –FTP sessions –Download manager – sending, receiving sessions –Web based (e.g., Gmail) –SSH sessions –SCP sessions –FPS, MMORPG gaming sessions –Streaming:  Radio  Video  Web based The traffic mix of the measurement

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Validation results (1) – Success  Combined traffic classification method (described in [1]) with the addition that the classification of VoIP applications has been extended with ideas from [2]  Accurately identified: – –Filetransfer –Streaming –Secure channel –Gaming traffic  Success due to: –Well-documented protocols –Open standards –Do not constantly change  Difficulties in case of…? –Encryption:  But: session initiation phase is critical as this phase can be identified accurately  Success: SSH or SCP [1] G. Szabo, I. Szabo and D. Orincsay: Accurate Traffic Classification [2] M. Perenyi and S. Molnar: Enhanced Skype Traffic Identification The results of the classification compared [1] to the reference measurement

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Validation results (2) – P2P Difficulties:  Many TCP flows containing 1-2 SYN packets probably to disconnected peers –No payload in these packets =>the signature based methods can not work –Dynamically allocated source ports towards not well-known destination ports => the port based methods fail –Server search and P2P communication heuristic [1] methods also fail => there are no other successful flows to such IPs  Also some small non-P2P flows were misclassified into the P2P class –Not fully proper content of the port- application database –Creating too many port-application associations easily results in the rise of the misclassification ratio.  The constant change of P2P protocols –New features added to P2P clients day-by-day –Working mechanism can be typical for a selected client not the whole protocol itself [1] G. Szabo, I. Szabo and D. Orincsay: Accurate Traffic Classification [2] M. Perenyi and S. Molnar: Enhanced Skype Traffic Identification The results of the classification compared [1] to the reference measurement

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Validation results (3) – Philosophy  Traffic which is the derivation of other traffic: –E.g., DNS traffic –MSN: HTTP protocol for transmitting chat messages –MSN client transmits advertisements over HTTP, but this cannot be recognized as deliberate web browsing  Hit := the classification outcome and the generating application type (the validation outcome) agreed –E.g., the chat on the DirectConnect hubs which has been classified as chat could have been considered as actually correct but in this comparison it was considered as misclassification The results of the classification compared [1] to the reference measurement [1] G. Szabo, I. Szabo and D. Orincsay: Accurate Traffic Classification

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms Validation results (4) – VoIP: MSN, Skype  High VoIP hit ratio is due to the successful identification –MSN Messenger –Skype  Skype is difficult to identify –Same problem as in the case of P2P –Proprietary protocol designed to ensure secure communication –[2] characteristic feature: the application sends packets even when there is no ongoing call with an exact 20 sec interval. –In [1]: a P2P identification heuristic which was designed to track any message which has a periodicity in packet sending –Extension of [1] was straightforward  The validation showed: –The deficiency of the classification of Skype  Simple extension of the algorithm –Idea of [1] has been validated as it proved to be robust for the extension with new application recognition –Also the validation mechanism proved to be useful [1] G. Szabo, I. Szabo and D. Orincsay: Accurate Traffic Classification [2] M. Perenyi and S. Molnar: Enhanced Skype Traffic Identification The results of the classification compared [1] to the reference measurement

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms It is independent from classification methods About each packet the test provides reference information The test is deterministic Feasibility: creates large tests in a highly automated way Summary  We introduced a new active measurement method which can help in the validation of traffic classification methods.  The introduced method is a network driver –Mark the outgoing packets from the clients with an application specific marking  With the introduced method we created a measurement and used this to validate the method presented in [1] –The method has been proved to be working accurately –Some deficiencies in the classification  P2P applications  Skype [1] G. Szabo, I. Szabo and D. Orincsay: Accurate Traffic Classification Benefits:

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt On the Validation of Traffic Classification Algorithms /17 Further work  Use the marking method at the measurement side for online traffic classification –Assumptions:  The terminals accessing an operator’s network are all installed with the proposed driver  The driver is made tamper-proof to avoid users forging the marking –Online clustering of the traffic into QoS classes based on the resource requirements of the generating application –Used by operators to charge on the basis of the used application by the user  Extension of the marking by other information about the traffic generating application –E.g., version number  Operator could track the security risks of an old application

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt On the Validation of Traffic Classification Algorithms /17 Questions, discussion…  Thank you very much for your kind attention!  Contact: –

Top right corner for field-mark, customer or partner logotypes. See Best practice for example. Slide title 40 pt Slide subtitle 24 pt Text 24 pt Bullets level pt /17 On the Validation of Traffic Classification Algorithms