1. Modeling Network Traffic as Images Seong Soo Kim and A. L. Narasimha Reddy Computer Engineering Department of Electrical Engineering Texas A&M University {skim, reddy}@ee.tamu.edu

2. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 2 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

3. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 3 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

4. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 4 Attack/ Anomaly Bandwidth attacks/anomalies, Flash crowds DoS – Denial of Service : –UDP flooding, TCP SYN flooding, ICMP flooding Typical Types: -Single attacker (DoS) -Multiple Attackers (DDoS) -Multiple Victims (Worm)  Aggregate Packet header data as signals  Signal/image based anomaly/attack detectors

5. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 5 Motivation (1) Previous studies looked at individual flow’s behavior -Partial state -RED-PD  These become ineffective with DDoS  Aggregate Link speeds are increasing -currently at G b/s, soon to be at 10~100 G b/s  Need simple, effective mechanisms to implement at line speeds. Look at aggregate information of traffic -Use sampling to reduce the cost of processing  Process aggregate data to detect anomalies.

6. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 6 Motivation (2) Signature (rule)-based approaches are tailored to known attacks – Look for packets with port number #1434 (SQL Slammer) -Become ineffective when traffic patterns or attacks change  New threats are constantly emerging  Do not want to rely on attack specific information Most current monitoring/policing tools are done off-line -Flowscan, FlowAnalyzer, AutoFocus  Quick identification of network anomalies is necessary to contain threat Can we design generic (and generalized) mechanisms for attack detection and containment?  Measurement (network)-based real-time detection

7. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 7 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

8. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 8 Packet Header Carry a rich set of information -Data : Packet counts, Byte counts, Number of Flows -Domain : source/destination Address, source/destination Port numbers, Protocol numbers  Image/Video can represent each data in each domain Image processing/Video analysis decipher the patterns of traffic -single  multiple (Worm) : horizontal lines -multiple  single (DDoS) : vertical lines

9. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 9 Domain size Reduction(1) Header fields may have large domain spaces –IPv4 addresses 2 32, IPv6 addresses 2 64 Need to minimize storage and processing complexity for real-time processing Employ “domain folding” For example: A data structure of a 2 dimensional array count[i][j] -To record the packet count for the address j in i th field of the IP address Effects -32-bit address into four 8-bit fields -Smaller memory 2 32 (4G)  4*256 (1K) -Running time O(n) to O(lgn) -Form of hashing -Advantages -It is possible to reverse the hashing to identify the target IP address restrictively

10. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 10 Simple example IP 1 = 165. 91. 212. 255, No. of Flows = 3 IP 2 = 64. 58. 179. 230, No. of Flows = 2 IP 3 = 216. 239. 51. 100, No. of Flows = 1 IP 4 = 211. 40. 179. 102, No. of Flows = 10 IP 5 = 203. 255. 98. 2, No. of Flows = 2 Data structure for reducing domain size (2) 0 64 128 192 255 3 3 3 3

11. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 11 Simple example IP 1 = 165. 91. 212. 255, No. of Flows = 3 IP 2 = 64. 58. 179. 230, No. of Flows = 2 IP 3 = 216. 239. 51. 100, No. of Flows = 1 IP 4 = 211. 40. 179. 102, No. of Flows = 10 IP 5 = 203. 255. 98. 2, No. of Flows = 2 0 64 128 192 255 2 3 2 10 1 10 2 3 1 2 1 2 12 3 2 1 10 2 3 Data structure for reducing domain size (2)

12. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 12 Visual Representation

13. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 13 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

14. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 14 Image based analysis Generating useful signals based on traffic image Treat the traffic data as images Apply image processing based analysis Enables applying image/video processing for the analysis of network traffic. –Some attacks become clearly visible to the human eye. –Video compression techniques lead to data reduction –Scene change analysis leads to anomaly detection –Motion prediction leads to attack prediction –Pattern recognition leads to anomaly identification

15. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 15 Sampling Rates –For discriminating current traffic situation based on stationary property, we should select a sampling frequency for deriving the most stable images –The periodicity of traffic Impacts of Design Factors for presenting Network traffic as Images (1)

16. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 16 Impacts of Design Factors for presenting Network traffic as Images (2) Sampling Rates –The traffic is stationary in normal times and the selection of sampling period is not crucial. –The traffic changes dynamically with time in attack times and the sampling period is a crucial factor. –30 ~ 120 sec. sampling.

17. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 17 Flow-based Network Traffic Images The number of flows based visual representation –The number of flows in (source/destination) address domain –The black dots/lines illustrate more concentrated traffic intensity. –An analysis is effective for revealing flood types of attacks Image reveals the characteristics of traffic –Normal behavior mode –A single target (DoS) –Semi-random target : a subnet is fixed and other portion of address is changed (Prefix-based attacks) –Random target : horizontal (Worm) and vertical scan (DDoS)

18. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 18 Network traffic as images – normal network traffic Standard deviation of most significant DCT coefficients of images –energy distribution of number of flows over address domain. At normal traffic state, this signal is at a middle level between later two anomalous cases. Legitimate flows do not form any regular shape due to their random distribution over address domain.

19. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 19 Network traffic as images – semi-random targeted attacks The difference between attackers (or victims) and legitimate users is remarkable –higher variance than normal traffic The specific area of data structure is shown in a darker shade. –traffic is concentrated on a (aggregated) single destination or a subnet.

20. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 20 Network traffic as images –random targeted attacks Worm propagation type attack DDoS propagation type attack All of the addresses are exploited in hostscans attacks –Uniform intensity  low variances Whole region of the image in uniform intensity. Horizontal/vertical lines indicate anomalies in 2D image Random (sequential, dictionary scan) attacks -Horizontal scan : From the same source aimed at multiple targets -- Worm propagation -Vertical scan : From several machines (in a subnet) to a single destination -- DDOS

21. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 21 Summary of Visual representation of traffic data Worm attacks – horizontal line in 2D image DDoS attacks – vertical line in 2D image  Line detection algorithm Visual images look different in different traffic modes Motion prediction can lead to attack prediction

22. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 22 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

23. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 23 Generation of useful Signal Scene change analysis - DCT We can apply various image processing techniques From generated images, we can generate useful signals through DCT (Discrete Cosine Transform) DCT is effective for storage reduction and approximation of the energy distribution in image Variance of leading DCT coefficients in 8-by-8 blocks  Instead of whole DCT coefficients, we can choose only the dominant coefficient

24. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 24 Impact of Selecting DCT coefficients (1) TCG ( G T ) : Transformation Coding Gain –TCG measures the amount of energy packed in the low frequency (leading) coefficient –The higher TCG leads to smaller intra-frame MSE and higher compression

25. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 25 Intra_frame DCT –Random traffic can be packed within fewer coefficients than semi- random traffic –Using inter-frame differential coding,we can improve the G T –For MSE of 0.3349, the required coefficients reduce from 42 to 3 –TCG increases 2.6 times Impacts of Selecting DCT coefficients (2)

26. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 26 Impacts of Design Factors for presenting Network traffic as Images Sampling rates on DCT coefficients –A sampling rate of 60 seconds maintains the minimum intra- frame MSE over the entire range of retained DCT coefficients -We can choose 30 ~ 120 sec. as appropriate sampling period.

27. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 27 Attack Estimation (1) - Motion prediction Step 1: complexity reduction –Pixels below a mean packet count –Normalized absolute difference similarity Step 2: to find a block of addresses

28. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 28 Attack Estimation (2) - Motion prediction Step 3: to calculate the quantitative components –Starting position –Motion vector Step 4: compensating errors

29. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 29 Advantages Not looking for specific known attacks Generic mechanism Works in real-time –Latencies of a few samples –Simple enough to be implemented inline

30. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 30 Contents Introduction and Motivation Network Traffic as Images -Visual Representation Requirements for Representing Network Traffic as Images -Sampling Rates -Visual modeling Network Traffic as Images  normal traffic, semi-random attacks, random attacks Image Processing for Network Traffic -Validity of intra-frame DCT -Inter-frame differential coding Conclusion

31. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 31 Conclusion We studied the feasibility of analyzing packet header data through Image and DCT analysis for detecting traffic anomalies. We evaluated the effectiveness of our approach by employing network traffic. Can rely on many tools from signal/image processing area –More robust offline analysis possible –Concise for logging and playback Real-time resource accounting is feasible Real-time traffic monitoring is feasible –Simple enough to be implemented inline

32. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 32 Thank you !!

33. Seong Soo Kim and A. L. Narasimha ReddyTexas A & M University ICC 2005 33 Processing and memory complexity Two samples of packet header data 2*P, P is the size of the sample data Summary information (DCT coefficients etc.) over samples S Total space requirement O(P+S) P is 2 32  4*256 = 1024 (1D), 2 64  256K (2D) S is 32*32  16  Memory requires 258K Processing O(P+S) Update 4 counters per domain Per-packet data-plane cost low.