1. A DoS Resilient Flow-level Intrusion Detection Approach for High-speed Networks Yan Gao, Zhichun Li, Yan Chen Lab for Internet and Security Technology (LIST) Northwestern University

2. Outline Motivation Background on sketches Design of the HiFIND system Evaluation Conclusion

3. The Spread of Sapphire/Slammer Worms

4. Existing Network IDSes Insufficient Signature based IDS cannot recognize unknown or polymorphic intrusions Statistical IDSes for rescue, but –Flow-level detection: unscalable Vulnerable to DoS attacks e.g. TRW [IEEE SSP 04], TRW-AC [ USENIX Security Symposium 04], Superspreader [NDSS 05] for port Symposium 04], Superspreader [NDSS 05] for port scan detection scan detection –Overall traffic based detection: inaccurate, high false positives e.g. Change Point Monitoring for flooding attack e.g. Change Point Monitoring for flooding attack detection [IEEE Trans. on DSC 04] detection [IEEE Trans. on DSC 04] Key features missing –Distinguish SYN flooding and various port scans for effective mitigation –Aggregated detection over multiple vantage points

5. Our Solution: HiFIND System Goal: accurate High-speed Flow-level INtrusion Detection (HiFIND) system Leverage our data streaming techniques: reversible sketches Select an optimal small set of metrics from TCP/IP headers for monitoring and detection Design efficient two-dimensional sketches to distinguish different types of attacks Aggregate compact sketches from multiple routers for distributed detection

6. Deployment of HiFIND Attached to a router/switch as a black box Edge network detection particularly powerful Original configuration Monitor each port separately Monitor aggregated traffic from all ports Router LA N Inter net Switch LA N (a) Router LAN Inter net LA N (b) HiFIND system scan port Splitter Router LA N Inter net LA N (c) Splitter HiFIND system Switch HiFIND system HiFIND system

7. Outline Motivation Background on sketches Design of the HiFIND system Evaluation Conclusion

8. k-ary sketch 1 j H 01K-1 … … … hj(k)hj(k) hH(k)hH(k) h1(k)h1(k) Update (k, v): T j [ h j (k)] += v (for all j) Estimate v(S, k): sum of updates for key k The first to monitor and detect flow-level heavy changes in massive data streams at network traffic speeds [IMC 03] + =  S=Combine( ,S1, ,S2):

9. Reversible Sketch Report keys with heavy changes Significantly improve its usage [IMC 2004, INFOCOM 2006, ACM/IEEE ToN to appear] Efficient data recording For the worst case traffic, all 40-byte packet streams Software: 526Mbps on a P4 3.2Ghz PC Hardware: 16 Gbps on a single FPGA broad INFERENCE(S,t) ? ?

10. Outline Motivation Background on sketches Design of the HiFIND system –Architecture –Sketch-based intrusion detection –Intrusion classification with 2D sketches –Feature analysis Evaluation Conclusion

11. Architecture of the HiFIND system

12. Threat model –TCP SYN flooding (DoS attack) –Port scan Horizontal scan Vertical scan Block scan Forecast methods –EWMA –Holt-Winter Forecasting Algorithm

13. Sketch-based Detection Algorithm RS({DIP, Dport}, #SYN - #SYN/ACK) –Detect SYN flooding attacks RS({SIP, DIP}, #SYN - #SYN/ACK) –Detect any intruder trying to attack a particular IP address RS({SIP, Dport}, #SYN - #SYN/ACK) –Detect any source IP which causes a large number of uncompleted connections to a particular destination port KeysSYN floodingHscanVscanScore {SIP, Dport}non-spoofedYesNo1.5 {DIP, Dport}YesNo 1 {SIP, DIP}non-spoofedNoYes1.5 {SIP}non-spoofedYes 2.5 {DIP}YesNoYes2 {Dport}Yes No2

14. Intrusion Classification Major challenge –Can not completely differentiate different types of attacks –E.g., if destination port distribution unknown, it is hard to distinguish non-Spoofing SYN flooding attacks from vertical scans by RS({SIP, DIP}, #SYN - #SYN/ACK) Bi-modal distribution SYN floodings Vertical scans

15. Two-dimensional (2D) Sketch For example: differentiate vertical scan from SYN flooding attack The two-dimensional k-ary sketches An example of UPDATE operation Accuracy analysis Examples: 5 hash tables, 3.2MB memory consumption –Vertical scan detected at least 99.56% –SYN attack classified correctly at least 99.99%

16. DoS Resilience Analysis HiFIND system is resilient to various DoS attacks as follows Send source spoofed SYN packets to a fixed destination –Detected as SYN flooding attack Send source spoofed packet to random destinations –Evenly distributed in the buckets of each hash table, no false positives Reverse-engineer the hash functions to create collisions –Difficult to reverse engineering of hash functions Unknown hash output of each hash function Multiple hash tables and different hash functions Even know the hash functions of sketches –Very hard to find collisions through exhaustive search 5.2×10 -18E.g. given 6 hash functions, the probability of a collision of two random keys in 5 hash functions is 5.2×10 -18

17. Distributed Intrusion Detection Naive solution: Transport all the packet traces or connection states to the central site HiFIND: Summarize the traffic with compact sketches at each edge router, and deliver them to the central site SYN1 SYN/ACK1 SYN2 SYN/ACK2

18. Outline Motivation Background on sketches Design of the HiFIND system Evaluation Conclusion

19. Evaluation Methodology Router traffic traces –Lawrence Berkeley National Laboratory One-day trace with ~900M netflow records –Northwestern University One day experiment in May 2005 with 239M netflow records, 1.8TB traffic and 1:1 packet samples Evaluation metrics –Detection accuracy –Online performance: Speed Memory consumption Memory access per packet

20. Highly Accurate

21. Detection Validation SYN flooding –Backscatter [USENIX Security Symposium 2001] Hscans and Vscans –The knowledge of port number e.g. 5 major scenarios of the top 10 Hscans e.g. 5 major scenarios of the bottom 10 Hscans Anonymized SIPDport# DIPCause 204.10.110.38143356275SQLSnake scan 5.4.247.103143354788SQLSnake scan 109.132.101.1992245014Scan SSH 95.30.62.202330625964MySQL Bot scans 15.192.50.153489923687Rahack worm Anonymized SIPDport# DIPCause 98.198.251.16813564Nachi or MSBlast worm 3.66.52.22744564Sasser and Korgo worm 2.0.28.9013964NetBIOS scan 98.198.0.10113564Nachi or MSBlast worm 165.5.42.10555462Sasser worm

22. Online performance evaluation Small memory access per packet –16 memory accesses per packet with parallel recording Small memory consumption Recording speed –Worst case: recording 239M items in 20.6 seconds i.e., 11M insertions/sec Detection speed –Detection on 1430 minute intervals Average detection time: 0.34 seconds Maximum detection time: 12.91 seconds –Stress experiments in each hour interval Detecting top 100 anomalies with average 35.61 seconds and maximum 46.90 seconds

23. Conclusion Proposed the first online DoS resilient flow-level IDS for high-speed networks Scalable to high–speed networks Highly accurate DoS attack resilient Distinguish SYN flooding and various port scans Aggregate detection over multiple vantage points

24. Thank You ! Questions? For more info: http://list.cs.northwestern.edu

25. K-ary Sketch 1 j H 01K-1 … … … hj(k)hj(k) hH(k)hH(k) h1(k)h1(k) Update (k, u): T j [ h j (k)] += u (for all j) Estimate v(S, k): sum of updates for key k Online data recording & estimation [IMC 2003] + =  S=COMBINE( ,S1, ,S2):

26. Two-dimensional (2D) Sketch Accuracy analysis –Given a key k of a vertical scan, the majority of the H hash matrices will classify k as a vertical scan attack with probability at least, where. ( ) –Given a key k of a SYN flooding, the majority of the H hash matrices will classify k as a SYN flooding attack with probability at least, where.

27. Related work [J. Jung et al. 2004]Threshold Random Walk (TRW) for port scan detection [J. Jung et al. 2004] –Not DoS resilient TRW with approximate caches (TRW-AC) [N. Weaver et al. 2004] –High false negatives under DoS attack [H. Wang et al. 2002]Change Point Monitoring (CPM) [H. Wang et al. 2002] –Detecting port scans as SYN floodings [D. Moore et al. 2001]Backscatter [D. Moore et al. 2001] –Only targeting randomly spoofed DoS attacks [S. Venkataraman et al. 2005]Superspreader [S. Venkataraman et al. 2005] –High false positives with P2P traffic [R. Kompella et al. 2004]Partial Completion Filters (PCF) [R. Kompella et al. 2004] –Not reversible