1. Towards a High-speed Router-based Anomaly/Intrusion Detection System (HRAID) Zhichun Li, Yan Gao, Yan Chen ({lizc,ygao,ychen}@cs.northewstern.edu) Northwestern Lab for Internet and Security Technology (LIST) Department of Computer Science, Northwestern University http://list.cs.northwestern.edu/hpnaidm.html Router LAN Internet Switch (a) Router LAN (b) HRAID system scan port Splitter Router (c) Splitter HRAID system HRAID system HRAID system Switch LAN Internet Attach HRAID black boxes to high-speed routers (a) original configuration, (b) distributed configuration for which each port is monitored separately, (c) aggregate configuration for which a splitter is used to aggregate the traffic from all the ports of a router. Current Intrusion Detection Systems and Shortcomings Mostly host-based and not scalable to high- speed networks Slammer worm infected 75,000 machines in <10 mins High speed gateways and backbone are vantage points to detect attacks Mostly signature-based and cannot recognize unknown anomalies/intrusions existing flow level approach not scalable to high volume traffic. overall traffic based detection cannot cooperate with attack mitigation EWMA RS((DIP, Dport), SYN-SYN/ACK) RS((SIP, DIP), SYN-SYN/ACK) RS((SIP, Dport), SYN-SYN/ACK) The design of a HRAID system Features of HRAID System Online traffic recording compact data structure: reversible k -ary sketch small memory usage (fit in SRAM) small memory accesses per packet Online flow-level anomaly/intrusion detection & mitigation TCP SYN flooding, horizontal and vertical scan even when mixed Infer key characteristics of malicious flows for mitigation HRAID: First flow-level intrusion detection that can sustain 10s Gbps bandwidth even for worst case traffic of 40-byte packet streams Attack typesRS((DIP, Dport), SYN-SYN/ACK) RS((SIP, DIP), SYN-SYN/ACK) RS((SIP, Dport), SYN-SYN/ACK) SYN floodingYes Vertical scansNoYesNo Horizontal scansNo Yes Reversible Sketch Based Anomaly Detection Input stream: (key, update) (e.g., SIP, SYN-SYN/ACK) Sketch module Forecast module(s) Anomaly detection module (k,u) … Sketches Error Sketch Alarms Report flows with large forecast errors Infer the (characteristics) key for mitigation Summarize input stream using sketches Build forecast models on top of sketches Array of hash tables: T j [K] (j = 1, …, H) Similar to count sketch, counting bloom filter, multi- stage filter, … 1 j H 01K-1 … … … hj(k)hj(k) hH(k)hH(k) h1(k)h1(k) Update (k, u): T j [ h j (k)] += u (for all j) k -ary Sketch: Update, Combine and Estimate Sketches are linear – Can combine sketches + = – Can aggregate data from different times, locations, and sources Estimate v(S, k): sum of updates for key k Streaming data recording reversible k -ary sketch value stored value Modular hashing IP mangling key Original k -ary sketch 2-universal hashing Heavy key detection reversible k -ary sketch Reverse hashing Reverse IP mangling heavy keys threshold Verified w/ original sketch Iterative approach The two-phase heavy key inference of Reversible k -ary Sketch Preliminary Evaluation Evaluated with NU traces (239M flows, 1.8TB traffic/day) Scalable - Can handle hundreds of millions of time series Accurate Anomaly Detection w/ Reversible Sketch - Compared with detection using complete flow-level tables - Provable probabilistic accuracy guarantees - Even more accurate on real Internet traces Efficient - For the worst case traffic, all 40 byte packets * 16 Gbps on a single FPGA board * 526 Mbps on a Pentium4 3.2 GHz PC - Only less than 3MB memory used - Only 15 memory access per packet for 48 bit reversible sketches and 16 per packet for 64 bit reversible sketches DescriptionDportcount SQLSnake14335 W32.Rahack48992 unknown scan61011 Scan SSH221 MySQL Bot scans102021 DescriptionDportcount Sasser or Korgo worm4453 W32.Sasser.B.Worm55541 Nachi or MSBlast worm1353 NetBIOS scan1393 Top 10 horizontal scans Bottom 10 horizontal scans 25 SYN flooding, 936 horizontal scans and 19 vertical scans detected (after sketch-based false positive reduction) 18 out of 25 SYN flooding verified w/ backscatter Scans verified (all for vscan, top and bottom 10 for hscan) Acknowledgment: Thank Yin Zhang for the original k -ary sketch slides