Yan Chen Northwestern Lab for Internet and Security Technology (LIST) Dept. of Computer Science Northwestern University Adaptive Intrusion Detection and Mitigation Systems for WiMAX Networks Motorola Liaisons Gregory W. Cox, Z. Judy Fu, Philip R. Roberts Motorola Labs

The Spread of Sapphire/Slammer Worms

Current Intrusion Detection Systems (IDS) Mostly host-based –not scalable to high-speed networks Mostly simple signature-based –Can’t deal with unknown attacks, polymorphic worms Statistical detection –Unscalable for flow-level detection –Overall traffic based: inaccurate, high false positives Cannot differentiate malicious events with unintentional anomalies

Adaptive Intrusion Detection and Mitigation for WiMAX Networks (WAIDM) Attached to a switch connecting BS as a black box Enable the early detection and mitigation of global scale attacks Could be differentiator for Motorola’s products Original configuration WAIDM deployed Inter net BS User s (a) (b) BS User s Switch/ BS controller Internet scan port WAIDM system BS Users BS Users Switch/ BS controller

Adaptive Intrusion Detection and Mitigation for WiMAX Networks (WAIDM) Scalability –Online traffic recording »Reversible sketch for data streaming computation »Record millions of flows (GB traffic) in a few hundred KB –Online sketch-based flow-level anomaly detection »Adaptively learn the traffic pattern changes Accuracy Integrated approach for false positive reduction –Automatic Polymorphic Worm signature generation (Hamsa) –Network element fault Diagnostics with Operational Determinism (ODD)

WAIDM Architecture Reversible sketch monitoring Filtering Sketch based statistical anomaly detection (SSAD) Local sketch records Sent out for aggregation Remote aggregated sketch records Per-flow monitoring Streaming packet data Normal flows Suspicious flows Intrusion or anomaly alarms Keys of suspicious flows Keys of normal flows Data path Control path Modules on the critical path Signature -based detection Polymorphic worm detection (Hamsa) Part I Sketch- based monitoring & detection Part II Per-flow monitoring & detection Modules on the non-critical path Network fault diagnosis (ODD)

Accomplishments Motorola Interactions –The first two components of WAIDM are ready for field test on Motorola WiMAX networks or testbed –Product teams interested to use as differentiator (Networks security service director: Randall Martin) –Close collaboration/interaction with Motorola Labs (Judy Fu, Phil Roberts, Steve Gilbert) Patents being filed through Motorola –Reverse Hashing for High-speed Network Monitoring: Algorithms, Evaluation, and Applications. –Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience. Students involved –Three Ph.D. students: Yan Gao, Zhichun Li, & Yao Zhao –One M.S. student: Prasad Narayana

Accomplishments on Publications Four conference papers and one journal papers (with another four under submission) –A DoS Resilient Flow-level Intrusion Detection Approach for High-speed Networks, to appear in IEEE International Conference on Distributed Computing Systems (ICDCS), 2006 (14%). –Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience, to appear in IEEE Symposium on Security and Privacy, 2006 (about 8%). –Reverse Hashing for High-speed Network Monitoring: Algorithms, Evaluation, and Applications, Proc. of IEEE INFOCOM, 2006 (18%). –IDGraphs: Intrusion Detection and Analysis Using Stream Compositing, to appear in IEEE Computer Graphics & Applications, special issue on Visualization for Cyber Security, »An earlier version also in Proc. of the IEEE Workshop on Visualization for Computer Security (VizSEC), 2005

Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience

Desired Requirements for Polymorphic Worm Signature Generation No existing work satisfies these requirements ! Network based, no host-level info Noise tolerant –Most network flow classifiers suffer false positives. –Even host based IDSes, such as honeynets, can be injected with noise. Attack resilience –Attackers always try to evade the IDS Efficient signature matching for high-speed links

Hamsa Architecture

Choice of Signatures Two classes of signatures Content based –Invariant content »Protocol Frame »Control Data: leading to control flow hijacking »Worm Executable Payload –Token: a substring with reasonable coverage to the suspicious traffic –Signatures: conjunction and/or sequence of tokens Behavior based Our choice: content based –Fast signature matching »ASIC based approach can achieve 6 ~ 8Gbps –Generic, not depend upon any protocol or server

Hamsa Design Key idea: model the uniqueness of worm invariants –Greedy algorithm for finding token conjunction signatures Highly accurate while much faster –Both analytically and experimentally –Compared with the latest work, polygraph –Suffix array based token extraction Provable attack resilience guarantee –Propose an adversary model Noise tolerant

Hamsa Signature Generator Core part: Model-based Greedy Signature Generation Iterative approach for multiple worms Signature refinement for better specificity –False positive is worse than false negative

Experiment Methodology Experiential setup: –Suspicious pool: »Three pseudo polymorphic worm based on real exploits (Code- Red II, Apache-Knacker and ATPhttpd), »Two polymorphic engine from Internet (CLET and TAPiON). –Normal pool: 2 hour departmental http trace (326MB) Signature evaluation: –False negative: 5000 generated worm samples per worm –False positive: »4-day departmental http trace (12.6 GB) »3.7GB web crawling including.mp3,.rm,.ppt,.pdf,.swf etc. »/usr/bin of Linux Fedora Core 4

Results on Signature Quality Single worm with noise –Suspicious pool size: 100 and 200 samples –Noise ratio: 0%, 10%, 30%, 50% –Noise samples randomly picked from the normal pool –Always get above signature and accuracy Multiple worms with similar results Worms Training FN Training FP Evaluation FN Evaluation FP Binary evaluation FP Signature Code-Red II00000 {'.ida?': 1, '%u780': 1, ' HTTP/1.0\r\n': 1, 'GET /': 1, '%u': 2} CLET00.109% %0.268% {'0\x8b': 1, '\xff\xff\xff': 1,'t\x07\xeb': 1}

Speed and Attack Resilience Results Implementation with hybrid of C++/Python –500 samples with 20% noise, 326MB normal traffic pool, 15 seconds on an XEON 2.8Ghz Provable attack resilience –We propose a new attack, token-fit –It fails the existing state-of-the-art, Polygraph –BUT We still can generate correct signature!

Ongoing Work Semantics Aided Signature Generation for Zero-day Polymorphic Worms –Some worms do not have any content invariant –Incorporate semantic information for more accurate detection Vulnerability Analysis for WiMAX Network Protocols –Use formal verification methods to automatically search for vulnerabilities in specs. –Completeness and correctness