On the Difficulty of Scalably Detecting Network Attacks Kirill Levchenko with Ramamohan Paturi and George Varghese

Which Intrusion Detection problems are hard (require per-flow state)? Counting distinct flows? Approximately? Detecting SYN Flooding? Port Scans? Are there efficient algorithms we don’t know yet? How to tell? Detecting Attacks

Abstract problem formulation Communication Complexity Source for NIDS problem reductions Establishes worst-case lower bounds Easy and fun to use Theoretical Techniques

Problem Motivation and Setting The Set Disjointness problem Application to SYN Flooding Implications Conclusion Outline

Problem Setting Idea: move the NIDS into the network, protect more systems!

Problem Setting Protect more hosts Single device to administer More informed position

Challenges: More hosts to protect Higher bandwidth links How do resource requirements scale? Are there fundamental limits? Problem Setting

Two packet sequences: traffic entering and leaving the network Goal: detect attack using packet sequences Problem Setting Outbound Inbound

Formalize the algorithmic problem An algorithm on packet sequences Reduce one problem to another Show solving one problem solves the other Algorithms & Reductions

Solving the NIDS problem solves the Set Disjointness problem Use strong bounds on Set Disjointness Establishes space lower bound for the NIDS problem Our Reduction Set Disjointness → NIDS Problem

Set Disjointness AliceBob Disjoint?

Set Disjointness AliceBob Communication: (even randomized)

Let’s Try It! (on SYN Flooding)

SYN Flooding NormalAttack Goal: detect unclosed connections.

SYN Flooding 1. Creates packet sequence corresponding to her set with SYN flag 2. Runs NIDS algorithm on input sequence 3. Suspends it after reading the last item 4. Sends the state of the algorithm to Bob

SYN Flooding Yes/No 1. Creates packet sequence corresponding to elements not in his set with FIN flag 2. Resumes NIDS algorithm on input sequence (using state sent by Alice) 3. Result indicates if sets intersect or not

SYN Flooding If A and B intersect, there is a SYN packet not followed by a FIN If A and B are disjoint, every SYN packet has a matching FIN Algorithm sees:

1. Alice sent Bob the state of the NIDS algorithm: 2. Set Disjointness requires Ω(n) bits ∴ The state of the algorithm is Ω(n) bits SYN Flooding no. of flows

Port Scans per-host state required TCP connection hijacking per-flow state required Evasion by fragmentation re-assembly required Other NIDS Problems

Sometimes per-flow state is required, but: Examples are artificial (may not occur in practice) Problem semantics may be a great help System can fail gracefully when out of memory Additional information may be available! Implications

Additional information can be useful: Count outgoing SYN+ACK and FIN packets Works if protected network can be trusted Hop-count filtering or other fingerprinting Small UDP fragments are unusual Implications

Set Disjointness is a useful reduction source Even hard under randomization Permits relaxations (see paper) Exposes and formalizes the hardness of some NIDS problems Guides practical algorithm design to consider hard cases and failure modes Conclusion