Presentation on theme: "Backtracking Algorithmic Complexity Attacks Against a NIDS"— Presentation transcript:
1Backtracking Algorithmic Complexity Attacks Against a NIDS Randy Smith, Cristian Estan, Somesh JhaUniversity of Wisconsin–Madison
2Algorithmic Complexity Attacks Vulnerable algorithm: algorithm whose worst case differs from typical case. The larger the difference, the more vulnerable the algorithm.Examples:AlgorithmAverageWorstQuicksortO(n log n)O(n2)Hash lookupconstantO(n)
3Algorithmic Complexity Attacks Algorithmic Complexity Attack – an attacker induces worst-case behavior in a vulnerable algorithm.Common observable effect is denial of service.Crosby and Wallach: induced worst-case behavior in hash function implementations.“Algorithms are now part of the attack surface” (Crosby and Wallach, 2003)
4Are NIDS vulnerable? NIDS and IPS are ubiquitous, but… Do they contain vulnerable algorithms? Can they be exploited?YES! Only need 1 packet every 3 seconds.
5Evading a NIDS Attacker’s Goal: Evade NIDS Two attack vectors in an evasion attempt:1st—alg. complexity attack targeting the NIDS2nd—true attack targeting the networkEffect of an algorithmic complexity attack:(NIDS) Packets enter network unexamined(fail-closed IPS) Packets are dropped
6Main results In Snort, vulnerability in rule-matching worst-case vs. typical case: 6 orders of magnitude.“Backtracking Attack”Easily exploitable through packet payloadsImproved rule-matching algorithm limits running time differences to within 1 order of magnitude.
14Safer backtrackingMemoization: maintain a table of subproblem “answers”; never evaluate a predicate twice at the same starting payload offsetalert tcp $EXT_NET any -> $HOME_NET 99(msg:”AudioPlayer jukebox exploit”;content:”fmt=”; //P1pcre:”/^(mp3|ogg)/”,relative; //P2content:”player=”; //P3pcre:”/.exe|.com/”,relative; //P4content:”overflow”,relative; //P5sid:5678)Identify constrained predicate sequencesMonotone memoization: don’t re-evaluate monotone predicates that have been evaluated at lower offsets
15Reductions in processing cost 41118P5P4P2P3P5P4P2P3P271421P3283542P4P4P4465054P5P5P5P5P5P5P5P5P5fmt=mp3fmt=mp3fmt=mp3player=player=player=.exe.exe.exe
19Live experiment Background Traffic @ 10Mbps AC Attack Targets Snort SMTP rule 3682Directed at sendmail serverTrue Attack: NIMDA300 exploit attempts, sent 1 byte per second.New exploit started every second.
20Live experiment results Attack DescriptionExploitsDetectedRequired Rate (kbps)Control (No attack)300/300--2 packets every 60 s.220/3000.41 packet every 5 s.4/3002.41 packet every 3 s.0/3004.020 packets initially0.8
21ConclusionsNIDS operation is complex. Many opportunities for vulnerable algorithms.In Snort, rule-matching is vulnerable and can be exploited by an attacker.Memoization, along with other semantics-preserving operations, significantly reduces vulnerability.Other vulnerable algoritms exist.
22Backtracking Algorithmic Complexity Attacks Against a NIDS Thank you.