Presentation is loading. Please wait.

Presentation is loading. Please wait.

Backtracking Algorithmic Complexity Attacks Against a NIDS Randy Smith, Cristian Estan, Somesh Jha University of Wisconsin–Madison.

Similar presentations


Presentation on theme: "Backtracking Algorithmic Complexity Attacks Against a NIDS Randy Smith, Cristian Estan, Somesh Jha University of Wisconsin–Madison."— Presentation transcript:

1 Backtracking Algorithmic Complexity Attacks Against a NIDS Randy Smith, Cristian Estan, Somesh Jha University of Wisconsin–Madison

2 Algorithmic Complexity Attacks Vulnerable algorithm: algorithm whose worst case differs from typical case. The larger the difference, the more vulnerable the algorithm. Examples: AlgorithmAverageWorst QuicksortO(n log n)O(n 2 ) Hash lookupconstantO(n)

3 Algorithmic Complexity Attacks Algorithmic Complexity Attack – an attacker induces worst-case behavior in a vulnerable algorithm.  Common observable effect is denial of service. Crosby and Wallach: induced worst-case behavior in hash function implementations. “Algorithms are now part of the attack surface” (Crosby and Wallach, 2003)

4 Are NIDS vulnerable? NIDS and IPS are ubiquitous, but… Do they contain vulnerable algorithms? Can they be exploited? YES! Only need 1 packet every 3 seconds.

5 Evading a NIDS Attacker’s Goal: Evade NIDS Two attack vectors in an evasion attempt: 1 st —alg. complexity attack targeting the NIDS 2 nd —true attack targeting the network Effect of an algorithmic complexity attack:  (NIDS) Packets enter network unexamined  (fail-closed IPS) Packets are dropped

6 Main results In Snort, vulnerability in rule-matching  worst-case vs. typical case: 6 orders of magnitude.  “Backtracking Attack”  Easily exploitable through packet payloads Improved rule-matching algorithm limits running time differences to within 1 order of magnitude.

7 Outline Snort rule matching Inducing backtracking attacks Countermeasures Measurement results Conclusion

8 Snort Rule Matching content:”fmt=”; //P1 content:”player=”; //P3 content:”overflow”,relative; //P5 alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”AudioPlayer jukebox exploit”; pcre:”/^(mp3|ogg)/”,relative; //P2 pcre:”/.exe|.com/”,relative; //P4 sid:5678)

9 Snort Rule Matching alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”AudioPlayer jukebox exploit”; content:”fmt=”; //P1 pcre:”/^(mp3|ogg)/”,relative; //P2 content:”player=”; //P3 pcre:”/.exe|.com/”,relative; //P4 content:”overflow”,relative; //P5 sid:5678) fmt=acc player=default fmt=mp3 rate=14kbps Rule matches!

10 Matching the packet P1 P5 P4 P3 P2 Rule matches! alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”AudioPlayer jukebox exploit”; content:”fmt=”; //P1 pcre:”/^(mp3|ogg)/”,relative; //P2 content:”player=”; //P3 pcre:”/.exe|.com/”,relative; //P4 content:”overflow”,relative; //P5 sid:5678) fmt=acc player=default fmt=mp3 rate=14kbps

11 Inducing Backtracking attacks P1,P2,P3,P4 match in 3 positions each P5 never matches fmt=acc player=default fmt=mp3 rate=14kbps fmt=mp3fmt=mp3fmt=mp3player=player=player=.exe.exe.exe alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”ReelAudio jukebox exploit”; content:”fmt=”; //P1 pcre:”/^(mp3|ogg)/”,relative; //P2 content:”player=”; //P3 pcre:”/.exe|.com/”,relative; //P4 content:”overflow”,relative; //P5 sid:5678) Leads to excessive packet traversals!

12 Matching the malicious packet alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”AudioPlayer jukebox exploit”; content:”fmt=”; //P1 pcre:”/^(mp3|ogg)/”,relative; //P2 content:”player=”; //P3 pcre:”/.exe|.com/”,relative; //P4 content:”overflow”,relative; //P5 sid:5678) fmt=mp3fmt=mp3fmt=mp3player=player=player=.exe.exe.exe P5 P2 P4 P5 P4 P5 P4 P3 P5 P4 P5 P4 P5 P4 P2 P3 P1 P5 P4 P5 P4 P5 P4 P2 P3 P1

13 Are real rules vulnerable? Rule number Processing (s/GB) Slowdown Same protoAll traffic 3682 (SMTP)30,933,874232,936X1,501,644X 2611 (Oracle)6,220,76856,296X301,979X 1382 (IRC)1,956,858134,031X94,993X 2403 (NetBIOS)357,777490X17,368X 1755 (IMAP)89,181444X4,329X

14 Safer backtracking Memoization: maintain a table of subproblem “answers”; never evaluate a predicate twice at the same starting payload offset alert tcp $EXT_NET any -> $HOME_NET 99 (msg:”AudioPlayer jukebox exploit”; content:”fmt=”; //P1 pcre:”/^(mp3|ogg)/”,relative; //P2 content:”player=”; //P3 pcre:”/.exe|.com/”,relative; //P4 content:”overflow”,relative; //P5 sid:5678) Identify constrained predicate sequences Monotone memoization: don’t re-evaluate monotone predicates that have been evaluated at lower offsets

15 Reductions in processing cost fmt=mp3fmt=mp3fmt=mp3player=player=player=.exe.exe.exe P5 P2 P4 P5 P4 P5 P4 P3 P5 P4 P5 P4 P5 P4 P2 P3 P1 P5 P4 P5 P4 P5 P4 P2 P

16 Outline Snort rule matching Inducing backtracking attacks Protecting against backtracking attacks Measurement results Conclusion

17 Measurement results Rule number Slowdown factor w.r.t. same protocol Beforew/ Memo (SMTP)232,936X0.95X 2611 (Oracle)56,296X1.57X 1382 (IRC)134,031X6.00X 2403 (NetBIOS)490X0.17X 1755 (IMAP)444X0.46X

18 Live experiment topology Background TrafficAC AttackTrue Attack

19 Live experiment Background 10Mbps AC Attack  Targets Snort SMTP rule 3682  Directed at sendmail server True Attack: NIMDA  300 exploit attempts, sent 1 byte per second.  New exploit started every second.

20 Live experiment results Attack Description Exploits Detected Required Rate (kbps) Control (No attack)300/ packets every 60 s.220/ packet every 5 s.4/ packet every 3 s.0/ packets initially0/ packet every 3 s.300/ packets initially300/300--

21 Conclusions NIDS operation is complex. Many opportunities for vulnerable algorithms. In Snort, rule-matching is vulnerable and can be exploited by an attacker. Memoization, along with other semantics- preserving operations, significantly reduces vulnerability. Other vulnerable algoritms exist.

22 Backtracking Algorithmic Complexity Attacks Against a NIDS Thank you.


Download ppt "Backtracking Algorithmic Complexity Attacks Against a NIDS Randy Smith, Cristian Estan, Somesh Jha University of Wisconsin–Madison."

Similar presentations


Ads by Google