1. Deterministic Memory- Efficient String Matching Algorithms for Intrusion Detection Nathan Tuck, Timothy Sherwood, Brad Calder, George Varghese Department of Computer Science and Engineering, University of California, San Diego Department of Computer Science, University of California, Santa Barbara

2. Abstract IDSs ： Intrusion Detection Systems Space and time efficient string matching algorithms Providing worst-case performance –Amenable to H/W implementation Aho-Corasick –Memory, performance

3. Introduction (i) Combating attacks at every level Automatically monitoring network traffic IDS uses a set of rules –Apply to matching packets Edge and core routers –Stringent worst-case performance bounds –Tight constraints on memory

4. Introduction (ii) At the heart of IDSs is a string matching algorithm –In Snort, 70% of total execution time and 80% of instructions executed Contributions of this paper –Characterization –New Algorithms –Evaluation

5. String matching for intrusion detection

6. Quantifying the Use of String Matching (i) Snort-An intrusion detection system –The rules are generated manually Extract relevant content strings from the payload and header of known attacks –The action can include logging, alerting, ignoring, …… –Rules are usually added as new vulnerabilities are discovered

7. Quantifying the Use of String Matching (ii) Scalability of the intrusion detection system database –Beneficial to avoid that has run-time proportional to the length of the rules in the database –New rules are being added to detect or combat new attacks

10. Quantifying the Use of String Matching (iii) Linearly searching through the of rules is becoming increasingly infeasible The database is growing at a rate that is well within Moore ’ s Law Need a technique with run-time performance

11. State of the Art in String Matching (i) Single-pattern string matching –Boyer-Moore, …… Multi-pattern string matching –Aho-Corasick, Wu-Manber, …… Imprecise string matching –Using hashing and signature-based –Be reverified using a precise string matching

12. State of the Art in String Matching (ii) Bad Character Heuristics –Easily exploitable by attackers Aho-Corasick –Use unoptimized data structure for space optimizations SFKSearch –Worst-case performance is quite poor Wu-Manber –Memory access to the shift and hash table

14. Applying IP Lookup Techniques to String Matching (i) IP-lookup ： a set of patterns to match, finding the longest possible match for a set of IP address that are streaming by String matching ： a set of strings to match, finding all of the places in the input stream where there is a match

15. Applying IP Lookup Techniques to String Matching (ii) Unibit and Multibit Tries –Wastes space with pointer Lulea Algorithm –Use the concepts of leaf pushing and bitmaps to compress the database Eatherton Algorithm –Internal bitmap and external bitmap

16. Optimizations for string matching

17. Bitmap compression (i) With 32-bits pointers In Aho-Corasick has 256 next state pointers Now using a single pointer to the first valid next state, and maintain a 256 bit bitmap Summing all the bits prior that bit number and adding them to the base next node pointer

19. Bitmap compression (ii) Original optimized Aho-Corasick –1028 bytes each node Bitmapped version –Only 44 bytes each node Incurs two costs –Doubles the worst-case of work –Performing a sum up to 256 prior bits

20. Path Compression (i) Bitmap is largely wasted information at the bottom nodes Any path compressed nodes must be equal in size to bitmapped nodes Failure pointers must include an offset

22. Path Compression (ii) On a 32 bits pointer –A single path compressed node can contain data equivalent to 4 bitmap compressed nodes –In practice, achieve a 2.54:1 compression ratio

24. Results

25. Instruction Detection in Hardware The number of rules go up by over a factor of 2.5, whereas the size of memory for our algorithm only goes up by 30% Focus our attention on the worst- case performance

27. Intrusion Detection in Software Examine both average-case and worst-case performance Wu-Manber is the fastest in the average-case because of hash function

30. Summary (i) Current software IDSs largely rely on common-case optimizations to gain speed Aho-Corasick is only has deterministic worst-case lookup times and friendly enough to use for wire speed H/W matching

31. Summary (ii) Contribution of this paper –Apply bitmap node compression and path compression to Aho-Corasick –Gain both compact storage and worst- case performance