1 FPGA-based ROM-free network intrusion detection using shift-OR circuit Department of Computer Science and Information Engineering National Cheng Kung University, Taiwan R.O.C. Authors : Wen-Jyi Hwang, Huang-Chun Roan, Ying-Nan Shih, Chia-Tien Dan Lo and Chien-Min Ou Publisher : Journal of Embedded Computing Present : Chen- Rong Chang Date : November, 18, 2009

OUTLINE Preliminaries  shift-or algorithm The architecture  Basic module circuit  Module circuit based on bitmap encoder  High throughput module circuit Experimental results and comparisons 2

Shift-Or algorithm(1/3) 3 Cycle 0 : R0R R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. aabaab An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P S

Shift-Or algorithm(1/3) 4 Cycle 1 : R0R ScSc 100 Input a R1R1 R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 5 Cycle 2 : R0R ScSc 100 Input a R1R1 ScSc 100 Input a R2R2 R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. match prefix “aa” of P An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P 6 Cycle 3 : R0R ScSc 100 Input a R1R1 ScSc 100 Input c R2R2 R2R2 ScSc R3R3 Input b R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. match match sub-pattern “aa” of P

Shift-Or algorithm(2/3)

Shift-Or algorithm(1/3) 8 Cycle 0 : R0R R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 9 Cycle 1 : R0R ScSc 100 Input a R1R1 R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 10 Cycle 2 : R0R ScSc 100 Input a R1R1 ScSc 111 Input c R2R2 R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 11 Cycle 3 : R0R ScSc 100 Input a R1R1 ScSc 111 Input c R2R2 R2R2 ScSc R3R3 Input a R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 12 Cycle 4 : R0R ScSc 100 Input a R1R1 ScSc 111 Input c R2R2 R2R2 ScSc R3R3 Input a ScSc R4R4 Input a R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Shift-Or algorithm(1/3) 13 Cycle 5 : R0R ScSc 100 Input a R1R1 ScSc 111 Input c R2R2 R2R2 ScSc R3R3 Input a ScSc R4R4 Input a 1000R4R4 ScSc R5R5 Input a match R j+1 [i] = ( R j [i] | S c [i] ) <<1, i=1,…,m. An example of shift-or algorithm with pattern P = aab and text T = aab, The bit vector S associated with each symbol s c ∈ Σ = {a, b, c } for the pattern P

Basic module circuit(1/2) The basic circuit of each module for exact pattern matching, (a) The block diagram of the circuit, (b) The shift register circuit during clock cycle j

Basic module circuit(1/2) The basic circuit of each module for exact pattern matching, (a) The block diagram of the circuit, (b) The shift register circuit during clock cycle j + 1. scsc abcde… i … … … Pattern: aabc =4 256 symbols 15

Basic module circuit(2/2) scsc abcother i Pattern: aabc 2 =4 Fig. 4. The augment of a symbol encoder for reducing the ROM size. In this example, each input character is assumed to be an ASCII code (8 bits). We uses only 4 symbols in the alphabet. The output of the symbol encoder therefore is 2 bits. 16

Module circuit based on bitmap encoder(1/5) Therefore, the ROM implemented by embedded memory bits may become the bottleneck of the systems’s throughput. In addition, the same ROM cannot be shared by different rules. The consumption of embedded memory bits will be high for the circuits containing large number of Snort rules. 17

Module circuit based on bitmap encoder(2/5) Fig. 7. The increase of a symbol encoder for reducing the bitmap encoder size. In this example, each input character is assumed to be an ASCII code (8 bits). We uses only 7 symbols in the alphabet. The output of the symbol encoder is 3 bits. 18

Module circuit based on bitmap encoder(3/5) Fig.5 A simple example of the proposed circuit for the pattern aadc and the total symbol a, b, c, d, (a)The architecture (b)Table of the pattern. 19

Module circuit based on bitmap encoder(4/5) Fig.6 An example of three patterns (aadc, bdd and ddac) share the same bitmap encoder, (a) The architecture (b) Table of three patterns 20

Module circuit based on bitmap encoder(5/5) The sharing of the same symbol encoder and bitmap encoder by three different Snort rules. Each character is also assumed to be an ASCII. All the Snort rules use the same alphabet comprised of 7 symbols. 21

High throughput module circuit scsc *aabcdother i Pattern: aabcd Payload: 123aabcd scsc aabcd*other i Bitmap Encoder 1 Bitmap Encoder

Experimental results and comparisons(1/3) The performance of the ROM-based and bitmap encoding circuit with q = 1 for various rule sets sizes ranging from 500 characters to 8000 characters (a) LE per character (b) Operating Frequency. 23

Experimental results and comparisons(2/3) 24

Experimental results and comparisons(3/3) 25

Shift-And Algorithm The shift-or algorithm is a tricky implementation of shift-and. The idea is to avoid using the “0 m -1”mask of formula in order to speed up the computation. R j+1 [i] = (R j [i]<<1 | 0 m-1 1) & S c [i], i=1,…,m. 26 Shift-and algorithm formula: R j+1 [i] = R j [i]<<1 | S c [i], i=1,…,m. Shift-or algorithm formula:

Shift-Or algorithm(1/3) Let R j be a bit vector containing information about all matches of the prefixes of P that end at j. The vector contains m + 1 elements Rj [i], i = 0,...,m, where Rj [i] = 0 if the first i characters of the pattern P match exactly the last i characters up to j in the text (i.e., p 1 p 2...p i = t j−i+1 t j−i+2... t j ). The transition from R j to R j+1 is performed by the recurrence: where the initial conditions for the recurrence are given by R 0 [i] = 1, i = 1,...,m, and R j [0] = 0, j = 0,...,m. The recurrence can be implemented by the simple shift and OR operations.

Shift-Or algorithm(2/3) Suppose P =p 1 p 2...p m is a pattern to be searched inside a large text (or source) T = t 1 t 2... t n, where n>>m. Every character of P and T belongs to the same alphabet Σ = {s 1,..., s |Σ| }. Let R j be a bit vector containing information about all matches of the prefixes of P that end at j. The formula shows in follow: 28 The initial value: R j = 1 m-1 0, EX: R j =