Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 1 Chapter 5.2 String Searching - Part 2 Boyer-Moore Algorithm Rabin-Karp Algorithm

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 2 The Boyer-Moore String Algorithm lThis method can give substantially faster searches where the language contains a large number of symbols lE.g. Normal text (128 or 256 character alphabet) rather than binary strings lBM method incorporates two main ideas lstart matching at the right of the pattern so as to find the rightmost mismatch luse information about the possible alphabet of the text, as well as the characters in the pattern

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 3 Example Search for LEAN in CARPETS NEED CLEANING REGULARLY CARPETS NEED CLEANING REGULARLY LEAN N and P mismatch. Furthermore, P does not occur anywhere in the string LEAN. Hence move string all the way past P and compare with N again. CARPETS NEED CLEANING REGULARLY LEAN N and E mismatch, but E occurs in LEAN, so we move the E of LEAN to this position

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 4 Boyer-Moore preprocessing lIn order to implement the above idea, consider the characters in the alphabet which makes up the text. lC 0,C 1,…,C k (k+1 characters in the alphabet) Initialise an array skip such that for each C j in the pattern string set skip[j] to the distance of C j from the right hand end of the pattern skip[j] = M otherwise, where M is the length of the pattern.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 5 The skip array - example lSuppose pattern is LEAN and alphabet is, A,B,…,Z (C 0,C 1,…,C 26 ). lskip[12] = 3 (L) lskip[5] = 2 (E) lskip[1] = 1 (A) lskip[14] = 0 (N) lskip[X] = 4 (otherwise) skip[C] is the number of characters to move the pattern to the right after a mismatch in the text with character with index C

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 6 Using the skip array lTry to match the pattern from right to left Mismatch occurs between C n with index n and the (M-j) th position of the pattern. Get value of skip[n] If ( M-j) > skip[n] then shift pattern by 1 (since we have already passed the rightmost occurrence of C n in the pattern). Else shift pattern skip[n]-j positions, to try to align C n in the text with the rightmost occurrence of C n in the pattern.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 7 Example - shifting using skip Pattern X X A X X X Z Z Z Z M = 10 (length of pattern) skip[1] = 7 (distance of rightmost A from right) mismatch at position 10-4 Y Y Y Y Y A Z Z Z Z Z Z Z Z Z text X X A X X X Z Z Z Z mismatched pattern X X A X X X Z Z Z Z shift 3 positions Shift pattern by 7-4 = 3 positions

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 8 Boyer-Moore Algorithm (1) int boyermoore1(String P, String T){ int 1,j,t,M=P.length(),N=T.length(); initskip(P); // initialise skip array i = M-1; j = M-1; while (j > 0){ while (T[i] != P[j]){ t = skip[index(T[i])]; if ((M-j)>t) {i=i+M-j;} else {i=i+t;} if (I >= N) return N; // no match j = M-1;} i--; j--;} return i; } // successful match

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 9 Refinement to B-M Algorithm lWe can apply the KMP algorithm “right-to-left” Sometimes this gives a larger skip value than the skip index used above lE.g. Pattern BBAAA lskip[1] = 0 (skip value for A) lskip[2] = 3 (skip value for B) AAAAAAA BBAAA mismatch on A in text boyermoore1 algorithm shifts only one position However it’s clear that AAA does not occur anywhere to the left of positions 3,4,5

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 10 Boyer-Moore Refinement (2) Build KMP next array from right to left j = position of mismatch (from right) next[j] = no. of positions to shift pattern to right jnext[j]BBAAA 21 BBAAA 31 BBAAA 45 BBAAA 5 5 BBAAA Using the next array, a mismatch on B results in a shift of 5 positions

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 11 Refined Boyer-Moore Algorithm Initialise both the skip and the next arrays (right-to-left). Whenever a mismatch occurs, get the skip value for the mismatched character and the next value for the position of the mismatch. lShift the pattern right by whichever gives the greater value.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 12 Rabin-Karp String Matching lConsider a text and pattern consisting of characters represented by b bits each le.g. 7-bit ASCII characters lWe can regard a sequence of characters as a (large) binary number (as with keys when using hash tables) lIdea - compute a hash value for an M - character pattern and compare it successively with the hash values of each successive sequence of M characters in the text.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 13 Rabin-Karp matching - basic idea lExample. Consider the string CARPETS NEED CLEANING and the search string LEAN. lThen we compare h(LEAN) first against h(CARP), then against h(ARPE), h(RPET), h(PETS), and so on. lClearly h(LEAN) need be computed only once. lThe key to efficient comparison is to compute the successive hash values efficiently. lWe can exploit the fact that successive keys overlap, e.g. ARPE and RPET share 3 characters.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 14 Rabin-Karp - computing hash values lLet us use h(K) = K mod P as our hash function as before, where P is a large prime number lLet d = max number of characters (e.g. d=2 b ) lSuppose K = C 1,…,C n where C 1,…,C n is a sequence of characters in the text, and h(K) = X lIt can be shown that h( C 2,…,C n+1 )= h((X  C 1 *d n-1 )*d + C n+1 ), since lC 2,…,C n+1 can be rewritten as (C 1,…,C n - C 1 * d n-1 )*d + C n+1 ) lE.g. (d=10): = ( (3*10 4 ))* lThen use some properties such as h(X+Y) = h(h(X) +Y) and h(X*Y) = h(h(X) * Y) lHence h(45678) = h((h(34567) - (3*10 4 ))* lThus, successive values for h are efficiently computed, since we can reuse the previous has value to compute the next one.

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 15 Rabin-Karp Algorithm int rabinkarp(String P, String T){ int q= // a large prime int d=32 // size of alphabet int i,dM=1, h1=0, h2=0; int M=P.length(), N=T.length(); for (i=0;i<M;i++){dM=(d*dM) mod q;} for (i=0;i<M;i++){ h1=(h1*d+val(P[i])) mod q; // hash P h2=(h2*d+val(T[i])) mod q; } for (i=0; h1 != h2; i++){ h2=(h2+d*q-val(T[i]))*dM) mod q; h2=(h2*d + val(a[i+M])) mod q; if (i > N-M) return N;} \\ not found return i; }

Dept of Computer Science, University of Bristol. COMS Chapter 5.2 Slide 16 Rabin-Karp - analysis lIn the above algorithm, val(P[i]) is the number corresponding to the character P[i]. lh1 is the hash value of the pattern lh2 takes the hash value of successive sequences of M characters in the text. lStrictly, if h1=h2, we might not have a match, since a hash collision could occur. We still need to make a final comparison on the strings themselves. lWe can use a very large prime since we do not actually have to store the hash table; this makes collisions extremely unlikely. lAverage number of comparisons = N+M