1. 1 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Search Algorithms Winter Semester 2004/2005 29 Nov 2004 7th Lecture Christian Schindelhauer schindel@upb.de

2. Search Algorithms, WS 2004/05 2 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Chapter III Organization 29 Nov 2004 Mid Term Exam

3. Search Algorithms, WS 2004/05 3 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Mid Term Exam  Wednesday, 8 Dec 2004, 1pm-1.45pm, F1.110  4 parts –1. short questions, testing general understanding –2.-4. Show that you understand Text search algorithms Searching in Compressed Text The Pagerank algorithm  If you have successfully presented an exercise: –Only the best 3 of 4 parts count  If you fail, or if you receive a bad grade –then the oral exam at the end of the semester will cover the complete lecture  If you are happy with your grade –this grade counts half of the complete lecture –if you succeed within the oral exam (over the rest of the lecture)

4. Search Algorithms, WS 2004/05 4 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Chapter II Searching in Compressed Text 15 Nov 2004

5. Search Algorithms, WS 2004/05 5 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Searching in LZW-Codes Inside a node Example: Search for tapioca abtapiocaab blahblahabarb tapioca is “inside” a node Then we have found tapioca For all nodes u of a trie: Set: Is_inside[u]=1 if the text of u contains the pattern

6. Search Algorithms, WS 2004/05 6 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Searching in LZW-Codes Torn apart  Example: Search for tapioca carasi abrastapio Starting somewhere in a node Parts are hidden in some other nodes The end is the start of another node All parts are nodes of the LZW-Trie

7. Search Algorithms, WS 2004/05 7 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Finding the start: longest_prefix The Suffix of Nodes = Prefix of Patterns  Is the suffix of the node a prefix of the pattern –And if yes, how long is it? –Classify all nodes of the trie  For very long text encoded by a node only the last m letters matter  Can be computed using the KMP- Matcher-algorithm while building the Trie  Example: –Pattern: “manamana” pamana The last four letter are the first four of the pattern ma length of suffix of node which is prefix of patter is 2 papa result: 0 mana result: 4 amanaplanacanalpamana result: 4 amanaplanacanalpamana m amanaplanacanalpamanam

8. Search Algorithms, WS 2004/05 8 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Finding the End: longest_suffix Prefix of the Node = Suffix of the Pattern  Is the prefix of the node a suffix of the pattern –And if yes, does it complete the pattern, if already i letters were found? –Classify all nodes of the trie  For very long text encoded by a node only the first m letters matter  Since the text is added at the right side this property can be derived from the ancestor  Example: –Pattern: “manamana” ananimal Here 3 and 1 could be the solution We take 3, because 1 can be derived from 3 using the technique shown in KMP-Matcher (using  on the reverse string) manamanamana result: 8 panamacanal result: 0 manammanaaaaaaaaaaaa m manammanaaaaaaaaaaaam

9. Search Algorithms, WS 2004/05 9 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer How does it fit?  On the left side we have the maximum prefix of the pattern  On the right side we have the maximum suffix of the pattern panapamapama 10 letter pattern: pamapamapa pamapana 14 letters? Yet the pattern is inside, though, since the last 6 letters + the first 8 letters of the pattern give the pattern 8 letter prefix found 6 letter suffix found Solution: Define prefix-suffix-table PS-T(p,s) = 1 if p-letter prefix of P and s-letter suffix of P contain the pattern

10. Search Algorithms, WS 2004/05 10 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the PS-Table in time O(m 3 )  For all p and s such that p+s  m compute PS-T[p,s]  Run the KMP-Matcher for pattern P in P[m-p+1..m]P[1..s] – needs time O(m) for each combination of p and s  Leads to run time of O(n 3 ) xyzpamapamapamapaxyz 10 letter pattern: pamapamapa PS-T[8,6] = 1 If pattern pamapamapa found in text pamapamapamapa then

11. Search Algorithms, WS 2004/05 11 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table in Time O(m 2 ) - Preparation ptr[i,j] = next left position from i where the suffix of P of length j occurs = max{k < i | P[m-j+1..m] = P[k..k+j-1] or k = 0} pamapamapa pamapamapa pamapamapa pama p amapa pama p amapa

12. Search Algorithms, WS 2004/05 12 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table in time O(m 2 ) Initialization Init-ptr (P) 1.m  length(P) 2.for i  1 to m do ptr[i,0]  i-1 od 3.for j  1 to m-1 do 4. last  m-j+1 5. i  ptr[last+1,j-1]-1 6. while i  0 do 7. if P[i]=P[last] then 8. ptr[last,j]  i 9. last  i 10. fi 11. i  ptr[i+1,j-1]-1 12. od 13.od pamapamapa pamapamapa pamapamapa pama p amapa pama p amapa Run time: O(m 2 )

13. Search Algorithms, WS 2004/05 13 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table in time O(m 2 ) Init-PS-T(P) 1.m  length(P) 2.ptr  Init-ptr(P) 3.for i  1 to m-1 do 4. j  i+1 5. while j  0 do 6. PS-T[i,m-j+1]  1 7. j  ptr[j,m-i] 8. od 9.od pamapamapa pamapamapa ptr[9,2]ptr[5,2] PS-T[8,2]=1 PS-T[8,6]=1 PS-T[8,8]=1 Problem: What happens if the search pattern does not start at the beginning? This case is not covered, here!

14. Search Algorithms, WS 2004/05 14 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Counter-Example Search pattern: Init-PS-T(P) 1.m  length(P) 2.ptr  Init-ptr(P) 3.for i  1 to m-1 do 4. j  i+1 5. while j  0 do 6. PS-T[i,m-j+1]  1 7. j  ptr[j,m-i] 8. od 9.od Problem: What happens if the search pattern does not start at the beginning? This case is not covered, here! b a a b a a a b a b a a b a a a b a b a a b a a b a a a b a prefix length: 6suffix length: 6 b a a b a a a b a PS-T[6,6] = 1 PS-T[6,3] = 1 PS-T[6,7] = 1 Algorithm sets: PS-T[6,6] = 0

15. Search Algorithms, WS 2004/05 15 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Example of a Prefix-Suffix-Table suffices prefices babbabbbab abbabbabbabababbabbababbab b00000011 ba00000101 bab00001011 baba00010101 babab00101111 bababb01001011 bababba10110101 bababbab11111111 babab babbab bababbab babab abbab bababbab babab ababbab bababbab babab bab bababbab

16. Search Algorithms, WS 2004/05 16 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table PS-T (revisited) Pr[i] := max{ k | 1  k  m-i+1 and P[i,..,i+k-1] = P[1,..,k] } Length of the longest prefix starting at P[i] Su[j] := max{k| 1  k  m-j+1 and P[j-k+1,..,j]= [m-j+1,..,m]} Length of the longest suffix ending at P[j] Can be computed in time O(m 2 ) b a b a b b a b b a b b b Pr[1]=8 Pr[2]=0 Pr[3]=3 Pr[4]=0 Pr[5]=1 Pr[6]=3 Pr[7]=0 Pr[8]=1 b a b a b b a b b a b b Su[1]=1 Su[2]=0 Su[3]=3 Su[4]=0 Su[5]=3 Su[6]=1 Su[7]=0 Su[8]=8 b b a b

17. Search Algorithms, WS 2004/05 17 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table PS-T (revisited) Pr[i] := length of the longest prefix starting at P[i] Su[j] := length of the longest suffix ending at P[j] For all i  {1,..,m+1},j  {0,..,m} do If Pr[i]+Su[j]  m then s  i+m-Su[j]-1 /* =i-1+Pr[i]-(Pr[i]+Su[j]-m) */ for k  0 to Su[j]+Pr[i]-m do PS-T[s+k,m-j+Su[j]-k] =1 od fi od Problem: Running time: O(n 3 ) b a b a b b a b b a b Pr[3]=3 b a b a b b a b Su[8]=8 b ab a b a b b a b b a ba b a b b a b b a b a b a b b a b a b b a b PS-T[2,8]=1 PS-T[3,7]=1 PS-T[4,6]=1 PS-T[5,5]=1 Overlapping: Pr[3]+Su[8]-8=3 Leads to 3+1=4 entries in PS-T:

18. Search Algorithms, WS 2004/05 18 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table PS-T (revisited) Pr[i] := length of the longest prefix starting at P[i] Su[j] := length of the longest suffix ending at P[j] For all i  {1,..,m+1},j  {0,..,m} do If Pr[i]+Su[j]  m then s  i+m-Su[j]-1 /* =i-1+Pr[i]-(Pr[i]+Su[j]-m) */ for k  0 to Su[j]+Pr[i]-m do PS-T[s+k,m-j+Su[j]-k] =1 od fi od Problem: Running time: O(n 3 ) Overlap: Pr[i]+Su[j]-m Prefix Pr[i] Suffix Su[j] 1 m Pattern i i+Pr[i] i+m -overlap 1 m j j-Su[j] m-j+Su[j]

19. Search Algorithms, WS 2004/05 19 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Example of a Prefix-Suffix-Table Pr Su -babbabbbab abbab babbabababbab bababbab Pr[9]=0Pr[8]=1 Pr[7]= 0 Pr[6]= 3 Pr[5]= 1 Pr[4]=0Pr[3]=3Pr[2]=0Pr[1]=8 -Su[0]=0 -------- [8,0] bSu[1]=1 -------- [7,2]... [8,0] baSu[2]=0 -------- [8,2] babSu[3]=3 -------- [5,6].. [8,3] babaSu[4]=0 -------- [8,4] bababSu[5]=3 -------- [5,8].. [8,5] bababb Su[6]=1 -------- [7,7],... [8,5] bababba Su[7]=0 -------- [8,7] bababbab Su[8]=8[8,8][7,8][6,8] [5,8],.., [8,5] [4,8], [5,7] [3,8] [2,8],.., [5,5] [1,8] [0,8],.., [8,0] bababbab -----++++++++ ------+++++++ -------++++++ --------+++++ bababbab -----++++++ ------+++++ -------++++ --------+++

20. Search Algorithms, WS 2004/05 20 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Example of a Prefix-Suffix-Table Pr Su -babbabbbab abbab babbabababbab bababbab Pr[9]=0Pr[8]=1 Pr[7]= 0 Pr[6]= 3 Pr[5]= 1 Pr[4]=0Pr[3]=3Pr[2]=0Pr[1]=8 -Su[0]=0 -------- [8,0] bSu[1]=1 -------- [7,1]... [8,0] baSu[2]=0 -------- [8,2] babSu[3]=3 -------- [5,3].. [8,0] babaSu[4]=0 -------- [8,4] bababSu[5]=3 -------- [5,5].. [8,2] bababb Su[6]=1 -------- [7,6],... [8,5] bababba Su[7]=0 -------- [8,7] bababbab Su[8]=8[8,8][7,8][6,8] [5,8],.., [8,5] [4,8], [5,7] [3,8] [2,8],.., [5,5] [1,8] [0,8],.., [8,0] 4

21. Search Algorithms, WS 2004/05 21 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Computing the Prefix-Suffix-Table PS-T in time O(n2) O(n 3 )-algorithm: PS-T[0..m,0..m]  0 For all i  {1,..,m+1},j  {0,..,m} do If Pr[i]+Su[j]  m then s  i+m-Su[j]-1 /* =i-1+Pr[i]-(Pr[i]+Su[j]-m) */ for k  0 to Su[j]+Pr[i]-m do PS-T[s+k,j-k] =1 od fi od PS-T[0..m,0..m]  -1 For all i  {1,..,m+1},j  {0,..,m} do s  i+m-Su[j]-1 PS-T[s+k,j-k] = max{ PS-T[s+k,j-k],Su[j]+Pr[i]-m} od for d  0 to 2m do z  -1 for i  0 to min{d,2m-d} do j  d-i z  max{z,PS-T[i,j]} if z  0 then PS-T[i,j]  1 z  z-1 else PS-T[i,j]  0 fi od Run time O(n 2 ) Mark diagonals with length of series with 1s For all diagonals Fill up the diagonals

22. Search Algorithms, WS 2004/05 22 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Chapter III Searching the Web 29 Nov 2004

23. Search Algorithms, WS 2004/05 23 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Searching the Web  Introduction  The Anatomy of a Search Engine  Google’s Pagerank algorithm –The Simple Algorithm –Periodicity and convergence  Kleinberg’s HITS algorithm –The algorithm –Convergence  The Structure of the Web –Pareto distributions –Search in Pareto-distributed graphs

24. Search Algorithms, WS 2004/05 24 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The Anatomy of a Web Search Engine  “The Anatomy of a Large-Scale Hypertextual Web Search Engine”, Sergey Brin and Lawrence Page, Computer Networks and ISDN Systems, Vol. 30, 1-6, p. 107-117, 1998  Design of the prototype –Stanford University 1998  Key components: –Web Crawler –Indexer –Pagerank –Searcher  Main difference between Google and other search engines (in 1998) –The Pagerank mechanism

25. Search Algorithms, WS 2004/05 25 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Simplified PageRank-Algorithmus  Simplified PageRank-Algorithmus –Rank of a wep-page R(u)  [0,1] –Important pages hand their rank down to the pages they link to. –c is a normalisation factor such that ||R(u)|| 1 = 1, i.e. the sum of all page ranks add to 1 –Predecessor nodes B u –sucessor nodes F u

26. Search Algorithms, WS 2004/05 26 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The Simplifed Pagerank Algorithm and an example

27. Search Algorithms, WS 2004/05 27 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Matrix representaion R  c M R, where R is a vector (R(1),R(2),… R(n)) and M denotes the following n  n – Matrix

28. Search Algorithms, WS 2004/05 28 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The Simplified Pagerank Algorithm  Does it converge?  If it converges, does it converge to a single result?  Is the result reasonable?

29. Search Algorithms, WS 2004/05 29 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The Eigenvector and Eigenvalue of the Matrix  For vector x and n  n-matrix and a number λ: –If M x = λ x then x is called the eigenvector and λ the eigen-value  Every n  n-matrix M has at most n eigenvalues  Compute the eigenvalues by eigen-decomposition M x = λ x  (M - I λ) x = 0, where I is the identity matrix –This equality has only non-trivial solutions if Det(M - I λ) = 0 –This leads to a polynomial equation of degree n, which has always n solutions λ 1, λ 2,..., λ n (Fundamental theorem of algebra) –Solving the linear equations (M - I λ i ) x = 0 lead to the eigenvectors  The eigenvektor of the matrix is a fix point of the recursion of the simplified pagerank algorithm

30. Search Algorithms, WS 2004/05 30 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer  Consider n discrete states and a sequence of random variable X 1, X 2,... over this set of states  The sequence X 1, X 2,... is a Markov chain if  A stochastic matrix M is the transition matrix for a finite Markov chain, also called a Markov matrix: –Elements of the matrix M must be real numbers of [0, 1]. –The sum of all column in M is 1  Observation for the matrix M of the simpl. pagerank algorithm –M is stochastic if all nodes have at least one outgoing link Stochastic Matrices

31. Search Algorithms, WS 2004/05 31 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The Random Surfer  Consider the following algorithm –Start in a random web-page according to a probability distribution –Repeat the following for t rounds If no link is on this page, exit and produce no output Uniformly and randomly choose a link of the web-page Follow that link and go to this web-page –Output the web-page Lemma The probability that a web-page i is output by the random surfer after t rounds started with probability distribution x 1,.., x n is described by the i-th entry of the output of the simplified Pagerank-algorithm iterated for t rounds without normalization. Proof follows applying the definition of Markov chains

32. Search Algorithms, WS 2004/05 32 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Eigenvalues of Stochastic Matrices  Notations –Die L1-Norm of a vector x is defined as –x  0, if for all i: x i  0 –x  0, if for all i: x i  0  Lemma For every stochastic matrix M and every vector x we have || M x || 1  || x || 1 || M x || 1 = || x || 1, if x  0 or x  0  Eigenvalues of M | i |   1  Theorem For every stochastic matrix M there is an eigenvector x with eigenvalue 1 such that x  0 and ||x|| 1 = 1

33. Search Algorithms, WS 2004/05 33 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Eigenvalues of Stochastic Matrices  Lemma For every stochastic matrix M and every vector x we have || M x || 1  || x || 1 || M x || 1 = || x || 1, if x  0 or x  0  Proof (x  0 or x  0)

34. Search Algorithms, WS 2004/05 34 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Eigenvalues of Stochastic Matrices  Lemma For every stochastic matrix M and every vector x we have || M x || 1  || x || 1 || M x || 1 = || x || 1, if x  0 or x  0  Proof –Decompose x in two vectors p  0 and n  0 with x = p + n –Use triangle inequality |a+b|  |a| + |b| for all metrics

35. Search Algorithms, WS 2004/05 35 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The problem of periodicity - Example

36. Search Algorithms, WS 2004/05 36 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Periodicity - Example 2

37. Search Algorithms, WS 2004/05 37 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Periodic Matrices  Definition –A square matrix M such that the matrix power M k =M for k a positive integer is called a periodic matrix. –If k is the least such integer, then the matrix is said to have period k-1. –If the period is 1, then M 2 = M and M is called idempotent.  Fact –For non-periodic matrices there are vectors x, such that lim k  M k x does not converge.  Definition –The directed graph G=(V,E) of a n x n-matrix consistis of the node set V={1,..., n} and has edges E = {(i,j) | M ij  0} –A path is a sequence of edges (u 1,u 2 ),(u 2,u 3 ),(u 3,u 4 ),..,(u t,u t+1 ) of a graph –A graph cycle is a path where the start node is the end node –A strongly connected subgraph S is a minimum sub-graph such that every graph cycle starting and ending in a node of S is contained in S.

38. Search Algorithms, WS 2004/05 38 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Necessary and Sufficient Conditions for Periodicity  Theorem (necessary condition) –If the stochastic matrix M is periodic with periodicity t  2, then for the graph G of M there exists a strongly connected subgraph S of at least two nodes such that every directed graph cycle within S has a length of the form i t for natural number i.  Theorem (sufficient condition) –Let the graph consist of one strongly connected subgraph and –let L 1,L 2,..., L m be the lengths all directed graph cycles of maximal length n –Then M is non-periodic if and only if gcd(L 1,L 2,..., L m ) = 1  Notation: –gcd(L 1,L 2,..., L m ) = greatest common divisor of numbers L 1,L 2,..., L m  Corollary –If the graph is strongly connected and there exists a graph cycle of length 1 (i.e. a loop), then M is non-periodic.

39. Search Algorithms, WS 2004/05 39 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Disadvantages of the Simplified Pagerank- Algorithm  The Web-graph has sinks, i.e. pages without links  M is not a stochastic matrix  The Web-graph is periodic  Convergence is uncertain  The Web-graph is not strongly connected  Several convergence vectors possible  Rank-sinks –Strongly connected subgraphs absorb all weight of the predecessors –All predecessors pointing to a web-page loose their weight.

40. Search Algorithms, WS 2004/05 40 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer The (non-simplified) Pagerank-Algorithm  Add to a sink links to all web-pages  Uniformly and randomly choose a web-page –With some probability q < 1 perform a step of the simplified Pagerank algorithm –With probability 1-q start with the first step (and choose a random web-page)  Note M ist stochastic

41. Search Algorithms, WS 2004/05 41 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Properties of the Pagerank-Algorithm  Graph of the matrix is strongly connected  There are graph cycles of length 1 Theorem In non-periodic matrices of strongly connected graphs the Markov-chain converges to a unique eigenvector with eigenvalue 1.  PageRank converges to this unique eigenvector

42. 42 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Christian Schindelhauer Thanks for your attention End of 7th lecture Next lecture:Mo 29 Nov 2004, 11.15 am, FU 116 Next exercise class: Mo 29 Nov 2004, 1.15 pm, F0.530 or We 01 Dec 2004, 1.00 pm, E2.316