1. CSC401 -- Analysis of Algorithms 3-1 CSC401 – Analysis of Algorithms Chapter 3 Search Trees and Skip Lists Objectives: Review binary search trees and present operations on binary search trees Analyze the performance of binary search tree operations Review balanced binary search trees: AVL tree and Red-Black tree, and multiway search trees Introduce splay trees and skip lists Analyze the performance of splay trees and skip lists

2. CSC401 -- Analysis of Algorithms 3-2 Binary Search Tree A binary search tree is a binary tree storing keys (or key-element pairs) at its internal nodes and satisfying the following property: –Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u)  key(v)  key(w) External nodes do not store items An inorder traversal of a binary search trees visits the keys in increasing order 6 92 418

3. CSC401 -- Analysis of Algorithms 3-3 Search To search for a key k, we trace a downward path starting at the root The next node visited depends on the outcome of the comparison of k with the key of the current node If we reach a leaf, the key is not found and we return NO_SUCH_KEY Example: findElement(4) Algorithm findElement(k, v) if T.isExternal (v) return NO_SUCH_KEY if k  key(v) return findElement(k, T.leftChild(v)) else if k  key(v) return element(v) else { k  key(v) } return findElement(k, T.rightChild(v)) 6 9 2 4 1 8   

4. CSC401 -- Analysis of Algorithms 3-4 Insertion To perform operation insertItem(k, o), we search for key k Assume k is not already in the tree, and let let w be the leaf reached by the search We insert k at node w and expand w into an internal node Example: insert 5 6 9 2 4 18    w 6 92 418 5 w

5. CSC401 -- Analysis of Algorithms 3-5 Deletion To perform operation removeElement( k ), we search for key k Assume key k is in the tree, and let let v be the node storing k If node v has a leaf child w, we remove v and w from the tree with operation removeAboveExternal( w ) Example: remove 4 6 9 2 5 18 6 9 2 4 18 5 v w  

6. CSC401 -- Analysis of Algorithms 3-6 Deletion (cont.) We consider the case where the key k to be removed is stored at a node v whose children are both internal –we find the internal node w that follows v in an inorder traversal –we copy key(w) into node v –we remove node w and its left child z (which must be a leaf) by means of operation removeAboveExternal( z ) Example: remove 3 3 1 8 6 9 5 v w z 2 5 1 8 6 9 v 2

7. CSC401 -- Analysis of Algorithms 3-7 Performance Consider a dictionary with n items implemented by means of a binary search tree of height h –the space used is O(n) –methods findElement, insertItem and removeElement take O(h) time The height h is O(n) in the worst case and O(log n) in the best case

8. CSC401 -- Analysis of Algorithms 3-8 AVL Tree Definition AVL trees are balanced. An AVL Tree is a binary search tree such that for every internal node v of T, the heights of the children of v can differ by at most 1. An example of an AVL tree where the heights are shown next to the nodes

9. CSC401 -- Analysis of Algorithms 3-9 Height of an AVL Tree Fact: The height of an AVL tree storing n keys is O(log n). storing n keys is O(log n). Proof: Let us bound n(h): the minimum number of internal nodes of an AVL tree of height h. We easily see that n(1) = 1 and n(2) = 2 For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of height n-2. That is, n(h) = 1 + n(h-1) + n(h-2) Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). So n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by induction), n(h) > 2 i n(h-2i) Solving the base case we get: n(h) > 2 h/2-1 Taking logarithms: h < 2log n(h) +2 Thus the height of an AVL tree is O(log n) 3 4 n(1) n(2)

10. CSC401 -- Analysis of Algorithms 3-10 Insertion in an AVL Tree Insertion is as in a binary search tree Always done by expanding an external node. Example: 44 1778 325088 4862 54 w b=x a=y c=z 44 1778 325088 4862 before insertionafter insertion

11. CSC401 -- Analysis of Algorithms 3-11 Trinode Restructuring let (a,b,c) be an inorder listing of x, y, z perform the rotations needed to make b the topmost node of the three b=y a=z c=x T0T0 T1T1 T2T2 T3T3 b=y a=z c=x T0T0 T1T1 T2T2 T3T3 c=y b=x a=z T0T0 T1T1 T2T2 T3T3 b=x c=ya=z T0T0 T1T1 T2T2 T3T3 case 1: single rotation (a left rotation about a) case 2: double rotation (a right rotation about c, then a left rotation about a) (other two cases are symmetrical)

12. CSC401 -- Analysis of Algorithms 3-12 Insertion Example, continued

13. CSC401 -- Analysis of Algorithms 3-13 Restructuring (Single Rotations) Single Rotations:

14. CSC401 -- Analysis of Algorithms 3-14 Restructuring (Double Rotations) double rotations:

15. CSC401 -- Analysis of Algorithms 3-15 Removal in an AVL Tree Removal begins as in a binary search tree, which means the node removed will become an empty external node. Its parent, w, may cause an imbalance. Example: 44 17 783250 88 48 62 54 44 17 7850 88 48 62 54 before deletion of 32after deletion

16. CSC401 -- Analysis of Algorithms 3-16 Rebalancing after a Removal Let z be the first unbalanced node encountered while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of y with the larger height. We perform restructure(x) to restore balance at z. As this restructuring may upset the balance of another node higher in the tree, we must continue checking for balance until the root of T is reached 44 17 7850 88 48 62 54 w c=x b=y a=z 44 17 78 5088 48 62 54

17. CSC401 -- Analysis of Algorithms 3-17 Running Times for AVL Trees a single restructure is O(1) –using a linked-structure binary tree find is O(log n) –height of tree is O(log n), no restructures needed insert is O(log n) –initial find is O(log n) –Restructuring up the tree, maintaining heights is O(log n) remove is O(log n) –initial find is O(log n) –Restructuring up the tree, maintaining heights is O(log n)

18. CSC401 -- Analysis of Algorithms 3-18 Red-Black Tree A red-black tree can also be defined as a binary search tree that satisfies the following properties: –Root Property: the root is black –External Property: every leaf is black –Internal Property: the children of a red node are black –Depth Property: all the leaves have the same black depth 9 15 4 62 12 7 21 Theorem: A red-black tree storing n items has height O(log n) –The height of a red- black tree is at most twice the height of its associated (2,4) tree, which is O(log n) The search algorithm for a binary search tree is the same as that for a binary search tree By the above theorem, searching in a red-black tree takes O(log n) time

19. CSC401 -- Analysis of Algorithms 3-19 Insertion To perform operation insertItem (k, o), we execute the insertion algorithm for binary search trees and color red the newly inserted node z unless it is the root –We preserve the root, external, and depth properties –If the parent v of z is black, we also preserve the internal property and we are done –Else ( v is red ) we have a double red (i.e., a violation of the internal property), which requires a reorganization of the tree Example where the insertion of 4 causes a double red: 6 3 8 6 3 8 4 z vv z

20. CSC401 -- Analysis of Algorithms 3-20 Remedying a Double Red Consider a double red with child z and parent v, and let w be the sibling of v 4 6 7 z vw 2 4 6 7.. 2.. Case 1: w is black –The double red is an incorrect replacement of a 4-node –Restructuring: we change the 4-node replacement Case 2: w is red –The double red corresponds to an overflow –Recoloring: we perform the equivalent of a split 4 6 7 z v 2 4 6 7 2 w

21. CSC401 -- Analysis of Algorithms 3-21 Restructuring A restructuring remedies a child-parent double red when the parent red node has a black sibling It is equivalent to restoring the correct replacement of a 4-node The internal property is restored and the other properties are preserved 4 6 7 z v w 2 4 6 7.. 2.. 4 6 7 z v w 2 4 6 7.. 2..

22. CSC401 -- Analysis of Algorithms 3-22 Restructuring (cont.) There are four restructuring configurations depending on whether the double red nodes are left or right children 2 4 6 6 2 4 6 4 2 2 6 4 2 6 4

23. CSC401 -- Analysis of Algorithms 3-23 Recoloring A recoloring remedies a child-parent double red when the parent red node has a red sibling The parent v and its sibling w become black and the grandparent u becomes red, unless it is the root It is equivalent to performing a split on a 5-node The double red violation may propagate to the grandparent u 4 6 7 z v 2 4 6 7 2 w 4 6 7 z v 6 7 2 w … 4 … 2

24. CSC401 -- Analysis of Algorithms 3-24 Analysis of Insertion Recall that a red-black tree has O(log n) height Step 1 takes O(log n) time because we visit O(log n) nodes Step 2 takes O(1) time Step 3 takes O(log n) time because we perform –O(log n) recolorings, each taking O(1) time, and –at most one restructuring taking O(1) time Thus, an insertion in a red-black tree takes O(log n) time Algorithm insertItem(k, o) 1.We search for key k to locate the insertion node z 2.We add the new item (k, o) at node z and color z red 3. while doubleRed(z) if isBlack(sibling(parent(z))) z  restructure(z) z  restructure(z)return else { sibling(parent(z) is red } z  recolor(z) z  recolor(z)

25. CSC401 -- Analysis of Algorithms 3-25 Deletion To perform operation remove (k), we first execute the deletion algorithm for binary search trees Let v be the internal node removed, w the external node removed, and r the sibling of w –If either v of r was red, we color r black and we are done –Else ( v and r were both black) we color r double black, which is a violation of the internal property requiring a reorganization of the tree Example where the deletion of 8 causes a double black: 6 3 8 4 v rw 6 3 4 r

26. CSC401 -- Analysis of Algorithms 3-26 Remedying a Double Black The algorithm for remedying a double black node w with sibling y considers three cases Case 1: y is black and has a red child –We perform a restructuring, equivalent to a transfer, and we are done Case 2: y is black and its children are both black –We perform a recoloring, equivalent to a fusion, which may propagate up the double black violation Case 3: y is red –We perform an adjustment, equivalent to choosing a different representation of a 3-node, after which either Case 1 or Case 2 applies Deletion in a red-black tree takes O(log n) time

27. CSC401 -- Analysis of Algorithms 3-27 Red-Black Tree Reorganization remedy double red Insertion double red removed or propagated up splitrecoloring double red removed change of 4-node representation restructuring result (2,4) tree action Red-black tree action remedy double black Deletion restructuring or recoloring follows change of 3-node representation adjustment double black removed or propagated up fusionrecoloring double black removed transferrestructuring result (2,4) tree action Red-black tree action

28. CSC401 -- Analysis of Algorithms 3-28 Multi-Way Search Tree A multi-way search tree is an ordered tree such that –Each internal node has at least two children and stores d  1 key-element items (k i, o i ), where d is the number of children –For a node with children v 1 v 2 … v d storing keys k 1 k 2 … k d  1 keys in the subtree of v 1 are less than k 1 keys in the subtree of v i are between k i  1 and k i (i = 2, …, d  1) keys in the subtree of v d are greater than k d  1 –The leaves store no items and serve as placeholders 11 24 2 6 815 30 27 32

29. CSC401 -- Analysis of Algorithms 3-29 Multi-Way Inorder Traversal We can extend the notion of inorder traversal from binary trees to multi-way search trees Namely, we visit item (k i, o i ) of node v between the recursive traversals of the subtrees of v rooted at children v i and v i    1 An inorder traversal of a multi-way search tree visits the keys in increasing order 11 24 2 6 815 30 27 32 13579111319 1517 2461418 812 10 16

30. CSC401 -- Analysis of Algorithms 3-30 Multi-Way Searching Similar to search in a binary search tree A each internal node with children v 1 v 2 … v d and keys k 1 k 2 … k d  1 –k  k i (i = 1, …, d  1) : the search terminates successfully –k  k 1 : we continue the search in child v 1 –k i  1  k  k i (i = 2, …, d  1) : we continue the search in child v i –k  k d  1 : we continue the search in child v d Reaching an external node terminates the search unsuccessfully Example: search for 30 11 24 2 6 815 30 27 32

31. CSC401 -- Analysis of Algorithms 3-31 (2,4) Tree A (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi- way search with the following properties –Node-Size Property: every internal node has at most four children –Depth Property: all the external nodes have the same depth Depending on the number of children, an internal node of a (2,4) tree is called a 2-node, 3-node or 4-node 10 15 24 2 81227 3218

32. CSC401 -- Analysis of Algorithms 3-32 Height of a (2,4) Tree Theorem: A (2,4) tree storing n items has height O(log n) Proof: –Let h be the height of a (2,4) tree with n items –Since there are at least 2 i items at depth i  0, …, h  1 and no items at depth h, we have n  1  2  4  …  2 h  1  2 h  1 –Thus, h  log (n  1) Searching in a (2,4) tree with n items takes O(log n) time 1 2 2h12h1 0 items 0 1 h1h1 h depth

33. CSC401 -- Analysis of Algorithms 3-33 Insertion We insert a new item (k, o) at the parent v of the leaf reached by searching for k –We preserve the depth property but –We may cause an overflow (i.e., node v may become a 5-node) Example: inserting key 30 causes an overflow 27 32 35 10 15 24 2 81218 v 10 15 24 2 812 27 30 32 35 18 v

34. CSC401 -- Analysis of Algorithms 3-34 Overflow and Split We handle an overflow at a 5-node v with a split operation: –let v 1 … v 5 be the children of v and k 1 … k 4 be the keys of v –node v is replaced nodes v' and v" v' is a 3-node with keys k 1 k 2 and children v 1 v 2 v 3 v" is a 2-node with key k 4 and children v 4 v 5 –key k 3 is inserted into the parent u of v (a new root may be created) The overflow may propagate to the parent node u 15 24 12 27 30 32 35 18 v u v1v1 v2v2 v3v3 v4v4 v5v5 15 24 32 12 27 30 18 v'v' u v1v1 v2v2 v3v3 v4v4 v5v5 35 v"v"

35. CSC401 -- Analysis of Algorithms 3-35 Analysis of Insertion Algorithm insertItem(k, o) 1.We search for key k to locate the insertion node v 2.We add the new item (k, o) at node v 3. while overflow(v) if isRoot(v) create a new empty root above v create a new empty root above v v  split(v) Let T be a (2,4) tree with n items –Tree T has O(log n) height –Step 1 takes O(log n) time because we visit O(log n) nodes –Step 2 takes O(1) time –Step 3 takes O(log n) time because each split takes O(1) time and we perform O(log n) splits Thus, an insertion in a (2,4) tree takes O(log n) time

36. CSC401 -- Analysis of Algorithms 3-36 Deletion We reduce deletion of an item to the case where the item is at the node with leaf children Otherwise, we replace the item with its inorder successor (or, equivalently, with its inorder predecessor) and delete the latter item Example: to delete key 24, we replace it with 27 (inorder successor) 27 32 35 10 15 24 2 81218 32 35 10 15 27 2 81218

37. CSC401 -- Analysis of Algorithms 3-37 Underflow and Fusion Deleting an item from a node v may cause an underflow, where node v becomes a 1-node with one child and no keys To handle an underflow at node v with parent u, we consider two cases Case 1: the adjacent siblings of v are 2-nodes –Fusion operation: we merge v with an adjacent sibling w and move an item from u to the merged node v' –After a fusion, the underflow may propagate to the parent u 9 14 2 5 710 u v 9 10 14 u v'v'w 2 5 7

38. CSC401 -- Analysis of Algorithms 3-38 Underflow and Transfer To handle an underflow at node v with parent u, we consider two cases Case 2: an adjacent sibling w of v is a 3-node or a 4-node –Transfer operation: 1. we move a child of w to v 2. we move an item from u to v 3. we move an item from w to u –After a transfer, no underflow occurs 4 9 6 82 u vw 4 8 62 9 u vw

39. CSC401 -- Analysis of Algorithms 3-39 Analysis of Deletion Let T be a (2,4) tree with n items –Tree T has O(log n) height In a deletion operation –We visit O(log n) nodes to locate the node from which to delete the item –We handle an underflow with a series of O(log n) fusions, followed by at most one transfer – Each fusion and transfer takes O(1) time Thus, deleting an item from a (2,4) tree takes O(log n) time

40. CSC401 -- Analysis of Algorithms 3-40 From (2,4) to Red-Black Trees A red-black tree is a representation of a (2,4) tree by means of a binary tree whose nodes are colored red or black In comparison with its associated (2,4) tree, a red-black tree has –same logarithmic time performance –simpler implementation with a single node type 2 6 73 54 4 6 27 5 3 3 5 OR

41. CSC401 -- Analysis of Algorithms 3-41 Splay Tree Definition a splay tree is a binary search tree where a node is splayed after it is accessed (for a search or update) –deepest internal node accessed is splayed –splaying costs O(h), where h is height of the tree – which is still O(n) worst-case O(h) rotations, each of which is O(1) Binary Search Tree Rules: –items stored only at internal nodes –keys stored at nodes in the left subtree of v are less than or equal to the key stored at v –keys stored at nodes in the right subtree of v are greater than or equal to the key stored at v An inorder traversal will return the keys in order Search proceeds down the tree to the found item or an external node.

42. CSC401 -- Analysis of Algorithms 3-42 Splay Trees do Rotations after Every Operation (Even Search) new operation: splay –splaying moves a node to the root using rotations right rotation makes the left child x of a node y into y’s parent; y becomes the right child of x y x T1T1 T2T2 T3T3 y x T1T1 T2T2 T3T3 left rotation makes the right child y of a node x into x’s parent; x becomes the left child of y y x T1T1 T2T2 T3T3 y x T1T1 T2T2 T3T3 (structure of tree above y is not modified) (structure of tree above x is not modified) a right rotation about ya left rotation about x

43. CSC401 -- Analysis of Algorithms 3-43 Splaying: is x the root? stop is x a child of the root? right-rotate about the root left-rotate about the root is x the left child of the root? is x a left-left grandchild? is x a left-right grandchild? is x a right-right grandchild? is x a right-left grandchild? right-rotate about g, right-rotate about p left-rotate about g, left-rotate about p left-rotate about p, right-rotate about g right-rotate about p, left-rotate about g start with node x “ x is a left-left grandchild” means x is a left child of its parent, which is itself a left child of its parent p is x ’s parent; g is p ’s parent no yes no yes zig-zig zig-zag zig-zig zig

44. CSC401 -- Analysis of Algorithms 3-44 Visualizing the Splaying Cases zig-zag y x T2T2 T3T3 T4T4 z T1T1 y x T2T2 T3T3 T4T4 z T1T1 y x T1T1 T2T2 T3T3 z T4T4 zig-zig y z T4T4 T3T3 T2T2 x T1T1 zig x w T1T1 T2T2 T3T3 y T4T4 y x T2T2 T3T3 T4T4 w T1T1

45. CSC401 -- Analysis of Algorithms 3-45 Splaying Example let x = (8,N) –x is the right child of its parent, which is the left child of the grandparent –left-rotate around p, then right-rotate around g (20,Z) (37,P)(21,O) (14,J) (7,T) (35,R)(10,A) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (8,N) (7,P) (36,L) (10,U) (40,X) x g p (10,A) (20,Z) (37,P)(21,O) (35,R) (36,L) (40,X) (7,T) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (14,J) (8,N) (7,P) (10,U) x g p (10,A) (20,Z) (37,P)(21,O) (35,R) (36,L) (40,X) (7,T) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (14,J) (8,N) (7,P) (10,U) x g p 1. (before rotating) 2. (after first rotation) 3. (after second rotation) x is not yet the root, so we splay again

46. CSC401 -- Analysis of Algorithms 3-46 Splaying Example, Continued now x is the left child of the root –right-rotate around root (10,A) (20,Z) (37,P)(21,O) (35,R) (36,L) (40,X) (7,T) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (14,J) (8,N) (7,P) (10,U) x (10,A) (20,Z) (37,P)(21,O) (35,R) (36,L) (40,X) (7,T) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (14,J) (8,N) (7,P) (10,U) x 1. (before applying rotation) 2. (after rotation) x is the root, so stop

47. CSC401 -- Analysis of Algorithms 3-47 Example Result tree might not be more balanced e.g. splay (40,X) –before, the depth of the shallowest leaf is 3 and the deepest is 7 –after, the depth of shallowest leaf is 1 and deepest is 8 (20,Z) (37,P)(21,O) (14,J) (7,T) (35,R)(10,A) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (8,N) (7,P) (36,L) (10,U) (40,X) (20,Z) (37,P) (21,O) (14,J) (7,T) (35,R) (10,A) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (8,N) (7,P) (36,L) (10,U) (40,X) (20,Z) (37,P) (21,O) (14,J) (7,T) (35,R) (10,A) (1,C) (1,Q) (5,G) (2,R) (5,H) (6,Y) (5,I) (8,N) (7,P) (36,L) (10,U) (40,X) before after first splay after second splay

48. CSC401 -- Analysis of Algorithms 3-48 Splay Trees & Ordered Dictionaries which nodes are splayed after each operation? use the parent of the internal node that was actually removed from the tree (the parent of the node that the removed item was swapped with) removeElement use the new node containing the item inserted insertElement if key found, use that node if key not found, use parent of ending external node findElement splay node method

49. CSC401 -- Analysis of Algorithms 3-49 Amortized Analysis of Splay Trees Running time of each operation is proportional to time for splaying. Define rank(v) as the logn(v), where n(v) is the number of nodes in subtree rooted at v. Costs: zig = $1, zig-zig = $2, zig-zag = $2. Thus, cost for playing a node at depth d = $d. Imagine that we store rank(v) cyber-dollars at each node v of the splay tree (just for the sake of analysis). Cost per Zig -- Doing a zig at x costs at most rank’(x) - rank(x): –cost = rank’(x) + rank’(y) - rank(y) rank’(y) - rank(y) -rank(x) -rank(x) < rank’(x) - rank(x). < rank’(x) - rank(x). zig x w T1T1 T2T2 T3T3 y T4T4 y x T2T2 T3T3 T4T4 w T1T1

50. CSC401 -- Analysis of Algorithms 3-50 Cost per zig-zig and zig-zag Doing a zig-zig or zig-zag at x costs at most 3(rank’(x) - rank(x)) - 2. –Proof: See Theorem 3.9, Page 192. y x T1T1 T2T2 T3T3 z T4T4 zig-zig y z T4T4 T3T3 T2T2 x T1T1 zig-zag y x T2T2 T3T3 T4T4 z T1T1 y x T2T2 T3T3 T4T4 z T1T1

51. CSC401 -- Analysis of Algorithms 3-51 Performance of Splay Trees Cost of splaying a node x at depth d of a tree rooted at r is at most 3(rank(r)-rank(x))-d+2 –Proof: Splaying x takes d/2 splaying substeps: Recall: rank of a node is logarithm of its size, thus, amortized cost of any splay operation is O(log n). In fact, the analysis goes through for any reasonable definition of rank(x). This implies that splay trees can actually adapt to perform searches on frequently-requested items much faster than O(log n) in some cases. (See Theorems 3.10 and 3.11.)

52. CSC401 -- Analysis of Algorithms 3-52 What is a Skip List A skip list for a set S of distinct (key, element) items is a series of lists S 0, S 1, …, S h such that –Each list S i contains the special keys  and  –List S 0 contains the keys of S in nondecreasing order –Each list is a subsequence of the previous one, i.e., S 0  S 1  …  S h –List S h contains only the two special keys We show how to use a skip list to implement the dictionary ADT 566478  313444  122326    31  64  3134  23 S0S0 S1S1 S2S2 S3S3

53. CSC401 -- Analysis of Algorithms 3-53 Search We search for a key x in a a skip list as follows: –We start at the first position of the top list –At the current position p, we compare x with y  key(after(p)) x  y : we return element(after(p)) x  y : we “scan forward” x  y : we “drop down” –If we try to drop down past the bottom list, we return NO_SUCH_KEY Example: search for 78   S0S0 S1S1 S2S2 S3S3  31  64  3134  23 56 6478  313444  122326

54. CSC401 -- Analysis of Algorithms 3-54 Randomized Algorithms A randomized algorithm performs coin tosses (i.e., uses random bits) to control its execution It contains statements of the type b  random() if b  0 do A … else { b  1} do B … Its running time depends on the outcomes of the coin tosses We analyze the expected running time of a randomized algorithm under the following assumptions –the coins are unbiased, and –the coin tosses are independent The worst-case running time of a randomized algorithm is often large but has very low probability (e.g., it occurs when all the coin tosses give “heads”) We use a randomized algorithm to insert items into a skip list

55. CSC401 -- Analysis of Algorithms 3-55 To insert an item (x, o) into a skip list, we use a randomized algorithm: –We repeatedly toss a coin until we get tails, and we denote with i the number of times the coin came up heads –If i  h, we add to the skip list new lists S h  1, …, S i  1, each containing only the two special keys –We search for x in the skip list and find the positions p 0, p 1, …, p i of the items with largest key less than x in each list S 0, S 1, …, S i –For j  0, …, i, we insert item (x, o) into list S j after position p j Example: insert key 15, with i  2 Insertion   S0S0 S1S1 S2S2 S3S3   103623 15   2315   10 36  23   S0S0 S1S1 S2S2 p0p0 p1p1 p2p2

56. CSC401 -- Analysis of Algorithms 3-56 Deletion To remove an item with key x from a skip list, we proceed as follows: –We search for x in the skip list and find the positions p 0, p 1, …, p i of the items with key x, where position p j is in list S j –We remove positions p 0, p 1, …, p i from the lists S 0, S 1, …, S i –We remove all but one list containing only the two special keys Example: remove key 34   4512   23   S0S0 S1S1 S2S2   S0S0 S1S1 S2S2 S3S3   451223 34     23 34 p0p0 p1p1 p2p2

57. CSC401 -- Analysis of Algorithms 3-57 Implementation We can implement a skip list with quad-nodes A quad-node stores: –item –link to the node before –link to the node after –link to the node below –link to the node after Also, we define special keys PLUS_INF and MINUS_INF, and we modify the key comparator to handle them x quad-node

58. CSC401 -- Analysis of Algorithms 3-58 Space Usage The space used by a skip list depends on the random bits used by each invocation of the insertion algorithm We use the following two basic probabilistic facts: Fact 1: The probability of getting i consecutive heads when flipping a coin is 1  2 i Fact 2: If each of n items is present in a set with probability p, the expected size of the set is np Consider a skip list with n items –By Fact 1, we insert an item in list S i with probability 1  2 i –By Fact 2, the expected size of list S i is n  2 i The expected number of nodes used by the skip list is Thus, the expected space usage of a skip list with n items is O(n)

59. CSC401 -- Analysis of Algorithms 3-59 Height The running time of the search an insertion algorithms is affected by the height h of the skip list We show that with high probability, a skip list with n items has height O(log n) We use the following additional probabilistic fact: Fact 3: If each of n events has probability p, the probability that at least one event occurs is at most np Consider a skip list with n items –By Fact 1, we insert an item in list S i with probability 1  2 i –By Fact 3, the probability that list S i has at least one item is at most n  2 i By picking i  3log n, we have that the probability that S 3log n has at least one item is at most n  2 3log n  n  n 3  1  n 2 Thus a skip list with n items has height at most 3log n with probability at least 1  1  n 2

60. CSC401 -- Analysis of Algorithms 3-60 Search and Update Times The search time in a skip list is proportional to –the number of drop-down steps, plus –the number of scan- forward steps The drop-down steps are bounded by the height of the skip list and thus are O(log n) with high probability To analyze the scan- forward steps, we use yet another probabilistic fact: Fact 4: The expected number of coin tosses required in order to get tails is 2 When we scan forward in a list, the destination key does not belong to a higher list –A scan-forward step is associated with a former coin toss that gave tails By Fact 4, in each list the expected number of scan-forward steps is 2 Thus, the expected number of scan-forward steps is O(log n) We conclude that a search in a skip list takes O(log n) expected time The analysis of insertion and deletion gives similar results

61. CSC401 -- Analysis of Algorithms 3-61 Summary A skip list is a data structure for dictionaries that uses a randomized insertion algorithm In a skip list with n items –The expected space used is O(n) –The expected search, insertion and deletion time is O(log n) Using a more complex probabilistic analysis, one can show that these performance bounds also hold with high probability Skip lists are fast and simple to implement in practice