1. CSC401 – Analysis of Algorithms Lecture Notes 6 Dictionaries and Search Trees Objectives: Introduce dictionaries and its diverse implementations Introduce binary search trees and present operations on binary search trees Analyze the performance of binary search tree operations Introduce balanced binary search trees: AVL tree and Red-Black tree Introduce a design pattern: Locator

2. 2 Dictionary ADT The dictionary ADT models a searchable collection of key- element items The main operations of a dictionary are searching, inserting, and deleting items Multiple items with the same key are allowed Applications: –address book –credit card authorization –mapping host names (e.g., cs16.net) to internet addresses (e.g., 128.148.34.101) Dictionary ADT methods: –findElement(k): if the dictionary has an item with key k, returns its element, else, returns the special element NO_SUCH_KEY –insertItem(k, o): inserts item (k, o) into the dictionary –removeElement(k): if the dictionary has an item with key k, removes it from the dictionary and returns its element, else returns the special element NO_SUCH_KEY –size(), isEmpty() –keys(), Elements()

3. 3 Log File A log file is a dictionary implemented by means of an unsorted sequence –We store the items of the dictionary in a sequence (based on a doubly-linked lists or a circular array), in arbitrary order Performance: –insertItem takes O(1) time since we can insert the new item at the beginning or at the end of the sequence –findElement and removeElement take O(n) time since in the worst case (the item is not found) we traverse the entire sequence to look for an item with the given key The log file is effective only for dictionaries of small size or for dictionaries on which insertions are the most common operations, while searches and removals are rarely performed (e.g., historical record of logins to a workstation)

4. 4 Binary Search Binary search performs operation findElement(k) on a dictionary implemented by means of an array-based sequence, sorted by key –similar to the high-low game –at each step, the number of candidate items is halved –terminates after a logarithmic number of steps Example: findElement(7) 13457 8 91114161819 1 3 457891114161819 134 5 7891114161819 1345 7 891114161819 0 0 0 0 m l h m l h m l h l  m  h

5. 5 Lookup Table A lookup table is a dictionary implemented by means of a sorted sequence –We store the items of the dictionary in an array-based sequence, sorted by key –We use an external comparator for the keys Performance: –findElement takes O(log n) time, using binary search –insertItem takes O(n) time since in the worst case we have to shift n  2 items to make room for the new item –removeElement take O(n) time since in the worst case we have to shift n  2 items to compact the items after the removal The lookup table is effective only for dictionaries of small size or for dictionaries on which searches are the most common operations, while insertions and removals are rarely performed (e.g., credit card authorizations)

6. 6 Binary Search Tree A binary search tree is a binary tree storing keys (or key-element pairs) at its internal nodes and satisfying the following property: –Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u)  key(v)  key(w) External nodes do not store items An inorder traversal of a binary search trees visits the keys in increasing order 6 92 418

7. 7 Search To search for a key k, we trace a downward path starting at the root The next node visited depends on the outcome of the comparison of k with the key of the current node If we reach a leaf, the key is not found and we return NO_SUCH_KEY Example: findElement(4) Algorithm findElement(k, v) if T.isExternal (v) return NO_SUCH_KEY if k  key(v) return findElement(k, T.leftChild(v)) else if k  key(v) return element(v) else { k  key(v) } return findElement(k, T.rightChild(v)) 6 9 2 4 1 8   

8. 8 Insertion To perform operation insertItem(k, o), we search for key k Assume k is not already in the tree, and let let w be the leaf reached by the search We insert k at node w and expand w into an internal node Example: insert 5 6 9 2 4 18    w 6 92 418 5 w

9. 9 Deletion To perform operation removeElement( k ), we search for key k Assume key k is in the tree, and let let v be the node storing k If node v has a leaf child w, we remove v and w from the tree with operation removeAboveExternal( w ) Example: remove 4 6 9 2 5 18 6 9 2 4 18 5 v w  

10. 10 Deletion (cont.) We consider the case where the key k to be removed is stored at a node v whose children are both internal –we find the internal node w that follows v in an inorder traversal –we copy key(w) into node v –we remove node w and its left child z (which must be a leaf) by means of operation removeAboveExternal( z ) Example: remove 3 3 1 8 6 9 5 v w z 2 5 1 8 6 9 v 2

11. 11 Performance Consider a dictionary with n items implemented by means of a binary search tree of height h –the space used is O(n) –methods findElement, insertItem and removeElement take O(h) time The height h is O(n) in the worst case and O(log n) in the best case

12. 12 AVL Tree Definition AVL trees are balanced. An AVL Tree is a binary search tree such that for every internal node v of T, the heights of the children of v can differ by at most 1. An example of an AVL tree where the heights are shown next to the nodes:

13. 13 Height of an AVL Tree Fact: The height of an AVL tree storing n keys is O(log n). storing n keys is O(log n). Proof: Let us bound n(h): the minimum number of internal nodes of an AVL tree of height h. We easily see that n(1) = 1 and n(2) = 2 For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of height n-2. That is, n(h) = 1 + n(h-1) + n(h-2) Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). So n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by induction), n(h) > 2 i n(h-2i) Solving the base case we get: n(h) > 2 h/2-1 Taking logarithms: h < 2log n(h) +2 Thus the height of an AVL tree is O(log n) 3 4 n(1) n(2)

14. 14 Insertion in an AVL Tree Insertion is as in a binary search tree Always done by expanding an external node. Example: 44 1778 325088 4862 54 w b=x a=y c=z 44 1778 325088 4862 before insertionafter insertion

15. 15 Trinode Restructuring let (a,b,c) be an inorder listing of x, y, z perform the rotations needed to make b the topmost node of the three b=y a=z c=x T0T0 T1T1 T2T2 T3T3 b=y a=z c=x T0T0 T1T1 T2T2 T3T3 c=y b=x a=z T0T0 T1T1 T2T2 T3T3 b=x c=ya=z T0T0 T1T1 T2T2 T3T3 case 1: single rotation (a left rotation about a) case 2: double rotation (a right rotation about c, then a left rotation about a) (other two cases are symmetrical)

16. 16 Insertion Example, continued

17. 17 Restructuring (Single Rotations) Single Rotations:

18. 18 Restructuring (Double Rotations) double rotations:

19. 19 Removal in an AVL Tree Removal begins as in a binary search tree, which means the node removed will become an empty external node. Its parent, w, may cause an imbalance. Example: 44 17 783250 88 48 62 54 44 17 7850 88 48 62 54 before deletion of 32after deletion

20. 20 Rebalancing after a Removal Let z be the first unbalanced node encountered while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of y with the larger height. We perform restructure(x) to restore balance at z. As this restructuring may upset the balance of another node higher in the tree, we must continue checking for balance until the root of T is reached 44 17 7850 88 48 62 54 w c=x b=y a=z 44 17 78 5088 48 62 54

21. 21 Running Times for AVL Trees a single restructure is O(1) –using a linked-structure binary tree find is O(log n) –height of tree is O(log n), no restructures needed insert is O(log n) –initial find is O(log n) –Restructuring up the tree, maintaining heights is O(log n) remove is O(log n) –initial find is O(log n) –Restructuring up the tree, maintaining heights is O(log n)

22. 22 Red-Black Tree A red-black tree can also be defined as a binary search tree that satisfies the following properties: –Root Property: the root is black –External Property: every leaf is black –Internal Property: the children of a red node are black –Depth Property: all the leaves have the same black depth 9 15 4 62 12 7 21 Theorem: A red-black tree storing n items has height O(log n) –The height of a red- black tree is at most twice the height of its associated (2,4) tree, which is O(log n) The search algorithm for a binary search tree is the same as that for a binary search tree By the above theorem, searching in a red-black tree takes O(log n) time

23. 23 Insertion To perform operation insertItem (k, o), we execute the insertion algorithm for binary search trees and color red the newly inserted node z unless it is the root –We preserve the root, external, and depth properties –If the parent v of z is black, we also preserve the internal property and we are done –Else ( v is red ) we have a double red (i.e., a violation of the internal property), which requires a reorganization of the tree Example where the insertion of 4 causes a double red: 6 3 8 6 3 8 4 z vv z

24. 24 Remedying a Double Red Consider a double red with child z and parent v, and let w be the sibling of v 4 6 7 z vw 2 4 6 7.. 2.. Case 1: w is black –The double red is an incorrect replacement of a 4-node –Restructuring: we change the 4-node replacement Case 2: w is red –The double red corresponds to an overflow –Recoloring: we perform the equivalent of a split 4 6 7 z v 2 4 6 7 2 w

25. 25 Restructuring A restructuring remedies a child-parent double red when the parent red node has a black sibling It is equivalent to restoring the correct replacement of a 4-node The internal property is restored and the other properties are preserved 4 6 7 z v w 2 4 6 7.. 2.. 4 6 7 z v w 2 4 6 7.. 2..

26. 26 Restructuring (cont.) There are four restructuring configurations depending on whether the double red nodes are left or right children 2 4 6 6 2 4 6 4 2 2 6 4 2 6 4

27. 27 Recoloring A recoloring remedies a child-parent double red when the parent red node has a red sibling The parent v and its sibling w become black and the grandparent u becomes red, unless it is the root It is equivalent to performing a split on a 5-node The double red violation may propagate to the grandparent u 4 6 7 z v 2 4 6 7 2 w 4 6 7 z v 6 7 2 w … 4 … 2

28. 28 Analysis of Insertion Recall that a red-black tree has O(log n) height Step 1 takes O(log n) time because we visit O(log n) nodes Step 2 takes O(1) time Step 3 takes O(log n) time because we perform –O(log n) recolorings, each taking O(1) time, and –at most one restructuring taking O(1) time Thus, an insertion in a red-black tree takes O(log n) time Algorithm insertItem(k, o) 1.We search for key k to locate the insertion node z 2.We add the new item (k, o) at node z and color z red 3. while doubleRed(z) if isBlack(sibling(parent(z))) z  restructure(z) z  restructure(z)return else { sibling(parent(z) is red } z  recolor(z) z  recolor(z)

29. 29 Deletion To perform operation remove (k), we first execute the deletion algorithm for binary search trees Let v be the internal node removed, w the external node removed, and r the sibling of w –If either v of r was red, we color r black and we are done –Else ( v and r were both black) we color r double black, which is a violation of the internal property requiring a reorganization of the tree Example where the deletion of 8 causes a double black: 6 3 8 4 v rw 6 3 4 r

30. 30 Remedying a Double Black The algorithm for remedying a double black node w with sibling y considers three cases Case 1: y is black and has a red child –We perform a restructuring, equivalent to a transfer, and we are done Case 2: y is black and its children are both black –We perform a recoloring, equivalent to a fusion, which may propagate up the double black violation Case 3: y is red –We perform an adjustment, equivalent to choosing a different representation of a 3-node, after which either Case 1 or Case 2 applies Deletion in a red-black tree takes O(log n) time

31. 31 Red-Black Tree Reorganization remedy double red Insertion double red removed or propagated up splitrecoloring double red removed change of 4-node representation restructuring result (2,4) tree action Red-black tree action remedy double black Deletion restructuring or recoloring follows change of 3-node representation adjustment double black removed or propagated up fusionrecoloring double black removed transferrestructuring result (2,4) tree action Red-black tree action

32. 32 Locators A locators identifies and tracks a (key, element) item within a data structure A locator sticks with a specific item, even if that element changes its position in the data structure Intuitive notion: –claim check –reservation number Methods of the locator ADT: –key(): returns the key of the item associated with the locator –element(): returns the element of the item associated with the locator Application example: –Orders to purchase and sell a given stock are stored in two priority queues (sell orders and buy orders) the key of an order is the price the element is the number of shares –When an order is placed, a locator to it is returned –Given a locator, an order can be canceled or modified

33. 33 Locator-based Methods Locator-based priority queue methods: –insert(k, o): inserts the item (k, o) and returns a locator for it –min(): returns the locator of an item with smallest key –remove(l): remove the item with locator l –replaceKey(l, k): replaces the key of the item with locator l –replaceElement(l, o): replaces with o the element of the item with locator l –locators(): returns an iterator over the locators of the items in the priority queue Locator-based dictionary methods: –insert(k, o): inserts the item (k, o) and returns its locator –find(k): if the dictionary contains an item with key k, returns its locator, else return the special locator NO_SUCH_KEY –remove(l): removes the item with locator l and returns its element –locators(), replaceKey(l, k), replaceElement(l, o)

34. 34 Implementation The locator is an object storing –key –element –position (or rank) of the item in the underlying structure In turn, the position (or array cell) stores the locator Example: –binary search tree with locators 3 a 6 d 9 b 1 g 4 e 8 c

35. 35 Positions vs. Locators Position –represents a “place” in a data structure –related to other positions in the data structure (e.g., previous/next or parent/child) –implemented as a node or an array cell Position-based ADTs (e.g., sequence and tree) are fundamental data storage schemes Locator –identifies and tracks a (key, element) item –unrelated to other locators in the data structure –implemented as an object storing the item and its position in the underlying structure Key-based ADTs (e.g., priority queue and dictionary) can be augmented with locator-based methods