Search Trees Last Update: Nov 5, 2014 EECS2011: Search Trees1 “Grey Tree”, Piet Mondrian, 1912.

Binary Search Trees Last Update: Nov 5, 2014 EECS2011: Search Trees

Ordered Maps Keys are assumed to come from a total order. Items are stored in order by their keys Ordered Maps support nearest neighbor queries:  Item with largest key less than or equal to k  Item with smallest key greater than or equal to k  Hash Tables cannot perform such ordered map operations (effectively) Last Update: Nov 5, 2014 EECS2011: Search Trees3

Binary Search Binary search can perform nearest neighbor queries on an ordered map that is implemented with an array, sorted by key – similar to the high-low children’s game – at each step, the number of candidate items is halved – terminates after O(log n) steps Example: find(7) Last Update: Nov 5, 2014 EECS2011: Search Trees m l h m l h m l h l  m  h

Search Tables A search table is an ordered map implemented by means of a sorted sequence  We store the items in an array-based sequence, sorted by key  We use an external comparator for the keys Performance:  Searches take O(log n) time, using binary search  Inserting a new item takes O(n) time, since in the worst case we have to shift O(n) items to make room for the new item  Removing an item takes O(n) time, since in the worst case we have to shift O(n) items to compact the items after the removal The lookup table is effective only for ordered maps of small size or for maps on which searches are the most common operations, while insertions and removals are rarely performed (e.g., credit card authorizations) Last Update: Nov 5, 2014 EECS2011: Search Trees5

From binary search to Binary Search Trees Last Update: Nov 5, 2014 EECS2011: Search Trees6

The Binary Search Tree Hierarchy Last Update: Nov 5, 2014 EECS2011: Search Trees 7 «interface» Iterable «interface» Iterable AbstractTree Abstract Sorted Map ( § 10.3) Abstract Sorted Map ( § 10.3) Abstract BinaryTree TreeMap (Binary Search Tree) TreeMap (Binary Search Tree) Linked Binary Tree (§ 8.3) Linked Binary Tree (§ 8.3) Balanceable BinaryTree (nested) Balanceable BinaryTree (nested) Splay Tree Splay Tree «interface» Tree «interface» Tree «interface» Binary Tree «interface» Binary Tree Red-Black Tree Red-Black Tree AVL Tree AVL Tree chapter 11

Binary Search Tree (BST) A BST is a binary tree with the following three properties: 1.internal nodes store keys (or key-value entries) 2.external nodes are null & do not store items. 3.search property: The key at any node is > all keys in its left subtree, and < all keys in its right subtree.  Equivalently, an inorder traversal of the BST visits its keys in increasing order Last Update: Nov 5, 2014 EECS2011: Search Trees

Search To search for a key k, we trace a downward path starting at the root The next node visited depends on the comparison of k with the key of the current node If we reach an external node, the key is not found Example: get(4): – Call TreeSearch(4, root) The algorithms for nearest neighbor queries are similar Last Update: Nov 5, 2014 EECS2011: Search Trees9 Algorithm TreeSearch(k, node) if isExternal(node) return node // failed if k = key(node) return node // found if k < key(node) // search left subtree return TreeSearch(k, left(node)) // else, k > key(node): search right subtree return TreeSearch(k, right(node))   

Insertion To perform operation put(k, v), we search for key k (using TreeSearch) Assume k is not already in the tree, and let w be the external node reached by the search We expand w into an internal node & insert (k, v) in it Example: insert 5 Last Update: Nov 5, 2014 EECS2011: Search Trees    w w

Deletion To perform operation remove(k), we search for key k Assume key k is in the tree, and let x be the node storing k If x has an external child y, we remove x and y from the tree with the operation removeExternal(y), which removes y and its parent x Example: remove 4 Last Update: Nov 5, 2014 EECS2011: Search Trees x y  

Deletion (cont.) Now suppose both children of node x are internal – we find the internal node w that follows x in the inorder traversal – we copy key(w) into node x – we remove node w and its left child z (which must be external) by means of operation removeExternal(z) Example: remove 3 Last Update: Nov 5, 2014 EECS2011: Search Trees x w z x 2

Performance Consider an ordered map with n items implemented by a BST of height h – the space used is O(n) – methods get, put and remove take O(h) time h is O(n) in the worst case and O(log n) in the best case Last Update: Nov 5, 2014 EECS2011: Search Trees13

AVL Trees (Adelson-Velskii & Landis ) EECS2011: Search Trees14 Last Update: Nov 5,

AVL Tree Definition AVL trees are height balanced BSTs height balance property: heights of siblings can differ by at most 1 EECS2011: Search Trees15 Last Update: Nov 5, 2014 height Example: 4

Height of an AVL Tree EECS2011: Search Trees n(1) n(2) Fact: The height of an AVL tree storing n keys is O(log n). Last Update: Nov 5, 2014

Dictionary operations: Overview Search (and other similar access methods) remain the same as in BSTs. The worst-case running time is O(log n), thanks to the bound on the height of AVL trees. Insert, Delete (and other similar update methods) start off as in BSTs, but may alter the AVL tree structure. Hence they may affect the height balance property and require further restructuring of the tree to restore that property. We will show that all dictionary operations can be done in O(log n) time in the worst case. EECS2011: Search Trees17 Last Update: Nov 5, 2014

AVLTree Class The latter uses an added aux field per node with getAux() & setAux(). These extentions are dormant in TreeMap. aux is used as node height in AVLTree (and as red/black node color bit in RBTree). TreeMap provides hooks for the following operations that are overrided to specific use by AVLTree (and RBTree)  rebalanceAccess()  rebalanceInsert()  rebalanceDelete() EECS2011: Search Trees18 Last Update: Nov 5, 2014 TreeMap (Binary Search Tree) TreeMap (Binary Search Tree) Balanceable BinaryTree (nested) Balanceable BinaryTree (nested) Splay Tree Splay Tree Red-Black Tree Red-Black Tree AVL Tree AVL Tree The AVLTree class extends the TreeMap class (BST) which nests the BalanceableBinaryTree class.

imbalance! Insert insert(2) Last Update: Nov 5, 2014 EECS2011: Search Trees19

imbalance! Insert Imbalance may occur at any ancestor of the inserted node insert(2) Last Update: Nov 5, 2014 EECS2011: Search Trees20

AVL Insert: overview Last Update: Nov 5, 2014 EECS2011: Search Trees21

Link Rotation Rotation properties: – preserves the search property (doesn’t change the inorder sequence) – may affect the height balance property – This local restructuring takes O(1) time and can be applied to any accessed internal link of the BST – heights of nodes x and y may change, and are updated too. Rotation properties: – preserves the search property (doesn’t change the inorder sequence) – may affect the height balance property – This local restructuring takes O(1) time and can be applied to any accessed internal link of the BST – heights of nodes x and y may change, and are updated too. Last Update: Nov 5, 2014 EECS2011: Search Trees22 C y A B x A x B y C Right Rotate  Left Rotate  ’  ’’

imbalance! Insert: Rebalancing Strategy Step 2.1: probe upward – Starting at the inserted node, traverse toward the root until an imbalance is discovered. Last Update: Nov 5, 2014 EECS2011: Search Trees

Insert: Rebalancing Strategy Step 2.2: repair  The repair strategy is called trinode restructuring.  3 nodes x, y and z are distinguished:  z = the parent of the high sibling  y = the high sibling  x = the high child of the high sibling  We can now think of the subtree rooted at z as consisting of these 3 nodes plus their 4 subtrees Last Update: Nov 5, 2014 EECS2011: Search Trees24 imbalance!

Trinode restructuring Two other left-right symmetric cases not shown. Last Update: Nov 5, 2014 EECS2011: Search Trees25 D z C y A B x A x B y C D z same slant 2 1 Rotate 2 D z A y BC x x A B y C D z opposite slant 2 1 Rotate 1 then 2

Trinode restructuring Affected node heights: Last Update: Nov 5, 2014 EECS2011: Search Trees26 A x B y C D z h D z C y A B x h-1  h h  h+1 h+1  h+2 h-1 h h+1 Top node restores the old height. So, no further restructuring needed higher up Top node restores the old height. So, no further restructuring needed higher up

Trinode restructuring Affected node heights: Last Update: Nov 5, 2014 EECS2011: Search Trees27 x A B y C D z D z A y BC x h-1 h h h+1 h-1 <h-1 h-1 h-1  h h  h+1 h+1  h+2 Top node restores the old height. So, no further restructuring needed higher up Top node restores the old height. So, no further restructuring needed higher up

Example: Insert 12 EECS2011: Search Trees28 Last Update: Nov 5,

Example: Insert 12 EECS2011: Search Trees29 Last Update: Nov 5, w Step 1.1: top-down search

Example: Insert 12 EECS2011: Search Trees30 Last Update: Nov 5, w

Example: Insert 12 EECS2011: Search Trees31 Last Update: Nov 5, w imbalance

Example: Insert 12 EECS2011: Search Trees32 Last Update: Nov 5, Step 2.2: trinode discovered (needs double rotation) x y z

Example: Insert 12 EECS2011: Search Trees33 Last Update: Nov 5, Step 2.3: trinode restructured; balance restored. DONE! z y x

Insert: the whole process At most one trinode restructuring is needed  The tree was balanced before the insertion.  Insertion raised the height of the subtree by 1.  Rebalancing lowered the height of that subtree by 1.  Thus the whole tree is still balanced. Insert takes O(log n) time. Last Update: Nov 5, 2014 EECS2011: Search Trees34

imbalance! Delete Imbalance may occur at an ancestor of the removed node. Last Update: Nov 5, 2014 EECS2011: Search Trees35 delete(17)

Delete: Rebalancing Strategy Step 1: probe upward – Let w be the node actually removed (i.e., the node matching the key if it has a external child; otherwise the node following in the in-order traversal). – Starting at w, traverse toward the root until an imbalance is discovered. Last Update: Nov 5, 2014 EECS2011: Search Trees36 imbalance!

Delete: Rebalancing Strategy Step 2: Repair – We again use trinode restructuring. – 3 nodes x, y and z are distinguished: z = the parent of the high sibling y = the high sibling x = the high child of the high sibling (* if children are equally high, use same slant ) Last Update: Nov 5, 2014 EECS2011: Search Trees37 imbalance!

Trinode restructuring Two other left-right symmetric cases not shown. Last Update: Nov 5, 2014 EECS2011: Search Trees38 D z C y A B x A x B y C D z same slant 2 1 Rotate 2 D z A y BC x x A B y C D z opposite slant 2 1 Rotate 1 then 2

Trinode restructuring Affected node heights: Last Update: Nov 5, 2014 EECS2011: Search Trees39 A x B y C D z h D z C y A B x h h  h-1 h+2 h-1 or h h+1 h-1 or h h-1 h or h+1 h+1 or h+2 Top node height may decrease. Further restructuring may be needed higher up. Top node height may decrease. Further restructuring may be needed higher up.

Trinode restructuring Affected node heights: Last Update: Nov 5, 2014 EECS2011: Search Trees40 x A B y C D z D z A y BC x h-1 h+1 h h h-1 <h-1 h-1 h h  h-1 h+2 Top node height decreases. Further restructuring may be needed higher up. Top node height decreases. Further restructuring may be needed higher up.

Delete: the whole Process Along the node removal path many trinode restructuring may be needed Delete takes O(log n) time Last Update: Nov 5, 2014 EECS2011: Search Trees41

AVL Tree Performance Link structured AVL tree storing n items O(n) space O(1) time for a single restructuring step (e.g., rotation) O(log n) time: Search, Insert, Delete Search requires no restructuring Insert & delete:  initial search takes O(log n) time  restructuring up the tree, updating height fields, and rebalancing unbalanced nodes take O(log n) time too EECS2011: Search Trees42 Last Update: Nov 5, 2014

Java Implementation EECS2011: Search Trees 43 Last Update: Nov 5, 2014

Java Implementation, 2 EECS2011: Search Trees 44 Last Update: Nov 5, 2014

Java Implementation, 3 EECS2011: Search Trees 45 Last Update: Nov 5, 2014

Splay Trees (1985) EECS2011: Search Trees46 Last Update: Nov 5, 2014 Robert Tarjan Turing Award Winner, 1986 Daniel Sleator Tarjan’s former student

Splay Tree Self-adjusting BST  doesn’t use the “aux” node field  It is not height balanced  but tends to structurally adjust to pattern of usage Allows quick access to recently accessed elements Bad: worst-case O(n) Good: amortized case O(log n) Often outperforms other BSTs in practice Has several optimality properties unlike other BSTs Last Update: Nov 5, 2014 EECS2011: Search Trees47

Splaying Splaying is an operation performed on a node. It moves the node to the root of the tree by a series of rotations. In splay trees, each BST dictionary operation search, insert, delete is followed by a splay operation. In this way, recently searched and inserted elements are near the top of the tree, for quick access. Last Update: Nov 5, 2014 EECS2011: Search Trees48

Splay(x) Perform a series of splay steps, whichever applies, until x becomes the root of the tree (each splay step is one or two rotations) splay steps are of three types (shown on the next page) (left-right symmetric cases not shown) Perform a series of splay steps, whichever applies, until x becomes the root of the tree (each splay step is one or two rotations) splay steps are of three types (shown on the next page) (left-right symmetric cases not shown) Last Update: Nov 5, 2014 EECS2011: Search Trees49

A Splay Step on Node x Last Update: Nov 5, 2014 EECS2011: Search Trees50 A B C y root x x A B C y zig D z A B C y x zig-zig A x z D C B y zig-zag A x D C B D z A B C y x y z

51 Examples Last Update: Nov 5, 2014 EECS2011: Search Trees

Which Node to Splay? Q) Which node is splayed after a dictionary operation? A) The deepest accessed node still in the tree. splay the parent of the internal node that was actually removed from the tree (i.e., the parent of the node whose element replaced that of the node containing k) delete(k) splay the node (new/old) that contains the insertion key k insert(k,v) if key found, splay that node if key not found, splay parent of external node where search terminated search(k) splay nodemethod Last Update: Nov 5, 2014 EECS2011: Search Trees52

Performance Last Update: Nov 5, 2014 EECS2011: Search Trees53

Other Forms of Search Trees (2, 4) Trees  These are multi-way search trees (not binary trees) in which internal nodes may contain 1, 2, or 3 keys & have 2, 3, or 4 subtrees  all external nodes have exactly the same depth.  Worst-case O(log n) time per dictionary operation Red-Black Trees (RB Tree)  A form of balanced Binary Search Trees  Worst-case O(log n) time per dictionary operation These (and their generalizations) are covered in EECS4101 Last Update: Nov 5, 2014 EECS2011: Search Trees54

Comparison of Map Implementations EECS2011: Search Trees55 SearchInsertDelete Notes Hash Table O(1) expected O(n) worst case O(1) expected O(n) worst case O(1) expected O(n) worst case no ordered map methods simple to implement Skip List O(log n) high prob. randomized insertion simple to implement AVL, (2,4) Tree, RB Tree O(log n) worst-case complex to implement suited for time critical applications Splay Tree O(log n) amortized simple to implement suitable for batch-mode applications adjusts to usage pattern competitiveness properties Last Update: Nov 5, 2014

Summary Last Update: Nov 5, 2014 EECS2011: Search Trees 56 Binary Search Trees (BSTs):  O(n) worst-case time per dictionary operation AVL Trees:  Height = O(log n)  Node height: auxiliary info stored in each node  Rotation as local restructuring step  O(log n) worst-case time per dictionary operation Splay Trees:  Self adjusting BST.  No auxiliary info kept in nodes  O(log n) amortized time per dictionary operation. (2,4)-trees, Red-Black Trees: covered in EECS4101

Last Update: Nov 5, 2014 EECS2011: Search Trees 57