Chapter 12&13: Binary Search Trees (BSTs) Overview: Definition of dynamic sets Operations on dynamic sets Definition of BST Examples of balanced and unbalanced BSTs support dynamic-set operations T(n) proportional to height of tree Extensions, like Red-Black trees, ensure balance

Basic Definitions dynamic sets = sets that change due to manipulations performed by algorithms. dictionary = dynamic set that supports insertion, deletion, and membership testing. (priority queue is a dictionary) elements of dynamic set = objects with fields that can be examined and manipulated key = a field of an object that identifies it For any 2 keys a and b, either a<b, a=b, or a>b allows relationships to be defined between set members. (member with smallest key, member with next larger key, etc.) satellite data = information not used in searching queries = operations on sets that only extract information

Operations on dynamic sets Search(S,k) = query that returns a pointer x if there exist an element of S such that key[x] = k. Otherwise returns NIL Insert(S,x) = add element with pointer x to S Delete(S,x) = removes element with pointer x from S Minimum(S) = query that returns a pointer to the element of S with the smallest key Maximum(S) = query that returns a pointer to the element of S with the largest key Successor(S,x) = query that returns a pointer to the element with a key that is the next larger than the key(x) Predecessor(S,x) = query that returns a pointer to the element with a key that is the next smaller than the key(x)

Binary Search Trees Binary search trees (BSTs) are data structures that support dictionary operations (Search, Insert and Delete), as well as Minimum, Maximum, Successor, and Predecessor Running time is proportional to h = height of tree (number of edges in longest path from root to a leaf) Balanced BST with n nodes has h ~ lg(n). Extreme case of unbalanced tree is chain of nodes  h = n Performance of codes that use BSTs can be as good as O(lg(n)) or as bad as O(n) Red-Black BSTs are one of many “balanced” search-tree schemes that guarantee O(lg(n)) time for dynamic-set operations in the worst case

Binary-Search-Tree Property BST is a linked data structure Each node represents an object with a key (displayed) and satellite data (not shown) Nodes also contains fields left, right and p that point to left child, right child and parent, respectively (Contain NIL if a child or parent is missing) Root is only node with parent missing For any node x, keys in left subtree of x are no larger than key[x] and keys in right subtree of x are no smaller than key[x]

Examples of BSTs h = number of edges in longest path from root to a leaf h depends on which key is chosen for the root 5 3 2 Keys of BST are sorted to satisfy the BST property: For any node x, keys in left subtree of x are no larger than key[x] and keys in right subtree of x are no smaller than key[x] 5

See text for pseudo codes for Search, Min, Max Successor and Predecessor using BSTs. Lectures focus on Insert and Delete.

Insertion and Deletion When a nodes is added and deleted (as in operation of a priority queue), the binary search tree property must be maintained In algorithm Tree-Insert(T,z), T denotes the tree to which node z is add The key[z] determines where z should be placed in the tree z will be inserted as a leaf to some node y = p(z)  left[z] = right [z] = NIL After finding y, update the fields of y depending on whether z is a left or right child of y

Tree-Insert(T,z) x  root[T] (x will trace a path down tree to the place where z is inserted) y  NIL (y is maintained as the parent of x) while x  NIL (fails when place to insert z has been found) do y  x (candidate for p(z)) if key[z] < key[x] (get new assignment of x) then x  left[x] else x  right[x] (now we have found the parent of z) p[z]  y if y = NIL (T was empty when z was inserted) then root[T]  z else if key[z] < key[y] (update fields of p(z)) then left[y]  z else right[y]  z

Tree-Insert(T,z) x  root[T] y  NIL while x  NIL do y  x if key[z] < key[x] then x  left[x] else x  right[x] p[z]  y if y = NIL then root[T]  z else if key[z] < key[y] then left[y]  z else right[y]  z

Example: Insert a node with key = 13

Tree-Delete(T, z) has 3 cases case 1: z has no children modify p(z) by replacing z by NIL case 2: z has one child splice out z by making a link between p(z) and it’s child case 3: z has two children splice out z’s successor, y. copy key[y] into z How do I find z’s successor?

Successor node with key=15? Successor node with key=6? If the right[x] is not NIL, then successor of x is its left-most node (min of subtree) If right[x] = NIL, walk up the tree to find the lowest ancestor that whose left child is also an ancestor of x (any node on path from root to x). Successor node with key=15? Successor node with key=6? Successor node with key=4? Successor node with key=13?

If the right[x] is not NIL, then successor of x is its left-most node (min of subtree) If right[x] = NIL, walk up the tree to find the lowest ancestor that whose left child is also an ancestor of x (any node on path from root to x). Successor node with key=15 is node with key=17 (min of right subtree) Successor node with key=6 is node with key=7 (min of right subtree) Successor node with key=4 is node with key=6 Successor node with key=13 is node with key=15

Case 1: delete 13 No children

Case 2: delete 16 1 child

Case 3: delete 5 2 children case 3: z has two children splice out z’s successor, y. copy key[y] into z How do I find z’s successor? case 3: z has two children splice out z’s successor, y. copy key[y] into z Case 3: delete 5 2 children

Red-Black Trees Red-Black Tree is a BST with one extra field per node that denotes whether it is “red” or “black” By constraining the way nodes are colored we can ensure that no path from the root to a leaf is more than twice as long as any other (i.e. balanced tree) All leaves (also called external nodes) are black and do not hold keys May be drawn as single node black-height of node x = number of black nodes in any path between node x and a leaf. The leaf, which is black, is included in the count The node x, which may be red, is not included in the count height of node x = h(x) = number of edges in longest path to a leaf h(T) = h(root)

R-B BST with black heights shown

R-B tree with “sentinel” node All leaves and parent of root

Note: h(x) < 2bh(x) Properties of red-black trees: 1. Every node is either red or black 2. Root is black 3. nil[T] is black 4. If node is red, its children are black 5. All paths from any node to nil[T] have the same number of black nodes (unique bh). Note: h(x) < 2bh(x) h(x) = # edges in longest path to Nil(T)

No restrictions on children of black node 21-1=1 0, 1 or 2 can be red IN=1 22-1=3, IN=7 22-1=3, IN=4 21-1=1, IN=3 A subtree at x has at least 2bh(x) – 1 internal nodes (IN) The more red nodes in subtree, greater deviation from lower bound of 2bh(x) – 1 internal nodes

Proof by induction that subtree at x has at least 2bh(x) – 1 internal nodes Base: x = nil[T], bh(x) = 0, 2bh(x)-1 = 0, no internal nodes in subtree IH: Assume the subtrees of children at x have at least 2bh(child) -1 nodes Analyze case where both children at x are black. Gives the tightest lower bound. bh(black child at x) = bh(x) –1 By IH, each black child at x has at least 2bh(x) -1 – 1 nodes For 2 black children plus the root of subtree, which is x, n= 2(2bh(x) -1 – 1) + 1 = 2bh(x) - 1 nodes

Lemma 13.1: Proof that R-B tree is balanced R-B tree with n internal nodes (i.e. not counting leaves) has a height of at most 2lg(n+1) By property (4) of R-B trees (if node is red, its children are black), at least half of the nodes on the longest path between the root and nil[T] are black. (i.e. bh(root) is at least h(T)/2) Number of nodes in tree, n, is at least 2bh(root) – 1 (inductive proof) n > 2h/2 – 1 n+1 > 2h/2 lg(n +1) > h/2 h < 2 lg(n+1)

Implications of Lemma 13.1: Red-black trees can be used to implement dynamic-set operations Search, Minimum, Maximum, Successor, and Predecessor with running time O(lg n) Tree-Insert and Tree-Delete will also run on red-black trees in O(lg n); however, they may not preserve the red-black tree properties

To restore red-black tree properties after Insert or Delete, we must change color of some nodes and adjust pointer structure Rotation (only update algorithm we will discuss) is a local modification that changes the root of a subtree while preserving the BST property. Left-Rotate(T,x) replaces the root of subtree at x by its right child. x becomes y’s left child so x stays to the left of y y’s left child becomes x’s right child so it stays right of x Right-Rotate(T,y) replaces the root of subtree at y by its left child.

x x y y Example of left rotations in BST Left-Rotate(T,x) replaces the root of subtree at x by its right child. x becomes y’s left child so x stays to the left of y y’s left child becomes x’s right child so it stays right of x y x