Binary search trees Definition Binary search trees and dynamic set operations Balanced binary search trees –Tree rotations –Red-black trees Move to front “balancing” technique –Unsorted linked lists –Splay trees

Binary search trees Basic tree property –For any node x left subtree has nodes ≤ x right subtree has nodes ≥ x

BSTs and Dynamic Sets Dynamic set operations and binary search trees –Search(S,k) –Insert(S,x) –Delete(S,x) –Minimum or Maximum(S) –Successor or Predecessor (S,x) –List All(S) –Merge(S 1,S 2 )

Dynamic Set Operations Listall(T)? –time to list? Search(T,k)? –search time? Minimum(T)? Maximum(T)? –search time? Successor(T,x)? Predecessor(T,x)? –Search time Simple Insertion(T,x)

Simple deletion Delete(T,x): Three possible cases: –a) x is a leaf : –b) x has one child : –c) x has two children : Replace x with successor(x). Successor(x) has at most one child (why?); Use step a or b on successor(x)

Simple binary search trees What is the expected height of a binary search tree? Difficult to compute if we allow both insertions and deletions With insertions, analysis of section 12.4 shows that expected height is O(log n) Implications about BSTs as dynamic sets?

Tree-Balancing Algorithms Tree rotations Red-Black Trees Splay Trees Others –AVL Trees –2-3 Trees and Trees

Tree Rotations A B T1 T2T3 B A T1T2 Right Rotate(T,A) Left Rotate(T,B)

Red-Black Trees All nodes in the tree are either red or black. Every null-child is included and colored black. All red nodes must have two black children. Every path from any node x (including the root) to a leaf must have the same number of black nodes. How balanced of a tree will this produce? How hard will it be to maintain?

Example Red-Black Tree

Insertion(T,z) Find place to insert using simple insertion Color node z as red Fix up tree so that it is still a red-black tree What needs to be fixed? –Problem 1: z is root (first node inserted) Minor detail –Problem 2: parent(z) is red

RB-Insert-Fixup Situation: parent(z) is red and z is red –Case 1: uncle(z) is red Then make both uncle(z) and parent(z) black and p(p(z)) red and recurse up tree z z

RB-Insert-Fixup (parent(z) is right child of parent(parent(z))) Situation: parent(z) is red and z is red –Case 2: uncle(z) is black and z is a left child Right rotate to make into case 3 A B T1 T2T3 z B A T1T2 z

RB-Insert-Fixup (parent(z) is right child of parent(parent(z))) Situation: parent(z) is red and z is red –Case 3: uncle(z) is black and z is a left child Left rotate to make B root of tree A B C T1 T2T3 z A B C T1 T2T3

RB-Insert-Fixup Analysis (parent(z) is right child of parent(parent(z))) Situation: parent(z) is red and z is red –Case 1: no rotations, always moving up tree –Cases 2 and 3: At most 2 rotations total and tree ends up balanced No more need to fix up once these cases are met –Total cost: at most 2 rotations and log n operations

Delete(T,z) Find node y to delete using simple deletion Let x be a child of y if such a child exists (otherwise x is a null child) If y is black, fix up tree so that it is still a red- black tree What needs to be fixed? –Problem 1: y was root, so now we might have red root –Problem 2: x and parent(y) are both red –Problem 3: Removal of y violates black height properties of paths that used to go through y

Move To Front (MTF) Technique A powerful “balancing” mechanism is the “move to front” idea This technique is effective in managing both unsorted lists and binary search trees The idea: Whenever an item is accessed (by search or by insertion), it is always moved to the front –In a list, the front is well-defined –In a binary search tree, the front of the tree is the root –A tree that implements this idea is called a splay tree Rotations are not simple single rotations but occur in pairs We give some intuition about the power of MTF

Splay Tree Example

Effectiveness in lists Reference: Amortized efficiency of list update and paging rules, Sleator and Tarjan, CACM 1985 Problem statement: –Suppose you are maintaining an unsorted list where every search must progress from the front of the list to the item (or end of list if search is unsuccessful) –Operations: search, insert, delete Costs: finding or deleting the ith item costs i Inserting a new item costs n+1 Immediately after an insertion or search of an item i, item i may be moved anywhere closer to the front of the list at no extra cost Goal: Find a way to manage list that minimizes total cost of a sequence of operations

Notation for computing costs S: sequence of requests –(insertions, deletions, searches) A: any algorithm for maintaining list –including those those that know S in advance c A (S): cost incurred by algorithm A on sequence S not including paid exchanges x A (S): # of paid exchanges for A on S f A (S): # of free exchanges for A on S Example: –List: 5, 9, 2, 7, 3, 6 and we search for 7 –MTF then has list with 7, 5, 9, 2, 3, 6 –c MTF (S) increases by 4 –x MTF (S) increases by 0 since moving 7 to the front is a free –f MTF (S) increases by 3 since we made 3 free exchanges to move 7

Performance of MTF Thm: For any algorithm A and any sequence S x MTF (S) + c MTF (S) ≤ 2c A (S) + x A (S) – F A (S) – m Observation: x MTF (S) = 0 Interpretation –MTF incurs at most twice the cost of any other algorithm, even those that know the request sequence in advance

Direct Cost Comparison The ideal approach to proving this result is that for each operation, MTF incurs a cost that is at most twice that of algorithm A However, this is clearly not always true Example just before t th operation search(1): –A’s list: 1, 20, 7, 9, 3, 5, 24, 4, 8 A’s cost is just 1 –MTF’s list: 5, 24, 8, 3, 9, 7, 20, 4, 1 MTF’s cost is 9 How can this happen? –Well, since last access to item 1, items 5, 24, 8, 3, 9, 7, 20 and 4 have been accessed. –Thus, A must have done some extra work since last access to 1 in order to have 1 at the current front of the list –This leads to ideas of potential function and amortized analysis

Potential Function Φ Consider A, MTF, and S Let t be the number of operations performed so far (0 ≤ t ≤ |S|) For any t, we define Φ(t) to be the number of inversions between A’s list and MTF’s list –Inversion: a pair of elements x,y s.t. x appears before y in one list and y appears before x in the other list Example –MTF: 1, 7, 2, 5 –A: 2, 1, 5, 7 –Inversions: (1,2), (2,7), (5,7)

Amortized Cost Cost c A (t) is the cost of the t th operation for algorithm A Amortized cost a A (t) of the t th operation is c A (t) + Φ(t) - Φ(t-1) –Cost of t th operation + change in potential fct Key observation Σ t a MTF (t) = Σ t c MTF (t) + Φ(t) - Φ(t-1) = Φ(|S|) - Φ(0) + Σ t c MTF (t) Thus, c MTF (S) = Σ t c MTF (t) = Φ(0) - Φ(|S|) + Σ t a MTF (t) Thus, c MTF (S) ≤ Σ t a MTF (t) Note Φ(0) = 0 and Φ(|S|) ≥ 0

Amortized Cost Comparison Our revised goal is to show that MTF incurs an amortized cost that is at most twice that of algorithm A Example just before t th operation search(1): –A’s list: 1, 20, 7, 9, 3, 5, 24, 4, 8 A’s cost is just 1 –MTF’s list: 5, 24, 8, 3, 9, 7, 20, 4, 1 –MTF’s list afterwards: 1, 5, 24, 8, 3, 9, 7, 20, 4, 1 MTF’s direct cost is 9 The change in potential function is -8 as 8 inversions (all involving 1) are eliminated after 1 is moved to the front of the list MTF’s amortized cost is 1

Amortized Cost Comparison Cont’d General case –We are searching for x which is at position i in A’s list and k in MTF’s list Direct costs –A’s cost to access x is then i –MTF’s cost to access x is k Potential function changes –Let y be the number of items that precede x in MTF’s list but follow x in A’s list –This means k-y-1 items precede x in both lists Note k-y ≤ i. Why? –After x is moved to front of MTF’s list y inversions are eliminated k-y-1 inversions are created –Thus potential function change is k-2y-1 MTF’s amortized cost is thus: k + (k-2y-1) = 2(k-y) -1 ≤ 2i-1 Similar analysis holds for other operations

Splay Tree Performance Analysis of splay trees also uses a potential function and amortized analysis Individual operations may take O(n) time However, it can be shown that any sequence of m operations including n insertions starting with an empty tree take O(m log n) time Static optimality theorem –For any sequence of access operations, a splay tree is asymptotically as efficient as the optimum static search tree (that cannot perform any rotations)

Dynamic Optimality Conjecture Splay trees are as asymptotically fast on any sequence of operations as any other type of search tree with rotations. What does this mean? –Worst case sequence of splay tree operations takes amortized O(log n) time per operation –Some sequences of operations take less. Accessing the same ten items over and over again –Splay tree should then take less on these sequences as well. One special case that has been proven: –search in order from the smallest key to the largest key –the total time for all n operations is O(n)