1. ADSA: Balanced Trees/12 1 241-423 Advanced Data Structures and Algorithms Objectives – –discuss various kinds of balanced search trees: AVL trees, 2-3-4 trees, Red-Black trees, Semester 2, 2012-2013 12. Balanced Search Trees

2. ADSA: Balanced Trees/12 2 Contents 1. What is a Balanced Binary Search Tree? 2. AVL Trees 3. 2-3-4 Trees 4. Red-Black Trees

3. ADSA: Balanced Trees/12 3 1.What is a Balanced Binary Search Tree? A balanced search tree is one where all the branches from the root have almost the same height. A balanced search tree is one where all the branches from the root have almost the same height. balanced unbalanced continued

4. ADSA: Balanced Trees/12 4 As a tree becomes more unbalanced, search running time decreases from O(log n) to O(n) – –because the tree shape turns into a list We want to keep the binary search tree balanced as nodes are added/removed, so searching/insertion remain fast.

5. ADSA: Balanced Trees/12 5 1.1. Balanced BSTs: AVL Trees An AVL tree maintains height balance – –for each node, the difference in height of its two subtrees is in the range -1 to 1

6. ADSA: Balanced Trees/12 6 1.2. 2-3-4 Trees o o A multiway tree where each node has at most 4 children, and a node can hold up to 3 values. o o A 2-3-4- tree can be perfectly balanced no difference in height between branches requires complex nodes and links

7. ADSA: Balanced Trees/12 7 o o A red-black tree is a binary version of a 2-3-4 tree the nodes have a 'color' attribute: BLACK or RED drawn in Ford and Topp (and here) in white and gray!! the tree maintains a balance measure called the BLACK height BLACK RED 1.3. Red-Black Trees

8. ADSA: Balanced Trees/12 8 1.4. B-Trees A multiway tree where each node has at most m children, and a node can hold up to m-1 values A multiway tree where each node has at most m children, and a node can hold up to m-1 values –a more general version of a 2-3-4 tree B-Trees are most commonly used in databases and filesystems B-Trees are most commonly used in databases and filesystems –most nodes are stored in secondary storage such as hard drives

9. ADSA: Balanced Trees/12 9 2. AVL Trees For each AVL tree node, the difference between the heights of its left and right subtrees is either -1, 0 or +1 – –this is called the balance factor of a node balanceFactor = height(left subtree) - height(right subtree) – –if balanceFactor > 1 or < -1 then the tree is too unbalanced, and needs 'rearranging' to make it more balanced L - R

10. ADSA: Balanced Trees/12 10 Heaviness – –if the balanceFactor is positive, then the node is "heavy on the left" the height of the left subtree is greater than the height of the right subtree – –a negative balanceFactor, means the node is "heavy on the right" continued

11. ADSA: Balanced Trees/12 11 root is heavy on the right, but still balanced root is heavy on the left, but still balanced root is heavy on the right, but still balanced L – R = 0-1 L – R = 2-1 L – R = 1-2

12. ADSA: Balanced Trees/12 12 2.1. The AVLTree Class

13. ADSA: Balanced Trees/12 13 Using AVLTree String[] stateList = {"NV", "NY", "MA", "CA", "GA"}; AVLTree avltreeA = new AVLTree (); for (int i = 0; i < stateList.length; i++) avltreeB.add(stateList[i]); System.out.println("States: " + avltreeA); int[] arr = {50, 95, 60, 90, 70, 80, 75, 78}; AVLTree avltreeB = new AVLTree (); for (int i = 0; i < arr.length; i++) avltreeB.add(arr[i]); // display the tree System.out.println(avltreeB.displayTree(2)); avltreeB.drawTree(2);

14. ADSA: Balanced Trees/12 14 Execution States: [CA, GA, MA, NV, NY] 70(-1) 60(1) 90(1) 50(0) 78(0) 95(0) 75(0) 80(0) (1) (-1) root is heavy on the right, but still balanced

15. ADSA: Balanced Trees/12 15 The AVLTree Node An AVLNode contains the node's value, references to the node's two subtrees, and the node height. height(node) = max ( height(node.left), height(node.right) ) + 1; nodeValue height left right AVTTreeNode object continued

16. ADSA: Balanced Trees/12 16 private static class AVLNode { public T nodeValue; // node data public int height; // child links public AVLNode left, right; public AVLNode (T item) { nodeValue = item; height = 0; left = null; right = null; } nodeValue height left right Use of public is bad; the coding style is due to Ford & Topp

17. ADSA: Balanced Trees/12 17 The addition of a node may cause the tree to go out of balance. – –addNode() adds a node and may reorder nodes as it returns from the adding back to the root reordering is the new idea in AVL trees – –the reordering is done using single and double rotations 2.2. Adding a Node to the Tree

18. ADSA: Balanced Trees/12 18 2.2.1. Too Heavy on the Left 12 25 30 520 11 (2) 12 25 30 520 22 node P or left right Two cases outside grandchild inside grandchild left or right branch -- doesn't matter branch doesn't matter continued L – R = 3-1

19. ADSA: Balanced Trees/12 19 Inserting a node in the left subtree of P (e.g. adding 11 or 22) may cause P to become "too heavy on the left" – –balance factor == 2 The new node can either be in the outside or inside grandchild subtree: – –outside grandchild = left-left – –inside grandchild = left-right continued

20. ADSA: Balanced Trees/12 20

21. ADSA: Balanced Trees/12 21 2.2.2. Too Heavy on the Right 12 25 30 2745 40 (-2) 12 25 30 2745 29 node P or right left right Two cases outside grandchild inside grandchild branch doesn't matter branch doesn't matter continued L – R = 1-3

22. ADSA: Balanced Trees/12 22 Inserting a node in the right subtree of P (e.g. adding 29 or 40) may cause P to become "too heavy on the right" – –balance factor == -2 The new node can either be in the outside or inside grandchild subtree: – –outside grandchild = right-right – –inside grandchild = right-left continued

23. ADSA: Balanced Trees/12 23

24. ADSA: Balanced Trees/12 24 2.3. Single Rotations When a new item is added to the subtree for an outside grandchild, the imbalance is fixed with a single right or left rotation Two cases: – –left outside grandchild (left-left) --> single right rotation – –right outside grandchild (right-right) --> single left rotation

25. ADSA: Balanced Trees/12 25 2.3.1. Single Right Rotation A single right rotation occurs when a new element is added to the subtree of the left outside grandchild (left-left) continued

26. ADSA: Balanced Trees/12 26 add cut left outside grandchild (left-left) continued

27. ADSA: Balanced Trees/12 27 A single right rotation rotates the left child (LC) to replace the parent – –the parent becomes the new right child The right subtree of LC (RGC) is attached as a left child of P – –ok since the nodes in RGC are greater than LC but less than P

28. ADSA: Balanced Trees/12 28 singleRotateRight() // single right rotation on p private static AVLNode singleRotateRight( AVLNode p) { AVLNode lc = p.left; p.left = lc.right; // 1 & 4 on slide 26 lc.right = p; // 2 & 3 p.height = max(height(p.left), height(p.right)) + 1; lc.height = max(height(lc.left), height(rc.right)) + 1; return lc; }

29. ADSA: Balanced Trees/12 29 private static int height(AVLNode t) { if (t == null) return -1; else return t.height; }

30. ADSA: Balanced Trees/12 30 2.3.2. Single Left Rotation A single left rotation occurs when a new element is added to the subtree of the right outside grandchild (right-right). The rotation exchanges the parent (P) and right child (RC) nodes, and attaches the subtree LGC as the right subtree of P. continued

31. ADSA: Balanced Trees/12 31 add cut right outside grandchild (right-right)

32. ADSA: Balanced Trees/12 32 singleRotateLeft() // single left rotation on p private static AVLNode singleRotateLeft( AVLNode p) { AVLNode rc = p.right; p.right = rc.left; // 1 & 4 on slide 31 rc.left = p; // 2 & 3 p.height = max(height(p.left),height(p.right)) + 1; rc.height = max(height(rc.left), height(rc.right)) + 1; return rc; }

33. ADSA: Balanced Trees/12 33 2.4. Double Rotations When a new item is added to the subtree for an inside grandchild, the imbalance is fixed with a double right or left rotation – –a double rotation is two single rotations Two cases: – –left inside grandchild (left-right) --> double right rotation – –right inside grandchild (right-left) --> double left rotation

34. ADSA: Balanced Trees/12 34 Single left rotation about LC Single right rotation about P balanced 2.4.1. A Double Right Rotation left inside grandchild (left-right) Watch RGC rise to the top

35. ADSA: Balanced Trees/12 35 doubleRotateRight() private static AVLNode doubleRotateRight( AVLNode p) /* double right rotation on p is left rotation, then right rotation */ { p.left = singleRotateLeft(p.left); return singleRotateRight(p); }

36. ADSA: Balanced Trees/12 36 Single right rotation about RC Single left rotation about P balanced 2.4.2. A Double Left Rotation right inside grandchild (right-left) P P RC RGC LGC BA LC RC LGC RGC B A P LC RC LGC RGC B A Watch LGC rise to the top

37. ADSA: Balanced Trees/12 37 doubleRotateLeft() private static AVLNode doubleRotateLeft( AVLNode p) /* double left rotation on p is right rotation, then left rotation */ { p.right = singleRotateRight(p.right); return singleRotateLeft(p); }

38. ADSA: Balanced Trees/12 38 2.5. addNode() addNode() recurses down to the insertion point and inserts the node. As it returns, it visits the nodes in reverse order, fixing any imbalances using rotations. It must handle four cases: – –balance height == 2: left-left, left-right – –balance height == -2: right-left, right-right

39. ADSA: Balanced Trees/12 39 Basic addNode() private Node addNode(Node t, T item) { if (t == null) // found insertion point t = new Node (item); else if (((Comparable )item).compareTo(t.nodeValue) < 0) { t.left = addNode( t.left, item); // visit left subtree else if (((Comparable )item).compareTo(t.nodeValue) > 0 ) { else if (((Comparable )item).compareTo(t.nodeValue) > 0 ) { t.right = addNode(t.right, item); // visit right t.right = addNode(t.right, item); // visit right else else throw new IllegalStateException(); // duplicate error throw new IllegalStateException(); // duplicate error return t; return t; } // end of addNode() No AVL rotation code added yet was P in earlier slides

40. ADSA: Balanced Trees/12 40 private AVLNode addNode(AVLNode t, T item) { if(t == null) // found insertion point t = new AVLNode (item); else if (((Comparable )item).compareTo(t.nodeValue) < 0) { // visit left subtree: add node then maybe rotate t.left = addNode( t.left, item); // add node, then... if (height(t.left) - height(t.right) == 2 ) { //too heavy on left if (((Comparable )item).compareTo(t.left.nodeValue) < 0) // problem on left-left t = singleRotateRight(t); else // problem on left-right t = doubleRotateRight(t); // left then right rotation } } : continued AVL rotation code added

41. ADSA: Balanced Trees/12 41 else if (((Comparable )item).compareTo(t.nodeValue) > 0 ) { // visit right subtree: add node then maybe rotate t.right = addNode(t.right, item ); // add node, then... if (height(t.left)-height(t.right) == -2){ //too heavy on right if (((Comparable )item).compareTo(t.right.nodeValue) > 0) // problem on right-right t = singleRotateLeft(t); else // problem on right-left t = doubleRotateLeft(t); // right then left rotation } else // duplicate; throw IllegalStateException throw new IllegalStateException(); // calculate new height of t t.height = max(height(t.left), height(t.right)) + 1; return t; } // end of addNode()

42. ADSA: Balanced Trees/12 42 public boolean add(T item) { try { root = addNode(root, item); // start from root } catch (IllegalStateException e) { return false; } // item is a duplicate // increment the tree size and modCount treeSize++; modCount++; return true; // node was added ok } add() public interface for inserting an item

43. ADSA: Balanced Trees/12 43 2.6. Building an AVL Tree left outside grandchild (left-left) continued gray node is too heavy

44. ADSA: Balanced Trees/12 44 right outside grandchild (right-right) continued 45

45. ADSA: Balanced Trees/12 45 right inside grandchild (right-left) double rotate left (right then left rotation) continued

46. ADSA: Balanced Trees/12 46 left inside grandchild (left-right) double rotate right (left then right rotation) continued

47. ADSA: Balanced Trees/12 47 2.7. Efficiency of AVL Tree Insertion Detailed analysis shows: int(log 2 n)  height < 1.4405 log 2 (n+2) - 1.3277 So the worst case running time for insertion is O(log 2 n). The worst case for deletion is also O(log 2 n).

48. ADSA: Balanced Trees/12 48 AVL Trees48 2.8. Deletion in an AVL Tree Deletion can easily cause an imbalance – –e.g delete 32 44 17 783250 88 48 62 54 44 17 7850 88 48 62 54 before deletion of 32 after deletion

49. ADSA: Balanced Trees/12 49 3. 2-3-4 Trees In a 2-3-4 tree: – –a 2 ‑ node has 1 value and a max of 2 children – –a 3-node has 2 values and a max of 3 children – –a 4-node has 3 values and a max of 4 children The numbers refer to the maximum number of branches that can leave the node same as a binary tree node

50. ADSA: Balanced Trees/12 50 3.1. Searching a 2-3-4 Tree To find an item: – –start at the root and compare the item with all the values in the node; – –if there's no match, move down to the appropriate subtree; – –repeat until you find a match or reach an empty subtree

51. ADSA: Balanced Trees/12 51 Search Example Try finding 9 and 30

52. ADSA: Balanced Trees/12 52 3.2. Inserting into a 2-3-4 Tree Search to the bottom for an insertion node Search to the bottom for an insertion node –2-node at bottom: convert to 3-node –3-node at bottom: convert to 4-node –4-node at bottom: ??

53. ADSA: Balanced Trees/12 53 Splitting 4-nodes Transform tree on the way down: Transform tree on the way down: –ensures last node is not a 4-node –local transformation to split a 4-node Insertion at the bottom is now easy since it's not a 4-node

54. ADSA: Balanced Trees/12 54 Example To split a 4-node. move middle value up. To split a 4-node. move middle value up.

55. ADSA: Balanced Trees/12 55 3.3. Building continued insert 4 This 4-node will be split during the next insertion. This 4-node will be split during the next insertion.

56. ADSA: Balanced Trees/12 56 insert 10 Insertions happen at the bottom. This 4-node will be split during the next insertion.

57. ADSA: Balanced Trees/12 57 continued The insertion point is at level 1, so the new 4-node at level 0 is not split during this insertion. insert 55

58. ADSA: Balanced Trees/12 58 insert 11 This 4-node will be split during the next insertion.

59. ADSA: Balanced Trees/12 59 Another Example insert The search missed the 4-nodes on the left, so not changed.

60. ADSA: Balanced Trees/12 60 3.4. Efficiency of 2-3-4 Trees Searching for an item in a 2-3-4 tree with n elements: – –the max number of nodes visited during the search is int(log 2 n) + 1 Inserting an element into a 2 ‑ 3 ‑ 4 tree: – –requires splitting no more than int(log 2 n) + 1 4-nodes normally requires far fewer splits fast!

61. ADSA: Balanced Trees/12 61 3.5. Drawbacks of 2-3-4 Trees Since any node may become a 4-node, then all nodes must have space for 3 values and 4 links – –but most nodes are not 4-nodes – –lots of wasted memory, unless impl. is fancier Complex nodes and links – –slower to process than binary search trees

62. ADSA: Balanced Trees/12 62 4. Red-Black Trees A red-black tree is a binary search tree where each node has a 'color' – –BLACK or RED A red-black tree is a binary version of a 2-3-4 tree, using different color combinations to represent 3-nodes and 4-nodes. – –a 2-node is already a binary node

63. ADSA: Balanced Trees/12 63 BLACK RED BLACK and RED are drawn in Ford and Topp (and here) in white and gray!!

64. ADSA: Balanced Trees/12 64 4.1. From 2-3-4 Tree Nodes to Red-Black Nodes A 2-node is already a binary node so doesn't need to change its shape. The color of a 2-node is always BLACK (drawn as white in these slides). continued 2-node Conversion

65. ADSA: Balanced Trees/12 65 A 4-node has it's middle value become a BLACK (white) parent and the other values become RED (gray) children. BLACK RED continued 4-node Conversion

66. ADSA: Balanced Trees/12 66 Represent a 3-node as: – –a BLACK parent and a smaller RED left child or – –a BLACK parent and a larger RED right child OR 3-node Conversion

67. ADSA: Balanced Trees/12 67 4.2. Changing a 2-3-4 Tree into a Red-Black Tree change this node continued

68. ADSA: Balanced Trees/12 68 change this node continued

69. ADSA: Balanced Trees/12 69 change these nodes

70. ADSA: Balanced Trees/12 70 4.3. Three Properties of a Red-Black Tree 1. The root must always be BLACK (white in our pictures) 2. A RED parent never has a RED child – –in other words: there are never two successive RED nodes in a path continued that must always be true for the tree to be red-black

71. ADSA: Balanced Trees/12 71 3. Every path from the root to an empty subtree contains the same number of BLACK nodes – –called the black height We can use black height to measure the balance of a red-black tree.

72. ADSA: Balanced Trees/12 72 Check the Example Properties

73. ADSA: Balanced Trees/12 73 4.4. Inserting a Node 1. Search down the tree to find the insertion point, splitting any 4-nodes (a BLACK parent with two RED children) by coloring the children BLACK and the parent RED – –called a color flip Splitting a 4-node may involve additional rotations and color changes – –there are 4 cases to consider (section 4.4.1) continued Three things to do.

74. ADSA: Balanced Trees/12 74 2. Once the insertion point is found, add the new item as a RED leaf node (section 4.4.2) – –this may create two successive RED nodes again use rotation and recoloring to reorder/rebalance the tree 3. Keep the root as a BLACK node.

75. ADSA: Balanced Trees/12 75 4.4.1. Four Cases for Splitting a 4-Node 1234 LL/black parentLR/black parent LL/red parentLR/red parent (also mirror case, RR) G L L G L G L G L R L R (also mirror case, RL) (also mirror case, RR) (also mirror case, RL)

76. ADSA: Balanced Trees/12 76 If the parent is BLACK, only a color flip is needed. If the parent is BLACK, only a color flip is needed. Case 1 (LL/black P): An Example 1 L G L G L G L G L L L

77. ADSA: Balanced Trees/12 77 Case 2 (LR/black P): Insert 55 2 Only a color flip is required for same reason as case 1 L G L G L G L G R R R

78. ADSA: Balanced Trees/12 78 The color-flip creates two successive red nodes -- this breaks property 2, so must be fixed Case 3 (LL / red parent) 3 L L L L

79. ADSA: Balanced Trees/12 79 To fix the red color conflict, carry out a single left or right rotation of node P: To fix the red color conflict, carry out a single left or right rotation of node P: LL  right rotation of P LL  right rotation of P RR (mirror case)  left rotation of P RR (mirror case)  left rotation of P Also change the colors of nodes P and G. Also change the colors of nodes P and G. continued

80. ADSA: Balanced Trees/12 80 and P and G color changes 33 LL of G → right rot of PRR of G → left rot of P L L R R

81. ADSA: Balanced Trees/12 81 Case 3 (LL / red P) as a 2-3-4 Tree 3 X P G ABCD L L

82. ADSA: Balanced Trees/12 82 Case 4 (LR / red parent) The color-flip creates two successive red nodes -- this breaks property 2, so must be fixed 4 L R L R

83. ADSA: Balanced Trees/12 83 To fix the red color conflict, carry out a double left or right rotation of node X: To fix the red color conflict, carry out a double left or right rotation of node X: LR  double right rotation of X LR  double right rotation of X –left then right rotations RL (mirror case)  double left rot of X RL (mirror case)  double left rot of X –right then left rotations Also change the colors of nodes X and G. Also change the colors of nodes X and G.

84. ADSA: Balanced Trees/12 84 LR Example 4 X P X and G recoloured L R left rotation of X right rotation of X

85. ADSA: Balanced Trees/12 85 4 Same Example, with 2-3-4 Views L R

86. ADSA: Balanced Trees/12 86 4.4.2. Inserting a New Item Always add a new item to a tree as a RED leaf node – –this may create two successive RED nodes, which breaks property 2 Fix using single / double rotation and color flip as used for splitting 4-nodes: – –LL/RR  single right/left rotation of parent (P) – –LR/RL  double right/left rot of new node (X)

87. ADSA: Balanced Trees/12 87 Example: Insert 14 single left rotation of 12 and color flip R R

88. ADSA: Balanced Trees/12 88 10 12 Insert 10 instead of 14 R L single left rotation and color flip single right rotation of 10 RL = double left rotation of node 10 (right then left)

89. ADSA: Balanced Trees/12 89 4.5. Building a Red-Black Tree continued single right rotation of 20 and color flip L L

90. ADSA: Balanced Trees/12 90 continued 123 these numbers are from section 4.4. Insert 35 Insert 25 now

91. ADSA: Balanced Trees/12 91 Insert 30 L R LR = double right rotation of node 30 (left then right)

92. ADSA: Balanced Trees/12 92 4.6. Search Running Time The worst ‑ case running time to search a red- black tree or insert an item is O(log 2 n) – –the maximum length of a path in a red-black tree with black height B is 2*B-1 but this cannot happen if the insertion rules are followed

93. ADSA: Balanced Trees/12 93 4.7. Deleting a Node Deletion is more difficult than insertion! – –must usually replace the deleted node – –but no further action is necessary when the replacement node is RED continued replacement (next biggest)

94. ADSA: Balanced Trees/12 94 But deletion requires recoloring and rotations when the replacement node is BLACK. replacement

95. ADSA: Balanced Trees/12 95 4.8. The RBTree Class RBTree implements the Collection interface and uses RBNode objects to create a red-black tree. nodeValue color left right RBNode object parent

96. ADSA: Balanced Trees/12 96

97. ADSA: Balanced Trees/12 97 Ford and Topp's tutorial, "RBTree Class.pdf", provides more explanation of the RBTree class – –local copy at http://fivedots.coe.psu.ac.th/Software.coe/ ADSA/Ford%20and%20Topp/ Includes: – –a discussion of the private data – –explanation of the algorithms for splitting a 4 ‑ node and performing a top ‑ down insertion

98. ADSA: Balanced Trees/12 98 Using the RBTree Class import ds.util.RBTree; public class UseRBTree { public static void main (String[] args) { // 10 values for the red-black tree int[] intArr = {10, 25, 40, 15, 50, 45, 30, 65, 70, 55}; RBTree rbtree = new RBTree (); // load the tree with values for(int i = 0; i < intArr.length; i++) { rbtree.add(intArr[i]); rbtree.drawTrees(4); // display on-going tree } // in a JFrame :

99. ADSA: Balanced Trees/12 99 // display final tree to stdout System.out.println(rbtree.displayTree(2)); /* // remove red-node 25 rbtree.remove(25); rbtree.drawTrees(4); // tree shown in JFrame // remove black-node root 45 rbtree.remove(45); rbtree.drawTree(3); // tree shown in JFrame */ } // end of main() } // end of UseRBTree class

100. ADSA: Balanced Trees/12 100 Execution Tree changes are shown graphically by drawTrees() and drawTree()

101. ADSA: Balanced Trees/12 101 JFrame Output: Add 10 Values 25 40 15 50 split 25; color change 25 single left rot of 25 continued BLACKRED a b started red, then flipped to black

102. ADSA: Balanced Trees/12 102 45 30 65 double right rotation of 45 continued c d split 45

103. ADSA: Balanced Trees/12 103 70 55 single left rotation of 65 split 65; left rotation of 45 e f