1. Balanced Trees (AVL and RedBlack)

2. Binary Search Trees Optimal Behavior ▫ O(log 2 N) – perfectly balanced tree (e.g. complete tree with all levels filled) Degenerate Behavior ▫ O(N) – tree becomes list How to avoid degenerate trees? ▫ Keep them perfectly balanced  Impossible ▫ Keep them balanced within some criteria  Maintain balance over insertions and deletions

3. Balanced Trees Balanced binary trees ▫ AVL Tree ▫ Red-Black Tree Balanced N-ary trees ▫ B-Trees ▫ B+-Trees

4. Height of BST A BST is not balanced E.g., 5 12 23 35 85 65

5. AVL Trees Discovered by two Russian mathematicians, Adelson-Velskii and Landis, in 1962. Mathematical Definition : If T is a a nonempty binary search tree, then T is an AVL tree if and only if its left and right subtrees are AVL trees and |left height – right height| < 1. “In English” Definition : A binary search tree that is somewhat balanced, but not necessarily complete or full.

6. AVL Trees? 45 10 2070 30 50 60 10 3 715 9 11 1330

7. AVL Trees Representation: Use the linked representation, with an addition to the node class. The addition is a balance factor ( bf ), which is defined as the following for a node x. bf(x) = height of x’s left subtree – height of x’s right subtree The possible balance factors are –1, 0, and 1.

8. AVL Trees Insertion: Using the standard insertion logic for a binary search tree could cause problems with an AVL tree because it’s possible that some of the properties could be violated. In other words, it’s possible that the tree could become unbalanced.

9. AVL Trees JFK GCM DCAORD +1 0 0 Adding the node “DUS” results in the tree below: JFK GCM DCAORD +2 +1 -20 DUS 0

10. AVL Trees When an insertion is performed, the balancing information for ALL of the nodes on the path back to the root must be updated. If the AVL properties were destroyed, they can be fixed by a simple modification to the tree, which is known as a rotation.

11. AVL Trees To fix the tree, start at the inserted node and trace its path back to the root, stopping at the first node that violates the AVL property (call this node A). There are four possible violations that could occur: 1. Inserted into the left subtree of A’s left child 2. Inserted into the right subtree of A’s right child 3. Inserted into the right subtree of A’s left child 4. Inserted into the left subtree of A’s right child

12. AVL Trees 1. Inserted into the left subtree of A’s left child (lets call this B). This problem is fixed by applying a single right rotation. - Change the parent of A to point to B - Make A’s left link equal to B’s right link - Make B’s right link point to A 10 3 7 A B A 3 7 B

13. AVL Trees 2. Inserted into the right subtree of A’s right child (lets call this B). This problem is fixed by applying a single left rotation. - Change the parent of A to point to B - Make A’s right link equal to B’s left link - Make B’s left link point to A 45 65 55 A B 65 A 45 55 B

14. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. 3 2 1 3 2 13 2 1 4

15. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. 3 2 1 4 5 4 2 1 53

16. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. 4 2 1 53 6 2 1 4 53 6 After step 1 of a single left rotation

17. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. 2 1 4 53 6 2 13 4 5 6 After step 2 of a single left rotation

18. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. After step 3 of a single left rotation 2 13 4 5 6 2 13 4 5 6

19. AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7. 2 13 4 5 6 7 2 13 4 6 57

20. AVL Trees 3. Inserted into the right subtree of A’s left child (lets call this B). This problem is fixed by applying a left-right rotation. - Make A’s left link equal to B’s right link (call this C) - Make B’s right link equal to C’s left link - Make C’s left link equal to B - Change the parent of A to point to C - Make A’s left link equal to C’s right link - Make C’s right link point to A 15 A B 7 12 C 15 A B 12 7 C 15 AB 12 7 C

21. AVL Trees 4. Inserted into the left subtree of A’s right child (lets call this B). This problem is fixed by applying a right-left rotation. - Make A’s right link equal to B’s left link (call this C) - Make B’s left link equal to C’s right link - Make C’s right link equal to B - Change the parent of A to point to C - Make A’s right link equal to C’s left link - Make C’s left link point to A 15 A B 32 21 C 15 A B 21 32 C BA 21 15 C

22. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. 2 13 4 6 57 17 16 2 13 4 6 57 17 16 Right-left rotation

23. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation 2 13 4 6 57 17 16 2 13 4 6 57 17 16

24. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation 2 13 4 6 5 7 17 16 2 13 4 6 57 17 16

25. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation 2 13 4 6 5 7 17 16 2 13 4 6 5 717 16

26. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation 2 13 4 6 5 717 16 15 2 13 4 6 57 16 1715

27. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation 2 13 4 6 5 7 16 1715 2 13 4 6 57 16 1715

28. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. single left rotation 2 13 4 6 5 7 16 1715 13 2 13 4 6 5 7 16 1715 13

29. AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Left - right rotation 2 13 4 6 5 7 16 1714 13 14 2 13 4 6 5 7 16 1715 13 15

30. Red-Black Trees Similar idea to AVL Trees Similar worse case time complexity Keep tree somewhat balanced ▫ not as rigidly balanced as AVL ▫ A Red-Black tree with n nodes has height at most 2 log(n+1), i.e., most of the common operations are performed in O(log n). ▫ Longest path is no more than twice as long as the shortest branch

31. Red-Black Trees A red-black tree is an extended binary tree (every node has two children or is a leaf) which satisfies the following properties: ▫ The root is black (if non-empty)  Root property ▫ If a node is red, it’s parent must be black  Red property ▫ Every path from the root to a leaf/single-child node contains the same number of black nodes. This number is called the black- height of the tree.  Path property

32. Red-Black?

33. no yes

34. Red-Black?

35. no

36. Properties Revisited ▫ The root is black (if non-empty)  Root property ▫ If a node is red, it’s parent must be black  Red property ▫ Every path from the root to a leaf/single-child node contains the same number of black nodes. This number is called the black- height of the tree.  Path property

37. Adding null leaf nodes Every node has 2 children ▫ Don’t include null nodes in the black-height of the tree

38. Properties Revisited Objective: ▫ Restore red property violation while preserving other properties (i.e. path property). ▫ Techniques: re-color and/or rotate.

39. Property Restoration Invariant: ▫ There is exactly one red node x in the tree whose parent may be red.  When a new node is added, it is always added as a red node Strategy: ▫ Fix the violation of red property at x. ▫ This may violate the condition at some ancestor. ▫ Continue fixing the property at the ancestor (ancestor becomes x )

40. Property Restoration Terminology: ▫ x is the current node in violation of the red property  It is red, and its parent is also red ▫ parent( x ) is its parent ▫ parent(parent( x ) ) is its grandparent (i.e. grandparent(x)) ▫ The other sibling of parent( x ) is its uncle (i.e. uncle(x))

41. Property Restoration There are several cases: ▫ Case 1:  X is the root ▫ Case 2:  Parent(x) is black ▫ Case 3:  Parent(x) and Uncle(x) are both red ▫ Case 4:  Parent(x) is red, but Uncle(x) is black  Multiple parts of this case…

42. Property Restoration Case 1: X is the root Change its color to black. xx

43. Property Restoration Case 2: Parent(x) is black Don’t do anything p x p x

44. Property Restoration Case 3: Parent(x) and Uncle(x) are both red Change the color of the parent and its uncle to black. Change the color of the grandparent node to red, and repeat with x = grandparent(x) g up x g up x

45. Property Restoration Case 4: Parent(x) is red, but Uncle(x) is black There are 4 different cases, much like the 4 different AVL cases left-left left-right right-left right-right g up x g up x g pu x g pu x

46. Property Restoration Case 4 cont: left-left and right-right are symmetric left-right and right-left are symmetric left-left left-right right-left right-right g up x g up x g pu x g pu x

47. Property Restoration Case 4 left-left: First rotate the parent RIGHT Then recolor by switching colors of P and G Case 4 right-right is symmetric – simply rotate LEFT instead g u p 3x p gx u1 12 4523 45 p gx u123 45

48. Property Restoration Case 4 left-right: First rotate x LEFT (shown below) then rotate the parent RIGHT and recolor by switching colors of P and G Case 4 right-left is symmetric – simply rotate RIGHT then LEFT g u p 1x g ux p 12 45345

49. Property Restoration Case 4 left-right: First rotate x LEFT (shown previously) then rotate the parent RIGHT and recolor by switching colors of X and G (just like left-left case but where p and x switch rolls) Case 4 right-left is symmetric – simply rotate RIGHT then LEFT g u x 3p x gp u1 12 4523 45 x gp u123 45

50. Insertion Complexity Observations: ▫ Every one of the three cases take constant time. ▫ Case 3 brings x two steps closer to the root and only re-colors nodes without any rotations. ▫ For the multiple cases in 4, one or two rotations plus a re-coloring are required and we are done. Lemma: ▫ An insertion into a red-black tree with n nodes takes O(log 2 n) time and performs at most two rotations.

51. Insertion Example Snap-shots taken from:  http://www.ececs.uc.edu/~franco/C321/html/RedBlack/redblack.html http://www.ececs.uc.edu/~franco/C321/html/RedBlack/redblack.html  Note their rules are numbered differently than our ▫ Insert 50, 30, 70 (recall new nodes always insert as red):  Rule 1 for 50, Rule 2 for 30 and 70 ▫ Null nodes (all black are not shown)

52. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65 ▫ Red property violation  re-color (Case 3: Parent & Uncle are red)  Re-color parent and uncle black while re-coloring grandparent red

53. Insertion Example ▫ Continue with grandparent as X ▫ Root must be black! (Rule 1)  color it black

54. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80 ▫ No violations (rule 2)

55. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75 ▫ Case 3: Uncle & parent are both red  re-color them black and the grandparent red

56. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75 ▫ Continue on recursively with grandparent being x ▫ Case 2: no violations

57. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67 ▫ Case 4: Parent & uncle: diff. color: right-left ▫ Balance: first rotation 67 right, then left

58. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67 ▫ Flip colors of G (65) and X (67)

59. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67

60. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67, 55 ▫ Case 3: parent & uncle same color  re-color them and grandparent

61. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67, 55 ▫ Recursively continue with 67 being the new x ▫ Case 4: 70 and 30 are diff colors: right-left  First rotate 67 right

62. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67, 55 ▫ Then rotate 67 left

63. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67, 55 ▫ Then flip colors of 67 and 50

64. Insertion Example Consider the following actions; ▫ Insert 50, 30, 70, 65, 80, 75, 68, 67, 55 Done!

65. Benefits/Usage of Red-Black Trees Properties ▫ O(log 2 N) and Θ(log 2 N) for search, insert, remove operations Usage ▫ Underlying implementation structure for hash maps, hash sets, multimaps, multisets in Standard Template Library ▫ New scheduler for Linux

66. Comparison of Red-Black and AVL Both are Binary Search Trees Both have the same worst case big-O time complexity for: ▫ Search ▫ Insert ▫ Delete AVL trees are more rigidly balanced ▫ AVL trees are slightly faster for lookup ▫ AVL trees are more costly for insert and delete Most people are using Red-Black trees in their implementations