1. Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part R2. Binary Search Trees

2. Prof. Amr Goneid, AUC2 Dictionaries(2): Binary Search Trees Binary Trees Tree Traversal The Binary Search Tree (BST) Deleting Nodes from a BST Binary Search Tree ADT Template Class Specification Other Search Trees

3. Prof. Amr Goneid, AUC3 1. Binary Trees A Tree is a non-linear, hierarchical one-to- many data structure. Examples: Binary Trees, Binary Search Trees (BST) Can be implemented using arrays, structs and pointers Used in problems dealing with: Searching Hierarchy Ancestor/descendant relationship Classification

4. Prof. Amr Goneid, AUC4 Tree Terminology A tree consists of a finite set of nodes (or vertices) and a finite set of edges that connect pairs of nodes. A node is a container for data An edge represents a relationship between two nodes. a node a an edge ba

5. Prof. Amr Goneid, AUC5 Tree Terminology A non-empty tree has a root node (e.g.node a). Every other node can be reached from the root via a unique path (e.g. a-d-f). a is the parent of nodes b and c because there is an edge going from a down to each. Moreover, b and c are children of a. dagbecf root parent of g child of a

6. Prof. Amr Goneid, AUC6 Tree Terminology The descendants of a are (b, c, d, and e), nodes that can be reached by paths from a. The ancestors of e are (c and a), nodes found on the path from e to a. Nodes (b, d, and e) are leaf nodes (they have no children). Each of the nodes a and c has at least one child and is an internal node. caebd Siblings

7. Prof. Amr Goneid, AUC7 Tree Terminology Each node in the tree (except leaves) may have one or more subtrees For a tree with n nodes there are n-1 edges abdecfg

8. Prof. Amr Goneid, AUC8 The Binary Tree A binary tree is a tree in which a parent has at most two children. It consists of a root and two disjoint subtrees (left and right). The root is at level (L = 1) and the height h = maximum level The maximum number of nodes in level L is 2 L-1 abdecf Level 1 Level 2 Level 3 A binary tree of height h = 3 Left Right

9. Prof. Amr Goneid, AUC9 The Full Binary Tree A binary tree is full iff the number of nodes in level L is 2 L-1 abdecfg

10. Prof. Amr Goneid, AUC10 The Full Binary Tree A full binary tree of height h has n nodes, where

11. Prof. Amr Goneid, AUC11 The Full Binary Tree

12. Prof. Amr Goneid, AUC12 The Full Binary Tree The cost of search for nodes in level L is L 2 L-1 Hence, the total search cost is

13. Prof. Amr Goneid, AUC13 The Balanced Tree A balanced binary tree has the property that the heights of the left and right subtrees differ at most by one level. i.e. |h L – h R | ≤ 1 A Full tree is also a balanced tree hLhL hRhR

14. Prof. Amr Goneid, AUC14 A binary tree is Complete iff the number of Nodes at level 1 <= L <= h-1 is 2 L-1 and leaf nodes at level h occupy the leftmost positions in the tree i.e. all levels are filled except the rightmost of the last level. Complete Binary Tree Missing Leaves

15. Prof. Amr Goneid, AUC15 Complete Binary Tree A complete binary tree can be efficiently implemented as an array, where a node at index i has children at indexes 2i and 2i+1 and a parent at index i/2 (with one-based indexing). DE BC A 1 23 4 5 ABCDE

16. Prof. Amr Goneid, AUC16 A binary tree is a recursive structure e.g. it can be defined recursively as: if (not empty tree) 1. It has a root 2. It has a (Left Subtree) 3. It has a (Right Subtree) Recursive structure suggests recursive processing of trees (e.g. Traversal) Binary Tree as a Recursive Structure bc a de f

17. Prof. Amr Goneid, AUC17 Binary Tree as a Recursive Structure b,d,ec,f a de f a,b,c,d,e,f bc Empty

18. Prof. Amr Goneid, AUC18 2. Tree Traversal Traversal is to visit every node ( to display, process, …) exactly once It can be done recursively There are 3 different binary tree traversal orders: Pre-Order: Root is visited before its two subtrees In-Order: Root is visited in between its two subtrees Post-Order:Root is visited after its two subtrees

19. Prof. Amr Goneid, AUC19 Pre-Order Traversal Algorithm: PreOrder ( tree ) { if ( not empty tree) { Visit (root); PreOrder (left subtree); PreOrder (right subtree); } The resulting visit order = {a} {b, d, e} {c, f } bc a de f 1 2 3 5 6 4

20. Prof. Amr Goneid, AUC20 Pre-Order Traversal Pre-Order Traversal is also called Depth-First traversal 1 25 3467

21. Prof. Amr Goneid, AUC21 In-Order Traversal Algorithm: InOrder ( tree ) { if ( not empty tree) { InOrder (left subtree); Visit (root); InOrder (right subtree); } The resulting visit order = {d, b, e} {a} {f, c } bc a de f 4 2 1 6 5 3

22. Prof. Amr Goneid, AUC22 Post-Order Traversal Algorithm: PostOrder ( tree ) { if ( not empty tree) { PostOrder (left subtree); PostOrder (right subtree); Visit (root); } The resulting visit order = {d, e, b} {f, c } {a} bc a de f 6 3 1 5 4 2

23. Prof. Amr Goneid, AUC23 Example: Expression Tree The expression A – B * C + D can be represented as a tree. In-Order traversal gives: A – B * C + D This is the infix representation Pre-Order traversal gives: + - A * B C D This is the prefix representation Post-Order traversal gives: A B C * - D + This is the postfix (RPN) representation - D + A* B C

24. Prof. Amr Goneid, AUC24 http://www.cosc.canterbury.ac.nz/people/ mukundan/dsal/BTree.html Binary Tree Traversal Demo

25. Prof. Amr Goneid, AUC25 3. Binary Search Trees BST

26. Prof. Amr Goneid, AUC26 3. The Binary Search Tree (BST) A Binary Search Tree (BST) is a Dictionary implemented as a Binary Tree. It is a form of container that permits access by content. It supports the following main operations: Insert : Insert item in BST Remove : Delete item from BST Search : search for key in BST

27. Prof. Amr Goneid, AUC27 BST A BST is a binary tree that stores keys or key-data pairs in its nodes and has the following properties: A key identifies uniquely the node (no duplicate keys) If (u, v, w) are nodes such that (u) is any node in the left subtree of (v) and (w) is any node in the right subtree of (v) then: key(u) < key(v) < key(w) vuw

28. Prof. Amr Goneid, AUC28 Examples Of BST

29. Prof. Amr Goneid, AUC29 Examples Of BST These are NOT BSTs.

30. Prof. Amr Goneid, AUC30 Searching Algorithm if (tree is empty) target is not in the tree else if (the target key is the root) target found in root else if (target key smaller than the root’s key) search left sub-tree else search right sub-tree

31. Prof. Amr Goneid, AUC31 Searching for a key Search for the node containing e: Maximum number of comparisons is tree height, i.e. O(h)

32. Prof. Amr Goneid, AUC32 BST Demo http://www.cosc.canterbury.ac.nz/people/ mukundan/dsal/BST.html

33. Prof. Amr Goneid, AUC33 Building a Binary Search Tree Tree created from root downward Item 1 stored in root Next item is attached to left tree if value is smaller or right tree if value is larger To insert an item into an existing tree, we must first locate the item’s parent and then insert

34. Prof. Amr Goneid, AUC34 Algorithm for Insertion if (tree is empty) insert new item as root else if (root key matches item) skip insertion (duplicate key) else if (new key is smaller than root) insert in left sub-tree elseinsert in right sub-tree

35. Prof. Amr Goneid, AUC35 Example: Building a Tree Insert: 40,20,10,50,65,45,30

36. Prof. Amr Goneid, AUC36 Effect of Insertion Order The shape of the tree depends on the order of insertion. Shape determines the height (h) of the tree. Since cost of search is O(h), the insertion order will affect the search cost. The previous tree is full, and h = log 2 (n+1) so that search cost is O(log 2 n)

37. Prof. Amr Goneid, AUC37 Effect of Insertion Order The previous tree would look like a linked list if we have inserted in the order 10,20,30,…. Its height would be h = n and its search cost would be O(n) O(n) O(log n)

38. Prof. Amr Goneid, AUC38 http://www.cosc.canterbury.ac.nz/people/ mukundan/dsal/BSTNew.html Binary Search Tree Demo

39. Prof. Amr Goneid, AUC39 Traversing a Binary Search Tree Recursive inorder traversal of tree with root (t) traverse ( t ) { if (t is not empty) traverse ( t left ); visit (t); traverse ( t right ); }

40. Prof. Amr Goneid, AUC40 Find Minimum Key Find the minimum key in a tree with root (t) Minkey ( t ) { if ( t left is not empty) return MinKey( t left ); else return key(t); }

41. Prof. Amr Goneid, AUC41 Other Traversal Orders Pre-order (a.k.a. Depth-First traversal) can be implemented using an iterative (non-recursive) algorithm. In this case, a stack is used If the stack is replaced by a queue and left pointers are exchanged by right pointers, the algorithm becomes Level- order traversal (a.k.a. Breadth-First traversal)

42. Prof. Amr Goneid, AUC42 Iterative Preorder Traversal void iterative_preorder ( ) { t = root; Let s be a stack s.push (t); while(!s.stackIsEmpty()) {s.pop(t); process(t->key); if ( t right is not empty) s.push(t right ); if ( t left is not empty) s.push(t left ); }

43. Prof. Amr Goneid, AUC43 Pre-Order Traversal Traversal order: {D,B,A,C,F,E,G} D BF ACEG 1 2 3 4 5 67

44. Prof. Amr Goneid, AUC44 Level Order Traversal void levelorder ( ) { t = root; Let q be a queue; q.enqueue(t); while(!q.queueIsEmpty()) {q.dequeue(t); process(t->key); if ( t left is not empty) q.enqueue(t left ); if ( t right is not empty) q.enqueue(t right ); }

45. Prof. Amr Goneid, AUC45 Level-Order Traversal Traversal order: {D,B,F,A,C,E,G} ACEG BF D 1 23 4 5 67

46. Prof. Amr Goneid, AUC46 4. Deleting Nodes from a BST

47. Prof. Amr Goneid, AUC47 Deleting a ROOT Node

48. Prof. Amr Goneid, AUC48 Deleting a ROOT Node

49. Prof. Amr Goneid, AUC49 Deleting a ROOT Node (Special Case)

50. Prof. Amr Goneid, AUC50 Deleting a ROOT Node (Alternative)

51. Prof. Amr Goneid, AUC51 Deleting an Internal Node

52. Prof. Amr Goneid, AUC52 Search for Parent of a Node To delete a node, we need to find its parent. To search for the parent (p) of a node (x) with key (k) in tree (t): Set x = t; p = null; found = false; While (not found) and (x is not empty) { if k < key(x) descend left (i.e. set p = x; x = x left ) else if k > key(x) descend right (i.e. set p = x;x = x right ) else found = true } Notice that: P is null if (k) is in the root or if the tree is empty. If (k) is not found, p points to what should have been its parent.

53. Prof. Amr Goneid, AUC53 Algorithm to remove a Node Let k = key to remove its node t = pointer to root of tree x = location where k is found p = parent of a node sx = inorder successor of x s = child of x

54. Prof. Amr Goneid, AUC54 Algorithm to remove a Node Remove (t,k) { Search for (k) and its parent; If not found, return; else it is found at (x) with parent at (p): Case (x) has two children: Find inorder successor (sx) and its parent (p); Copy contents of (sx) into (x); Change (x) to point to (sx); Now (x) has one or no children and (p) is its parent

55. Prof. Amr Goneid, AUC55 Algorithm to remove a Node Case (x) has one or no children: Let (s) point to the child of (x) or null if there are no children; If p = null then set root to null; else if (x) is a left child of (p), set p left = s; else set p right = s; Now (x) is isolated and can be deleted delete (x); }

56. Prof. Amr Goneid, AUC56 Example: Delete Root 40201030605070 x p = null

57. Prof. Amr Goneid, AUC57 Example: Delete Root 40201030605070 x p sx

58. Prof. Amr Goneid, AUC58 Example: Delete Root 50201030605070 p x S = null

59. Prof. Amr Goneid, AUC59 Example: Delete Root 50201030605070 x null delete

60. Prof. Amr Goneid, AUC60 5. Binary Search Tree ADT Elements: A BST consists of a collection of elements that are all of the same type. An element is composed of two parts: key of and data of Structure: A node in a BST has at most two subtrees. The key value in a node is larger than all the keys in its left subtree and smaller than all keys in its right subtree. Duplicate keys are not allowed.

61. Prof. Amr Goneid, AUC61 Binary Search Tree ADT Data members rootpointer to the tree root Basic Operations binaryTreea constructor insertinserts an item emptychecks if tree is empty searchlocates a node given a key

62. Prof. Amr Goneid, AUC62 Binary Search Tree ADT Basic Operations (continued) retrieveretrieves data given key traversetraverses a tree (In-Order) preorderpre-order traversal levelorderLevel-order traversal removeDelete node given key graphsimple graphical output

63. Prof. Amr Goneid, AUC63 Linked Representation The nodes in the BST will be implemented as a linked structure: left element right 1645 32 40 32 1645 40 t

64. Prof. Amr Goneid, AUC64 Node Specification // The linked structure for a node can be // specified as a Class in the private part of // the main binary tree class. class treeNode// Hidden from user { public: keyType key; // key dataType data;// Data treeNode *left;// left subtree treeNode *right;// right subtree }; // end of class treeNode declaration //A pointer to a node can be specified by a type: typedef treeNode * NodePointer; NodePointer root;

65. Prof. Amr Goneid, AUC65 6. Template Class Specification Because node structure is private, all references to pointers are hidden from user This means that recursive functions with pointer parameters must be private. A public (User available) member function will have to call an auxiliary private function to support recursion. For example, to traverse a tree, the user public function will be declared as: void traverse ( ) const; and will be used in the form: BST.traverse ( );

66. Prof. Amr Goneid, AUC66 Template Class Specification Such function will have to call a private traverse function: void traverse2 (NodePointer ) const; Therefore, the implementation of traverse will be: template void binaryTree ::traverse() const { traverse2(root); } Notice that traverse2 can support recursion via its pointer parameter

67. Prof. Amr Goneid, AUC67 Template Class Specification For example, if we use in-order traversal, then the private traverse function will be implemented as: template void binaryTree ::traverse2 (NodePointer aRoot)const { if (aRoot != NULL) { // recursive in-order traversal traverse2 (aRoot->left); cout key << endl; traverse2 (aRoot->right); } } // end of private traverse

68. Prof. Amr Goneid, AUC68 Template Class Specification All similar functions will be implemented using the same method. For example: Public FunctionPrivate Function insert (key,data)insert2 (pointer,key,data) search(key)search2 (pointer,key) retrieve(key,data)retrieve2 (pointer,key,data) traverse( )traverse2 (pointer)

69. Prof. Amr Goneid, AUC69 BinaryTree.h // FILE: BinaryTree.h // DEFINITION OF TEMPLATE CLASS BINARY SEARCH // TREE #ifndef BIN_TREE_H #define BIN_TREE_H // Specification of the class template class binaryTree {

70. Prof. Amr Goneid, AUC70 BinaryTree.h public: // Public Member functions... // CREATE AN EMPTY TREE binaryTree(); // INSERT AN ELEMENT INTO THE TREE bool insert(const keyType &, const dataType &); // CHECK IF THE TREE IS EMPTY bool empty() const; // SEARCH FOR AN ELEMENT IN THE TREE bool search (const keyType &) const; // RETRIEVE DATA FOR A GIVEN KEY bool retrieve (const keyType &, dataType &) const;

71. Prof. Amr Goneid, AUC71 BinaryTree.h // TRAVERSE A TREE void traverse() const; // Iterative Pre-order Traversal void preorder () const; // Iterative Level-order Traversal void levelorder () const; // GRAPHIC OUTPUT void graph() const; // REMOVE AN ELEMENT FROM THE TREE void remove (const keyType &);.........

72. Prof. Amr Goneid, AUC72 BinaryTree.h private: // Node Class class treeNode { public: keyType key; // key dataType data;// Data treeNode *left;// left subtree treeNode *right;// right subtree }; // end of class treeNode declaration typedef treeNode * NodePointer; // Data member.... NodePointer root;

73. Prof. Amr Goneid, AUC73 BinaryTree.h // Private Member functions... // Searches a subtree for a key bool search2 ( NodePointer, const keyType &) const; //Searches a subtree for a key and retrieves data bool retrieve2 (NodePointer, const keyType &, dataType &) const; // Inserts an item in a subtree bool insert2 (NodePointer &, const keyType &, const dataType &);

74. Prof. Amr Goneid, AUC74 BinaryTree.h // Traverses a subtree void traverse2 (NodePointer ) const; // Graphic output of a subtree void graph2 ( int, NodePointer ) const; // LOCATE A NODE CONTAINING ELEMENT AND ITS // PARENT void parentSearch( const keyType &k, bool &found, NodePointer &locptr, NodePointer &parent) const; }; #endif // BIN_TREE_H #include “binaryTree.cpp”

75. Prof. Amr Goneid, AUC75 The CSCI 321 course web site contains full definitions and implementations of : binaryTree template class: BST class with linked implementation BST Template Class

76. Prof. Amr Goneid, AUC76 Analysis of BST operations Assuming a balanced BST: bool insert O(log n) bool empty() O(1) bool retrieve O(1) bool search O(log n) void traverse O(n) void graph O(n) void remove O(log n)

77. Prof. Amr Goneid, AUC77 7. Other Search Trees Binary Search Trees have worst case performance of O(n), and best case performance of O(log n) There are many other search trees that are balanced trees. Examples are: AVL Trees, Red-Black trees

78. Prof. Amr Goneid, AUC78 AVL Trees Named after its Russian inventors: Adel'son-Vel'skii and Landis (1962)

79. Prof. Amr Goneid, AUC79 AVL Trees An AVL tree is a self balancing binary search tree in which the heights of the right subtree and left subtree of the root differ by at most 1 the left subtree and the right subtree are themselves AVL trees rebalancing is done when insertion or deletion causes violation of the AVL condition.

80. Prof. Amr Goneid, AUC80 AVL Tree Notice that: N(h) = N(h-1) + N(h-2) + 1 h-1 h-2h

81. Prof. Amr Goneid, AUC81 AVL Tree