1. Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu

2. CS 477/6772 The Heap Data Structure Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (heap) property: for any node x Parent(x) ≥ x Heap 5 7 8 4 It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 2

3. CS 477/6773 Array Representation of Heaps A heap can be stored as an array A. –Root of tree is A[1] –Left child of A[i] = A[2i] –Right child of A[i] = A[2i + 1] –Parent of A[i] = A[  i/2  ] –Heapsize[A] ≤ length[A] The elements in the subarray A[(  n/2  +1).. n] are leaves The root is the maximum element of the heap A heap is a binary tree that is filled in order

4. CS 477/6774 Maintaining the Heap Property Suppose a node is smaller than a child –Left and Right subtrees of i are max-heaps Invariant: –the heap condition is violated only at that node To eliminate the violation: –Exchange with larger child –Move down the tree –Continue until node is not smaller than children

5. CS 477/6775 Building a Heap Convert an array A[1 … n] into a max-heap ( n = length[A] ) The elements in the subarray A[(  n/2  +1).. n] are leaves Apply MAX-HEAPIFY on elements between 1 and  n/2  2 148 1 16 7 4 3 910 1 23 4567 89 4132169101487 A:

6. CS 477/6776 Heapsort Goal: –Sort an array using heap representations Idea: –Build a max-heap from the array –Swap the root (the maximum element) with the last element in the array –“Discard” this last node by decreasing the heap size –Call MAX-HEAPIFY on the new root –Repeat this process until only one node remains

7. CS 477/6777 Alg: HEAPSORT(A) 1. BUILD-MAX-HEAP (A) 2. for i ← length[A] downto 2 3. do exchange A[1] ↔ A[i] 4. MAX-HEAPIFY (A, 1, i - 1) Running time: O(nlgn) O(n) O(lgn) n-1 times

8. CS 477/6778 Example:A=[7, 4, 3, 1, 2] MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2) MAX-HEAPIFY(A, 1, 1)

9. CS 477/6779 HEAP-MAXIMUM Goal: –Return the largest element of the heap Alg: HEAP-MAXIMUM (A) 1.return A[1] Running time: O(1) Heap A: Heap-Maximum(A) returns 7

10. CS 477/67710 HEAP-EXTRACT-MAX Goal: –Extract the largest element of the heap (i.e., return the max value and also remove that element from the heap Idea: –Exchange the root element with the last –Decrease the size of the heap by 1 element –Call MAX-HEAPIFY on the new root, on a heap of size n-1 Heap A: Root is the largest element

11. CS 477/67711 HEAP-EXTRACT-MAX Alg: HEAP-EXTRACT-MAX (A, n) 1. if n < 1 2. then error “heap underflow” 3. max ← A[1] 4. A[1] ← A[n] 5. MAX-HEAPIFY (A, 1, n-1) remakes heap 6. return max Running time: O(lgn)

12. CS 477/67712 Example: HEAP-EXTRACT-MAX 8 24 14 7 1 16 10 93 max = 16 8 24 14 7 1 10 93 Heap size decreased with 1 4 21 8 7 14 10 93 Call MAX-HEAPIFY (A, 1, n-1)

13. CS 477/67713 HEAP-INCREASE-KEY Goal: –Increases the key of an element i in the heap Idea: –Increment the key of A[i] to its new value –If the max-heap property does not hold anymore: traverse a path toward the root to find the proper place for the newly increased key 8 2 4 14 7 1 16 10 93 i Key [i] ← 15

14. CS 477/67714 HEAP-INCREASE-KEY Alg: HEAP-INCREASE-KEY (A, i, key) 1. if key < A[i] 2. then error “new key is smaller than current key” 3. A[i] ← key 4. while i > 1 and A[PARENT(i)] < A[i] 5. do exchange A[i] ↔ A[PARENT(i)] 6. i ← PARENT(i) Running time: O(lgn) 8 2 4 14 7 1 16 10 93 i Key [i] ← 15

15. CS 477/67715 Example: HEAP-INCREASE-KEY 14 28 15 7 1 16 10 93 i 8 2 4 14 7 1 16 10 93 i Key [ i ] ← 15 8 2 15 14 7 1 16 10 93 i 15 28 14 7 1 16 10 93 i

16. CS 477/67716 -- MAX-HEAP-INSERT Goal: –Inserts a new element into a max- heap Idea: –Expand the max-heap with a new element whose key is -  –Calls HEAP-INCREASE-KEY to set the key of the new node to its correct value and maintain the max-heap property 8 24 14 7 1 16 10 93 15 8 24 14 7 1 16 10 93

17. CS 477/67717 MAX-HEAP-INSERT Alg: MAX-HEAP-INSERT (A, key, n) 1. heap-size[A] ← n + 1 2. A[n + 1] ← -  3. HEAP-INCREASE-KEY (A, n + 1, key) Running time: O(lgn) -- 8 24 14 7 1 16 10 93

18. CS 477/67718 Example: MAX-HEAP-INSERT -- 8 24 14 7 1 16 10 93 Insert value 15: - Start by inserting -  15 8 24 14 7 1 16 10 93 Increase the key to 15 Call HEAP-INCREASE-KEY on A[11] = 15 7 8 24 14 15 1 16 10 93 7 8 24 15 14 1 16 10 93 The restored heap containing the newly added element

19. CS 477/67719 Summary We can perform the following operations on heaps: –MAX-HEAPIFY O(lgn) –BUILD-MAX-HEAP O(n) –HEAP-SORT O(nlgn) –MAX-HEAP-INSERT O(lgn) –HEAP-EXTRACT-MAX O(lgn) –HEAP-INCREASE-KEY O(lgn) –HEAP-MAXIMUM O(1)

20. CS 477/67720 The Search Problem Find items with keys matching a given search key Applications: Keeping track of customer account information at a bank –Search through records to check balances and perform transactions Keep track of reservations on flights –Search to find empty seats, cancel/modify reservations Search engine –Looks for all documents containing a given word

21. CS 477/67721 Symbol Tables (Dictionaries) Dictionary = data structure that supports two basic operations: insert a new item and return an item with a given key Queries: return information about the set –Search (S, k) –Minimum (S), Maximum (S) –Successor (S, x), Predecessor (S, x) Modifying operations: change the set –Insert (S, k) –Delete (S, k)

22. CS 477/67722 Implementations of Symbol Tables Key-indexed-array –Key values are distinct, small numbers –Store the items in an array, indexed by keys Ordered/unordered arrays Ordered/unordered linked lists InsertSearch ordered array ordered list unordered array unordered list N N N N 1 1 N N key-indexed array 1 1

23. CS 477/67723 Binary Search Trees Support many dynamic set operations –SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT Running time of basic operations on binary search trees –On average:  (lgn) The expected height of the tree is lgn –In the worst case:  (n) The tree is a linear chain of n nodes

24. CS 477/67724 Binary Search Trees Tree representation: –A linked data structure in which each node is an object Node representation: –Key field –Satellite data –Left: pointer to left child –Right: pointer to right child –p: pointer to parent ( p [root [T]] = NIL ) Satisfies the binary-search-tree property Left child Right child LR parent keydata

25. CS 477/67725 Binary Search Tree Example Binary search tree property: –If y is in left subtree of x, then key [y] ≤ key [x] –If y is in right subtree of x, then key [y] ≥ key [x] 2 3 5 5 7 9

26. CS 477/67726 Traversing a Binary Search Tree Inorder tree walk: –Prints the keys of a binary tree in sorted order –Root is printed between the values of its left and right subtrees: left, root, right Preorder tree walk: –root printed first: root, left, right Postorder tree walk: left, right, root –root printed last 2 3 5 5 7 9 Preorder: 5 3 2 5 7 9 Inorder: 2 3 5 5 7 9 Postorder: 2 5 3 9 7 5

27. CS 477/67727 Traversing a Binary Search Tree Alg: INORDER-TREE-WALK (x) 1. if x  NIL 2. then INORDER-TREE-WALK ( left [x] ) 3. print key [x] 4. INORDER-TREE-WALK ( right [x] ) E.g.: Running time: –  (n), where n is the size of the tree rooted at x 2 3 5 5 7 9 Output: 2 3 5 5 7 9

28. CS 477/67728 Searching for a Key Given a pointer to the root of a tree and a key k : –Return a pointer to a node with key k if one exists –Otherwise return NIL Idea –Starting at the root: trace down a path by comparing k with the key of the current node: If the keys are equal: we have found the key If k < key[x] search in the left subtree of x If k > key[x] search in the right subtree of x 2 3 4 5 7 9

29. CS 477/67729 Searching for a Key Alg: TREE-SEARCH (x, k) 1. if x = NIL or k = key [x] 2. then return x 3. if k < key [x] 4. then return TREE-SEARCH (left [x], k ) 5. else return TREE-SEARCH (right [x], k ) Running Time: O (h), h – the height of the tree 2 3 4 5 7 9

30. CS 477/67730 Example: TREE-SEARCH Search for key 13: –15  6  7  13 3 24 6 7 13 15 18 1720 9

31. CS 477/67731 Iterative Tree Search Alg: ITERATIVE-TREE-SEARCH (x, k) 1. while x  NIL and k  key [x] 2. do if k < key [x] 3. then x  left [x] 4. else x  right [x] 5. return x

32. CS 477/67732 Finding the Minimum in a Binary Search Tree Goal: find the minimum value in a BST –Following left child pointers from the root, until a NIL is encountered Alg: TREE-MINIMUM (x) 1. while left [x]  NIL 2. do x ← left [x] 3. return x Running time: O(h), h – height of tree 3 2 4 6 7 13 15 18 1720 9 Minimum = 2

33. CS 477/67733 Finding the Maximum in a Binary Search Tree 3 24 6 7 13 15 18 17 20 9 Maximum = 20 Goal: find the maximum value in a BST –Following right child pointers from the root, until a NIL is encountered Alg: TREE-MAXIMUM(x) 1. while right [x]  NIL 2. do x ← right [x] 3. return x Running time: O(h), h – height of tree

34. CS 477/67734 Successor Def: successor ( x ) = y, such that key [y] is the smallest key > key [x] E.g.: successor (15) = successor (13) = successor (9) = Case 1: right (x) is non empty –successor ( x ) = the minimum in right (x) Case 2: right (x) is empty –go up the tree until the current node is a left child: successor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the largest element 3 24 6 7 13 15 18 1720 9 17 15 13

35. CS 477/67735 Finding the Successor Alg: TREE-SUCCESSOR(x) 1. if right [x]  NIL 2. then return TREE-MINIMUM( right [x] ) 3. y ← p[x] 4. while y  NIL and x = right [y] 5. do x ← y 6. y ← p[y] 7. return y Running time: O (h), h – height of the tree 3 24 6 7 13 15 18 1720 9 y x

36. CS 477/67736 Predecessor Def: predecessor ( x ) = y, such that key [y] is the biggest key < key [x] E.g.: predecessor (15) = predecessor (9) = predecessor (13) = Case 1: left (x) is non empty –predecessor ( x ) = the maximum in left (x) Case 2: left (x) is empty –go up the tree until the current node is a right child: predecessor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the smallest element 3 24 6 7 13 15 18 1720 9 13 7 9

37. CS 477/67737 13 Insertion Goal: –Insert value v into a binary search tree Idea: –If key [x] < v move to the right child of x, else move to the left child of x –When x is NIL, we found the correct position –If v < key [y] insert the new node as y ’s left child else insert it as y ’s right child –Begining at the root, go down the tree and maintain: Pointer x : traces the downward path (current node) Pointer y : parent of x (“trailing pointer” ) 2 13 5 9 12 18 1519 17 Insert value 13

38. CS 477/67738 Example: TREE-INSERT 2 13 5 9 12 18 1519 17 x, y=NIL Insert 13: 2 13 5 9 12 18 1519 17 x 2 13 5 9 12 18 15 19 17 x x = NIL y = 15 13 2 13 5 9 12 18 1519 17 y y

39. CS 477/67739 Alg: TREE-INSERT (T, z) 1. y ← NIL 2. x ← root [T] 3. while x ≠ NIL 4. do y ← x 5. if key [z] < key [x] 6. then x ← left [x] 7. else x ← right [x] 8. p[z] ← y 9. if y = NIL 10. then root [T] ← z Tree T was empty 11. else if key [z] < key [y] 12. then left [y] ← z 13. else right [y] ← z 2 13 5 9 12 18 1519 17 13 Running time: O(h)

40. CS 477/67740 Readings Chapter 6 Chapter 12