1. Algorithms
Sorting-part2

2. Heap Sort

3. Heaps and Priority Queues2 6 5 7 9

4. Queues

5. Queue First-in, First-out (FIFO) structure Operations Sample use
enqueue: insert element at rear
dequeue: remove & return front element
front: return front element
isEmpty: check if the queue has no elements
size: return number of elements in the queue
Sample use
handling requests and reservations

6. wrapped-around configuration
Array-based Queue
Use an array of size N in a circular fashion
Two variables keep track of the front and rear
f index of the front element
r index immediately past the rear element
Array location r is kept empty
normal configuration Q 1 2 r f
wrapped-around configuration Q 1 2 f r

7. Queue Operations We use the modulo operator (remainder of division)
Algorithm size()
return (N - f + r) mod N
Algorithm isEmpty()
return (f = r)
We use the modulo operator (remainder of division) Q 1 2 r f Q 1 2 f r

8. Queue Operations (cont.)
Algorithm enqueue(o)
if size() = N  1 then
throw FullQueueException
else
Q[r]  o
r  (r + 1) mod N
Operation enqueue throws an exception if the array is full
This exception is implementation-dependent Q 1 2 r f Q 1 2 f r

9. Queue Operations (cont.)
Algorithm dequeue()
if isEmpty() then
throw EmptyQueueException
else
o  Q[f]
f  (f + 1) mod N
return o
Operation dequeue throws an exception if the queue is empty
This exception is specified in the queue ADT Q 1 2 r f Q 1 2 f r

10. Linked List Implementation
public class NodeQueue implements Queue {
private Node front;
private Node rear;
private int size; … }
front
rear
null

11. Enqueue
front
rear
null
null

12. Dequeue
front
rear
null
return this object

13. Time Complexity Analysis
enqueue() : O(1)
dequeue() : O(1)
isEmpty() : O(1)
size() : O(1)
front(): O(1)

14. Trees
Make Money Fast!
Stock Fraud
Ponzi Scheme
Bank Robbery

15. Definitions A tree is an abstract data type one entry point, the root
root node
internal nodes
A tree is an abstract data type
one entry point, the root
Each node is either a leaf or an internal node
An internal node has 1 or more children, nodes that can be reached directly from that internal node.
The internal node is said to be the parent of its child nodes
leaf nodes

16. Tree Terminology subtree Root: node without parent (A)
Internal node: node with at least one child (A, B, C, F)
External node (a.k.a. leaf ): node without children (E, I, J, K, G, H, D)
Ancestors of a node: parent, grandparent, grand-grandparent, etc.
Depth of a node: number of ancestors
Height of a tree: maximum depth of any node (3)
Descendant of a node: child, grandchild, grand-grandchild, etc.
Subtree: tree consisting of a node and its descendants A B D C G H E F I J K
subtree

17. Binary Tree A binary tree is a tree with the following properties:
Each internal node has two children left child and right child
The children of a node are an ordered pair
Alternative recursive definition: a binary tree is either
a tree consisting of a single node, or
a tree whose root has an ordered pair of children, each of which is a binary tree
Applications:
arithmetic expressions
decision processes
searching A B C D E F G H I

18. Arithmetic Expression Tree
Binary tree associated with an arithmetic expression
internal nodes: operators
external nodes: operands
Example: arithmetic expression tree for the expression (2  (a - 1) + (3  b)) +  - 2 a 1 3 b

19. Decision Tree Binary tree associated with a decision process
internal nodes: questions with yes/no answer
external nodes: decisions
Example: dining decision
Want a fast meal?
Yes No
How about coffee?
On expense account?
Yes No
Yes No
Starbucks
Spike’s
Al Forno
Café Paragon

20. Inorder Traversal
In an inorder traversal a node is visited after its left subtree and before its right subtree
Application: draw a binary tree
x(v) = inorder rank of v
y(v) = depth of v
Algorithm inOrder(v)
if isInternal (v)
inOrder (leftChild (v))
visit(v)
inOrder (rightChild (v)) 6 2 8 1 4 7 9 3 5

21. Data Structure for Binary Trees
A node is represented by an object storing
Element
Parent node
Left child node
Right child node  B   A D B A D     C E C E

22. Priority Queues
Sell
100
IBM
$122
300
$120
Buy
500
$119
400
$118

23. Priority queue A stack is first in, last out
A queue is first in, first out
A priority queue is least-first-out
The “smallest” element is the first one removed
(You could also define a largest-first-out priority queue)
If there are several “smallest” elements, the implementer must decide which to remove first
Remove any “smallest” element (don’t care which)
Remove the first one added

24. The Priority Queue ADT
A prioriy queue P supports the following methods:
-size(): Return the number of elements in P
-isEmpty(): Test whether P is empty
-insertItem(k,e): Insert a new element e with key k into P
-minElement(): Return (but don’t remove) an element of P with smallest key; an error occurs if P is empty.
-minKey(): Return the smallest key in P; an error occurs if P is empty
-removeMin(): Remove from P and return an element with the
smallest key; an error condidtion occurs if P is empty.

25. What is a heap
A heap is a binary tree storing keys at its internal nodes and satisfying the following properties:
Heap-Order: for every internal node v other than the root, key(v)  key(parent(v))
Complete Binary Tree: let h be the height of the heap
for i = 0, … , h - 1, there are 2i nodes of depth i
at depth h - 1, the internal nodes are to the left of the external nodes
The last node of a heap is the rightmost internal node of depth h - 1 2 5 6 9 7
last node

26. Binary Heap: Definition
Almost complete binary tree.
filled on all levels, except last, where filled from left to right
Min-heap ordered.
every child greater than (or equal to) parent 06 14 45 78 18 47 53 83 91 81 77 84 99 64

27. Binary Heap: Properties
Min element is in root.
Heap with N elements has height = log2 N. 06
N = 14 Height = 3 14 45 78 18 47 53 83 91 81 77 84 99 64

28. Heaps and Priority Queues
We can use a heap to implement a priority queue
We store a (key, element) item at each internal node
We keep track of the position of the last node
For simplicity, we show only the keys in the pictures
(2, Ahmed)
(5, Hassan)
(6, Mona)
(9, Sara)
(7, Peter)

29. Insertion into a Heap
Method insertItem of the priority queue corresponds to the insertion of a key k to the heap
The insertion algorithm consists of three steps
Find the insertion node z (the new last node)
Store k at z and expand z into an internal node
Restore the heap-order property (discussed next) 2 6 5 7 9 z
insertion node 2 5 6 z 9 7 1

30. Upheap
After the insertion of a new key k, the heap-order property may be violated
Algorithm upheap restores the heap-order property by swapping k along an upward path from the insertion node
Upheap terminates when the key k reaches the root or a node whose parent has a key smaller than or equal to k
Since a heap has height O(log n), upheap runs in O(log n) time 2 1 5 1 5 2 z z 9 7 6 9 7 6

31. Binary Heap: Properties
Min element is in root.
Heap with N elements has height = log2 N. 06
N = 14 Height = 3 14 45 78 18 47 53 83 91 81 77 84 99 64

32. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence 06 14 45 78 18 47 53 42
next free slot 83 91 81 77 84 99 64

33. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 06 14 45 78 18 47 53 42 42 83 91 81 77 84 99 64

34. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 06 14 45 78 18 47 42 42 83 91 81 77 84 99 64 53

35. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
O(log N) operations.
stop: heap ordered 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

36. Binary Heap: Decrease Key
Decrease key of element x to k.
Bubble up until it's heap ordered.
O(log N) operations. 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

37. Removal from a Heap
Method removeMin of the priority queue corresponds to the removal of the root key from the heap
The removal algorithm consists of three steps
Replace the root key with the key of the last node w
Compress w and its children into a leaf
Restore the heap-order property (discussed next) 2 6 5 7 9 w
last node 7 5 6 w 9

38. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

39. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted 53 14 42 78 18 47 45 83 91 81 77 84 99 64 06

40. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
exchange with left child 53 14 42 78 18 47 45 83 91 81 77 84 99 64

41. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
exchange with right child 14 53 42 78 18 47 45 83 91 81 77 84 99 64

42. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
O(log N) operations.
stop: heap ordered 14 18 42 78 53 47 45 83 91 81 77 84 99 64

43. Updating the Last Node
The insertion node can be found by traversing a path of O(log n) nodes
Go up until a left child or the root is reached
If a left child is reached, go to the right child
Go down left until a leaf is reached
Similar algorithm for updating the last node after a removal

44. Using a heap to sort
Using a heap-based priority queue, sorting can be done in O( n log n ) time
Insert all n elements in the priority queue
Repeatedly remove the elements from the priority queue (they will come out sorted)
2n operations on the priority queue, each taking O( log n ) time => O( n log n )
Sorting can be done within the array by treating the array as a heap

45. Heap sort example 6 10 5 12 3 9 20 2 15 8 18 6 10 5 9 20 12 3 2 15 8 18

46. Phase 1: build the heap for i  1 to n-1 do
insert element s[i] into the heap consisting of the elements s[0]…s[i-1]
- Once the heap is built, s[0] will contain the maximum element

47. Phase 1: build heap 6 10 5 9 20 12 3 2 15 8 18

48. Phase 1: build heap 10 6 5 9 20 12 3 2 15 8 18

49. Phase 1: build heap 10 6 5 9 20 12 3 2 15 8 18

50. Phase 1: build heap 12 10 5 9 20 6 3 2 15 8 18

51. Phase 1: build heap 12 10 5 9 20 6 3 2 15 8 18

52. Phase 1: build heap 12 10 9 5 20 6 3 2 15 8 18

53. Phase 1: build heap 20 10 12 5 9 6 3 2 15 8 18

54. Phase 1: build heap 20 10 12 5 9 6 3 2 15 8 18

55. Phase 1: build heap 20 15 12 5 9 10 3 2 6 8 18

56. Phase 1: build heap 20 15 12 5 9 10 8 2 6 3 18

57. Phase 1: build heap 20
Maximum element 18 12 5 9 10 15 2 6 3 8

58. Phase 2: repeatedly select max
for i  n-1 down to 1 do
swap s[0] and s[i] “demote” s[0] to its proper place in the heap consisting of the elements s[0]...s[i-1]

59. Phase 2: select max 20 18 12 5 9 10 15 2 6 3 8

60. Phase 2: select max 8 18 12 5 9 10 15 2 6 3 20

61. Phase 2: select max 18 15 12 5 9 10 8 2 6 3 20

62. Phase 2: select max 18 15 12 5 9 10 8 2 6 3 20

63. Phase 2: select max 3 15 12 5 9 10 8 2 6 18 20

64. Phase 2: select max 15 10 12 5 9 6 8 2 3 18 20

65. Phase 2: select max 15 10 12 5 9 6 8 2 3 18 20

66. Phase 2: select max 3 10 12 5 9 6 8 2 15 18 20

67. Phase 2: select max 12 10 9 5 3 6 8 2 15 18 20

68. Phase 2: select max 12 10 9 5 3 6 8 2 15 18 20

69. Phase 2: select max 2 10 9 5 3 6 8 12 15 18 20

70. Phase 2: select max 10 8 9 5 3 6 2 12 15 18 20

71. Phase 2: select max 3 8 9 5 10 6 2 12 15 18 20

72. Phase 2: select max 9 8 5 3 10 6 2 12 15 18 20

73. Phase 2: select max 3 8 5 9 10 6 2 12 15 18 20

74. Phase 2: select max 8 6 5 9 10 3 2 12 15 18 20

75. Phase 2: select max 2 6 5 9 10 3 8 12 15 18 20

76. Phase 2: select max 6 3 5 9 10 2 8 12 15 18 20

77. Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

78. Phase 2: select max 5 3 2 9 10 6 8 12 15 18 20

79. Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

80. Phase 2: select max 3 2 5 9 10 6 8 12 15 18 20

81. Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

82. Heap sort completed 2 3 5 6 8 9 10 12 15 18 20 2 3 5 9 10 6 8 12 15 18 20

83. In-place Heapification
8.4.2
Now, consider this unsorted array:
This array represents the following complete tree:
This is neither a min-heap, max-heap, or binary search tree

84. In-place Heapification
8.4.3
Let’s work bottom-up: each leaf node is a max heap on its own

85. In-place Heapification
8.4.3
Starting at the back, we note that all leaf nodes are trivial heaps
Also, the subtree with 87 as the root is a max-heap

86. In-place Heapification
8.4.3
The subtree with 23 is not a max-heap, but swapping it with 55 creates a max-heap
This process is termed percolating down

87. In-place Heapification
8.4.3
The subtree with 3 as the root is not max-heap, but we can swap 3 and the maximum of its children: 86

88. In-place Heapification
8.4.3
Starting with the next higher level, the subtree with root 48 can be turned into a max-heap by swapping 48 and 99

89. In-place Heapification
8.4.3
Similarly, swapping 61 and 95 creates a max-heap of the next subtree

90. In-place Heapification
8.4.3
As does swapping 35 and 92

91. In-place Heapification
8.4.3
The subtree with root 24 may be converted into a max-heap by first swapping 24 and 86 and then swapping 24 and 28

92. In-place Heapification
8.4.3
The right-most subtree of the next higher level may be turned into a max-heap by swapping 77 and 99

93. In-place Heapification
8.4.3
However, to turn the next subtree into a max-heap requires that 13 be percolated down to a leaf node

94. In-place Heapification
8.4.3
The root need only be percolated down by two levels

95. In-place Heapification
8.4.3
The final product is a max-heap

96. Black Board Example Sort the following 12 entries using heap sort
8.4.5
Sort the following 12 entries using heap sort
34, 15, 65, 59, 79, 42, 40, 80, 50, 61, 23, 46

97. Heap sort time complexity
for i  1 to n-1 do
insert element s[i] into the heap consisting of the elements s[0]…s[i-1]
for i  n-1 down to 1 do
swap s[0] and s[i] “demote” s[0] to its proper place in the heap consisting of the elements s[0]...s[i-1]
O( n log n )
O( log n ) operations
O( n log n )

98. About heap sort
Build heap phase can be improved to O( n ) if array is rearranged “bottom-up”
Overall complexity still O( n log n ) because of second phase
O( n log n ) time complexity is guaranteed
Note that heap sort is just a more clever version of selection sort since a maximum is repeatedly selected and placed in its proper position

99. Heapify – builds the heap
heapify(Item[]s, int p, int q) { while(p < q) { int c=2*p + 1; // left child if(c > q) break; if(c+1 <= q && s[c] < s[c+1]) ++c; if(s[p] >= s[c]) break; swap(s[p], s[c]); p = c; } }

100. Heapsort
heapsort(Item[]s) { n = s.length; for(j=(n-1)/2, k=n-1; j>=0; --j) heapify(s, j, k); for(j=n-1; j>0; --j) { swap(s[0], s[j]); heapify(s, 0, j-1); } }

101. Problems
Demonstrate, step by step, the operation of Build-Heap on the array
A=[5, 3, 17, 10, 84, 19, 6, 22, 9]