1. ΔΟΜΕΣ ΔΕΔΟΜΕΝΩΝ & ΑΡΧΕΙΩΝ
Priority Queues (ουρές προτεραιότητας)
HEAPS (σωροί) – binary heaps
HeapSort (ταξινόμηση σωρού)

2. Priority Queues A priority queue is an ADT where:
Each element has an associated priority
Efficient extraction of the highest-priority element is supported.
Typical operations of a priority queue are:
Enqueue (insert an element)
Dequeue (remove the highest-priority element)
Max (return the highest-priority element)
Meld (join two priority queues into one)
Implementation : Binary Heap

3. Priority Queues: Applications
Scheduling
Examples:
Printer scheduling.
Jobs sent to a printer by faculty have a higher priority than jobs sent by students
Process scheduling.
Several processes require CPU time. Priority may be based on various factors:
Shortest jobs have higher priority
Jobs with a shorter remaining time have higher priority
Jobs that have waited the longest have higher priority
etc.

4. Priority Queues: Applications
Discrete-event simulations
Simulations of systems modeled as collections of entities that interact with one another through events that occur at discrete times.
The priority assigned to an event is its start time.
Examples:
Truck fleet model
Consider the event "Truck leaves city A" with priority 10:00am. Its execution (dequeue) triggers events "Truck arrives at depot" with priority 6:00pm, and "Truck has unloaded" with priority 7:00pm, which are then enqueued, based on their priorities.
Circuit validation

5. Example How many aircraft are there in the air at peak traffic?
How would you represent all the aircraft?
– Arrays
– Linked Lists
– Trees

6. Characteristics of Flights
Created and removed dynamically
– Need to add and remove aircraft in realtime
Different aircraft have different priorities
– Different weights associated with different aircraft
Length of time aircraft has been flying
Emergencies
Special aircraft (military/dignitaries)

7. Array Based Representation
Removing Aircraft
Removing takes 1 step, but copying takes n
•Sorting Aircraft
– Insertion Sort -> n2 -> 25 * 106 operations
– Merge Sort -> n log n -> 20 * 103 operations

8. Heaps and priority queues
A heap is a data structure used to implement an efficient priority queue. The idea is to make it efficient to extract the element with the highest priority ­ the next item in the queue to be processed.
We could use a sorted linked list, with O(1) operations to remove the highest priority node and O(N) to insert a node. Using a tree structure will involve both operations being O(log2N) which is faster.

9. Tree Based Representation

10. Perfect Binary Tree For binary tree with height h
All nodes at levels h–1 or less have 2 children (full)
h = 1
h = 2
h = 3
h = 4

11. Complete Binary Trees For binary tree with height h
All nodes at levels h–2 or less have 2 children (full)
All leaves on level h are as far left as possible
h = 1
h = 3
h = 2

12. Complete Binary Trees
h = 4

13. Complete Trees Nodes can be numbered in level-by-level order:
The left child of the i-th node is the node 2*i and the right child is the node 2*i + 1
The parent of the i-th node is the node i / 2

14. Complete Trees (cont’d)
It is convenient to store a complete binary tree in an array (or an ArrayList) in the order of nodes
Any binary tree can be stored that way, but if the tree is not complete, some gaps (unused locations) will be left in the array.

15. Heaps
A complete binary tree is a full binary tree that has all leaves at the same depth
Is the tree below complete?
Heaps are nearly complete binary trees

16. Heaps
A heap is defined only on “nearly complete binary tress. What is a “nearly complete binary tree”?
A complete binary tree is one where all the internal nodes have exactly 2 children, and all the leaves are on the same level. An internal node comprises all the nodes except the leaf nodes.
Leaf nodes are nodes without children.
 Interior node
Edge 
 Leaf node

17. Heaps
Height of a node: The number of edges starting at that node, and going down to the furthest leaf.
Height of the heap: The maximum number of edges from the root to a leaf.
 Root
Height of root = 3
Height of blue node = 1

18. Heaps
Nearly complete – if not complete, then the only unfilled level is filled in from left to right. 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 1
Parent(i)
return i/2
Parent(i)
return 2i + 1
Left(i)
return 2i

19. Heaps Two key properties Example heap X  Y X  Z X Y Z
Complete binary tree
Value at node
Smaller than or equal to values in subtrees (min-heaps)
Example heap
X  Y
X  Z X Y Z

20. Heap & Non-heap Examples5 5 8 5 45 6 45 5 2 2 6 22 6 6 22 22 25 8 45 25 8 45 25
Heaps
Non-heaps

21. Heap Properties Key operations Key applications Insert ( X )
getSmallest ( )
Key applications
Heapsort
Priority queue

22. Heap Operations – Insert( X )
Algorithm
Add X to end of tree
While (X < parent)
Swap X with parent // X bubbles up tree
Complexity
# of swaps proportional to height of tree
O( log(n) )

23. 2) Compare to parent, swap if parent key larger
Heap Insert Example
Insert ( 20 ) 10 10 10 30 25 30 25 20 25 37 37 20 37 30
1) Insert to end of tree
2) Compare to parent, swap if parent key larger
3) Insert complete

24. 2) Compare to parent, swap if parent key larger
Heap Insert Example
Insert ( 8 ) 10 10 10 8 30 25 30 25 8 25 10 25 37 37 8 37 30 37 30
1) Insert to end of tree
2) Compare to parent, swap if parent key larger
3) Insert complete

25. Heap Operation – getSmallest()
Algorithm
Get (remove) smallest node at root
Replace root with X at end of tree
While ( X > child )
Swap X with smallest child // X drops down tree
Return smallest node
Complexity
# swaps proportional to height of tree
O( log(n) )

26. Heap GetSmallest Example8 30 10 10 25 10 25 30 25 37 30 37 37
1) Replace root with end of tree
2) Compare node to children, if larger swap with smallest child
3) Repeat swap if needed

27. Heap GetSmallest Example8 37 10 10 10 25 10 25 37 25 30 25 30 37 30 30 37
1) Replace root with end of tree
2) Compare node to children, if larger swap with smallest child
3) Repeat swap if needed

28. Binary Heaps Binary heap = a binary tree that is complete
every level except possibly the bottom one is completely filled and the leaves in the bottom level are as far left as possible
satisfies the heap property:
the key stored in every node is greater than or equal to the keys stored in its children
this is called a max-heap. If the key at each node is smaller than or equal to the keys of its children, then we have a min-heap.

29. Heaps
The heap property of a tree is a condition that must be true for the tree to be considered a heap.
Min-heap property: for min-heaps, requires
A[parent(i)]  A[i]
So, the root of any subtree holds the least value in that subtree.
Max-heap property: for max-heaps, requires
A[parent(i)]  A[i]
The root of any subtree holds the greatest value in the subtree.

30. Heaps - example
Looks like a max heap, but max-heap property is violated at indices 11 and 12. Why?
because those nodes’ parents have larger smaller values. 2 8 3 4 5 7 6 1 9 10 11 12

31. Heaps
Both adding and removing an item takes time proportional to the height of the heap (e.g., 20 for 1,000,000 nodes)
The algorithm for removeMin uses the reheap-down procedure
The algorithm for add uses the reheap-up procedure
Remove the root and place the last leaf at the root. Starting at the root, swap the node with its smaller child, as many times as needed to repair the heap.
Add a leaf. Starting at the last leaf, swap the node with its parent as many times as needed to repair the heap.

32. The removeMin Algorithm
Step 1: the root is removed
Step 2: the last leaf replaces the root
Step 3: “reheap down”: the new root value moves down, swapping places with the smaller child

33. Step 1: the new value is added as the last leaf
The add Algorithm
Step 1: the new value is added as the last leaf
Step 2: “reheap up”: the new value moves up, swapping places with its parent

34. Binary Tree -> Array representation
Use Breadth-First-Search
– The root node is A( 1 )
– Node i is A( i )

35. Heaps
Assume we are given a tree represented by matrix A, and such that i  length[A], and the trees rooted at Left(i) and Right(i) are heaps. Then, to create a heap rooted at A[i], use procedure Max-Heapify(A,i):
Max-Heapify(A,i)
1. l  Left(i) ) * l=left*
2. r  Right(i)
3. if l  heap-size[A] and A[l] > A[i]
4. then largest  l
5. else largest  i
6. if r  heap-size[A] and A[r] > A[largest]
7. then largest  r
8. if largest  i
9. then swap( A[i], A[largest])
Max-Heapify(A, largest)

36. Heaps So, what must we do to build a heap?
We call Max-Heapify(A,i) for every i starting at last node and going to the root.
Why top-down?
Because Max-Heapify() moves the larger node upward into the root. If we start at the top and go to larger node indices, the highest a node can move is to the root of that subtree. But what if there is a violation between the subtree’s root and its parent?
So, we must start from the bottom and go towards the root to fix the problems caused by moving larger nodes into the root of subtrees.

37. Max-Heapify(A,1) 20 9 19 18 6 7 17 16 1 10 10 9 19 18 6 7 17 16 1 20 10 9 19 18 6 7 17 16 1 20 19 9 10 18 6 7 17 16 1 20 19 9 10 18 6 7 17 16 1 20 19 9 17 18 6 7 10 16 1 20

38. Heaps In practice, heaps are usually implemented as arrays: A = 16 1410 8 7 9 3 2 4 1
A = 16 14 10 8 7 9 3 2 4 1 =

39. Heaps To represent a complete binary tree as an array:
The root node is A[1]
Node i is A[i]
The parent of node i is A[i/2] (note: integer divide)
The left child of node i is A[2i]
The right child of node i is A[2i + 1] 16 14 10 8 7 9 3 2 4 1
A = 16 14 10 8 7 9 3 2 4 1 =

40. Referencing Heap Elements
So…
Parent(i) { return i/2; }
Left(i) { return 2*i; }
right(i) { return 2*i + 1; }

41. The Heap Property Heaps also satisfy the heap property:
A[Parent( i )]  A[ i ] for all nodes i > 1
In other words, the value of a node is at most the value of its parent
Where is the largest element in a heap stored?

42. Heap Operations: Heapify()
Heapify(): maintain the heap property
Given: a node i in the heap with children l and r
Given: two subtrees rooted at l and r, assumed to be heaps
Problem: The subtree rooted at i may violate the heap property
Action: let the value of the parent node “float down” so subtree at i satisfies the heap property

43. Heapify() Example 16 4 10 14 7 9 3 2 8 1
A = 16 4 10 14 7 9 3 2 8 1

44. Heapify() Example 16 4 10 14 7 9 3 2 8 1
A = 16 4 10 14 7 9 3 2 8 1

45. Heapify() Example 16 4 10 14 7 9 3 2 8 1
A = 16 4 10 14 7 9 3 2 8 1

46. Heapify() Example 16 14 10 4 7 9 3 2 8 1
A = 16 14 10 4 7 9 3 2 8 1

47. Heapify() Example 16 14 10 4 7 9 3 2 8 1
A = 16 14 10 4 7 9 3 2 8 1

48. Heapify() Example 16 14 10 4 7 9 3 2 8 1
A = 16 14 10 4 7 9 3 2 8 1

49. Heapify() Example 16 14 10 8 7 9 3 2 4 1
A = 16 14 10 8 7 9 3 2 4 1

50. Heapify() Example 16 14 10 8 7 9 3 2 4 1
A = 16 14 10 8 7 9 3 2 4 1

51. Heapify() Example 16 14 10 8 7 9 3 2 4 1
A = 16 14 10 8 7 9 3 2 4 1

52. HeapSort

53. Heap Application – Heapsort
Use heaps to sort values
Heap keeps track of smallest element in heap
Algorithm
Create heap
Insert values in heap
Remove values from heap (in ascending order)
Complexity
O( nlog(n))

54. Heapsort Example Input View heap during insert, removal
11, 5, 13, 6, 1
View heap during insert, removal
As tree
As array

55. Heapsort – Insert Values

56. Heapsort – Remove Values

57. Heapsort – Insert in to Array 1
Input
11, 5, 13, 6, 1
Index =
Insert 11 11

58. Heapsort – Insert in to Array 2
Input
11, 5, 13, 6, 1
Index =
Insert
Swap 5 11

59. Heapsort – Insert in to Array 3
Input
11, 5, 13, 6, 1
Index =
Insert

60. Heapsort – Insert in to Array 4
Input
11, 5, 13, 6, 1
Index =
Insert
Swap …

61. Heapsort – Remove from Array 1
Input
11, 5, 13, 6, 1
Index =
Remove root
Replace
Swap w/ child

62. Heapsort – Remove from Array 2
Input
11, 5, 13, 6, 1
Index =
Remove root
Replace
Swap w/ child

63. Example of Heapsort(A)
Illustrate the operation of Heapsort(A) on the array
A = <27, 17, 3, 16, 13, 10, 1, 5, 7, 12, 4, 8, 9, 0>
Note: Since there a 14 nodes, and log 8 = 3 < log 14 < log 16 = 4, there are 4 levels (0,1,2 and 3) in the tree 17 3 16 13 10 1 5 7 12 4 8 9 27

64. 2. for i  length[A] downto 2 3. exchange A[1]  A[i]
Heapsort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
exchange A[1]  A[i]
heap-size[A]  heap-size[A]-1
Max-Heapify(A,1)
Build-Max-Heap(A)
1. heap-size[A]  length[A]
2. for i  length[A]/2 downto 1
do Max-Heapify(A,i) 17 3 16 13 10 1 5 7 12 4 8 9 27 17 10 16 13 3 1 5 7 12 4 8 9 27 17 10 16 13 9 1 5 7 12 4 8 3 27
Build-Max-Heap(A) is done:
A is a max-heap.
Now onto lines 2-5 of Heapsort

65. Heapsort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
exchange A[1]  A[i]
heap-size[A]  heap-size[A]-1
Max-Heapify(A,1) 17 10 16 13 9 1 5 7 12 4 8 3 27
exchange A[1]  A[i]
Max-Heapify(A,1) 17 10 16 13 9 1 5 7 12 4 8 3 27 10 16 13 9 1 5 7 12 4 8 3 17 27 16 10 13 9 1 5 7 12 4 8 3 17 27 16 10 7 13 9 1 5 12 4 8 3 17 27

66. exchange A[1]  A[i]
Max-Heapify(A,1) 16 10 7 13 9 1 5 12 4 8 17 3 27 16 10 7 13 9 1 5 12 4 8 17 3 27 3 10 7 13 9 1 5 12 4 8 17 16 27 13 10 7 3 9 1 5 12 4 8 17 16 27 13 10 7 12 9 1 5 3 4 8 17 16 27

67. exchange A[1]  A[i]
Max-Heapify(A,1) 13 10 7 12 9 1 5 3 4 16 17 8 27 13 10 7 12 9 1 5 3 4 16 17 8 27 8 10 7 12 9 1 5 3 4 16 17 13 27 12 10 7 8 9 1 5 3 4 16 17 13 27 12 10 7 4 9 1 5 3 8 16 17 13 27
etc...
Notice last 3 elements in array are the largest in A, and they are also sorted in increasing order.

68. Example of Heap-Extract-Max(A)17 10 16 13 9 1 5 7 12 4 8 3 27
Example of Heap-Extract-Max(A)
Heap-Extract-Max(A)
1. if heap-size[A] < 1
then error “heap underflow”
3. max  A[1]
4. A[1]  A[heap-size[A] ]
5. heap-size[A]  heap-size[A] - 1
6. Max-Heapify(A,1)
7. return max 17 10 16 13 9 1 5 7 12 4 8 3 10 16 13 9 1 5 7 12 4 8 3 17 16 10 13 9 1 5 7 12 4 8 3 17 16 10 7 13 9 1 5 12 4 8 3 17

69. Example Heap Sort
Let us look at this example: we must convert the unordered array with n = 10 elements into a max-heap
None of the leaf nodes need to be percolated down, and the first non-leaf node is in position n/2
Thus we start with position 10/2 = 5

70. Example Heap Sort
We compare 3 with its child and swap them

71. Example Heap Sort
We compare 17 with its two children and swap it with the maximum child (70)

72. Example Heap Sort
We compare 28 with its two children, 63 and 34, and swap it with the largest child

73. Example Heap Sort
We compare 52 with its children, swap it with the largest
Recursing, no further swaps are needed

74. Example Heap Sort
Finally, we swap the root with its largest child, and recurse, swapping 46 again with 81, and then again with 70

75. Heap Sort Example We have now converted the unsorted array
into a max-heap:

76. Heap Sort Example Suppose we pop the maximum element of this heap
This leaves a gap at the back of the array:

77. Heap Sort Example
This is the last entry in the array, so why not fill it with the largest element?
Repeat this process: pop the maximum element, and then insert it at the end of the array:

78. Heap Sort Example Repeat this process Pop and append 70

79. Heap Sort Example We have the 4 largest elements in order
Pop and append 52
Pop and append 46

80. Heap Sort Example
Continuing...
Pop and append 34
Pop and append 28

81. Heap Sort Example
Finally, we can pop 17, insert it into the 2nd location, and the resulting array is sorted

82. Heapsort For a binary heap, building the heap takes O(N) time.
In a binary heap, deleteMin gives the smallest element.
Running time of O(logN).
Basic strategy for heap sort given N elements
Build a binary heap from N elements - O(N)
Perform N deleteMin operations O(N*logN)
Put the result of deleteMins in a new array.
The new array is sorted.

83. Heapsort The problem is that: A way for using only one array.
This strategy uses a new array
This means waste of space for large N.
A way for using only one array.
When minimum item is deleted, put it to the end of the array.
After N deleteMins:
The original array will have the elements in sorted order from max to min.
If you want items in increasing order (from min to max), then use a Max-Heap.

84. Example Assume the input that is to be sorted:
31, 41, 59, 26, 53, 58, 97
N = 7.
We need first to build a binary heap (a Max-Heap)
A max-Heap is heap that has the largest item in root.

85. Example 97 1 2 3 4 5 6 7 8 9 10 53 59 97 53 59 26 41 58 31 58 31 26 41
deleteMax
Max-Heap after BuildHeap() 59 1 2 3 4 5 6 7 8 9 10 53 58 59 53 58 26 41 31 97 31 97 26 41
Heap after first DeleteMin()

86. Example
After 5 more deleteMins, the array will look like the following. 1 2 3 4 5 6 7 8 9 10 26 31 41 53 58 59 97
This is a sorted array

87. Swap first and last elements
i is running from N/2 to 0
template <class Comparable>
void heapsort( vector<Comparable> & a ) {
/* 1*/ for( int i = a.size( ) / 2; i >= 0; i-- ) /* buildHeap */
/* 2*/ percDown( a, i, a.size( ) );
/* 3*/ for( int j = a.size( ) - 1; j > 0; j-- )
/* 4*/ swap( a[ 0 ], a[ j ] ); /* deleteMax */
/* 5*/ percDown( a, 0, j ); }
j is running from N -1 to 1
Swap first and last elements

88. Point from where to start percolate down
Logical size of heap
template <class Comparable>
void percDown( vector<Comparable> & a, int i, int n ) {
int child;
Comparable tmp;
/* 1*/ for( tmp = a[ i ]; leftChild( i ) < n; i = child )
/* 2*/ child = leftChild( i );
/* 3*/ if( child != n - 1 && a[ child ] < a[ child + 1 ] )
/* 4*/ child++;
/* 5*/ if( tmp < a[ child ] )
/* 6*/ a[ i ] = a[ child ];
else
/* 7*/ break; }
/* 8*/ a[ i ] = tmp;

89. heapsort() 1 2 3 4 5 6 7 8 9 97 53 59 26 41 58 31
j=6
a.size = 7
swap a[0] with a[6] 1 2 3 4 5 6 7 8 9 31 53 59 26 41 58 97
a.size = 7
percolatedown(a, 0, 6)

90. percDown() 1 2 3 4 5 6 7 8 9 31 53 59 26 41 58 97
i=0
tmp = 31, n=6 1 2 3 4 5 6 7 8 9 59 53 26 41 58 97 1 2 3 4 5 6 7 8 9 59 53 26 41 58 97
i=2
tmp = 31, n=6 1 2 3 4 5 6 7 8 9 59 53 58 26 41 31 97
Go out
of for loop!

91. heapsort() 1 2 3 4 5 6 7 8 9 59 53 58 26 41 31 97 j=5 Swap(a[0], a[5])1 2 3 4 5 6 7 8 9 59 53 58 26 41 31 97
j=5
Swap(a[0], a[5]) 1 2 3 4 5 6 7 8 9 31 53 58 26 41 59 97
j=5
percolateDown(a, 0, 5)

92. percDown() 1 2 3 4 5 6 7 8 9 31 53 58 26 41 59 97
i=0 1 2 3 4 5 6 7 8 9 58 53 31 26 41 59 97

93. Applications of Priority Queues
The Selection Problem: find k-th largest number in a list of N elements
Straightforward solution: not efficient, O(N2)
Algorithm A: sort N numbers in decreasing order, then return the kth element – cost O(N2)
Algorithm B: read and sort k numbers in decreasing order, then read the rest elements one after another and compare to the kth element, if larger then replace it the correct place of the remaining k-1 elements. Finally return the kth element – cost O(kN), because k*k (read first k and sort) + k*(N-k) (the remaining N-k is larger and larger). Here k could be N/2, hence total O(N2) in worst case.

94. Solution with priority queue:
Algorithm A: (assume find kth smallest) apply buildHeap algorithm to the array (O(N)), then perform k deleteMin operations (k.O(log N)). The last element extracted from the heap is the answer. Total cost: O(N+k log N). Here k could be N/2, hence total O(NlogN) in worst case.
Algorithm B: the first k elements are placed into the heap in total time O(k) with a call to buildHeap. The time to process each remaining element is O(1), to test if the element goes into S, plus O(log k), to delete Sk and insert the element if this is necessary. Thus, the total time is O(k+(n-k)log k) = O(N log k). It is also O(N log N) in worst case

95. Source code 1

96. Source code 2

97. Source code 3

98. Source code 4

99. Source code 5

100. Source code 6

101. Source code 7