1. 1 Heaps Chapter 6 in CLRS

2. 2 Motivation Router: Dijkstra’s algorithm for single source shortest path Prim’s algorithm for minimum spanning trees

3. 3 Motivation Want to find the shortest route from New York to San Francisco Model the road-map with a graph

4. 4 A Graph G=(V,E) V is a set of vertices E is a set of edges (pairs of vertices)

5. 5 Model driving distances by weights on the edges 2 19 9 1 5 13 17 10 14 8 21 V is a set of vertices E is a set of edges (pairs of vertices)

6. 6 Source and destination V is a set of vertices E is a set of edges (pairs of vertices) 2 19 9 1 5 13 17 10 14 8 21

7. 7 Dijkstra’s algorithm Assume all weights are non-negative Finds the shortest path from some fixed vertex s to every other vertex

8. 8 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 Example

9. 9 Maintain an upper bound d(v) on the shortest path to v 0 ∞ ∞ ∞ ∞ s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 Example

10. 10 s s1s1 5 3 1 4 2 s3s3 s2s2 s4s4 15 8 Maintain an upper bound d(v) on the shortest path to v 0 ∞ ∞ ∞ ∞ A node is either scanned ( in S) or labeled ( in Q) Initially S =  and Q = V

11. 11 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 Pick a vertex v in Q with minimum d(v) and add it to S 0 ∞ ∞ ∞ ∞ Initially S =  and Q = V

12. 12 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 Pick a vertex v in Q with minimum d(v) and add it to S 0 ∞ ∞ ∞ ∞ Initially S =  and Q = V v w 15 For every edge (v,w) where w in Q relax(v,w)

13. 13 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 Relax(v,w) If d(v) + w(v,w) < d(w) then d(w) ← d(v) + w(v,w) π(v) ← w 0 ∞ ∞ ∞ ∞ v w For every edge (v,w) where w in Q relax(v,w) 15

14. 14 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 ∞ ∞ ∞ ∞ S = {s} Relax(s,s 4 )

15. 15 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 ∞ ∞ ∞ Relax(s,s 4 ) S = {s}

16. 16 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 ∞ ∞ ∞ Relax(s,s 4 ) Relax(s,s 3 ) S = {s}

17. 17 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 ∞ ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) S = {s}

18. 18 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 ∞ ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) Relax(s,s 1 ) S = {s}

19. 19 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) Relax(s,s 1 ) S = {s}

20. 20 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 10 S = {s} Pick a vertex v in Q with minimum d(v) and add it to S

21. 21 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 10 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S

22. 22 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 10 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 )

23. 23 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 )

24. 24 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 ∞ 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 ) Relax(s 1,s 2 )

25. 25 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 ) Relax(s 1,s 2 )

26. 26 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S

27. 27 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 9 S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S

28. 28 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 9 S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 2,s 3 )

29. 29 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 2,s 3 )

30. 30 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S

31. 31 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2,s 3 } Pick a vertex v in Q with minimum d(v) and add it to S

32. 32 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2,s 3,s 4 } Pick a vertex v in Q with minimum d(v) and add it to S

33. 33 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2,s 3,s 4 } When Q =  then the d() values are the distances from s The π function gives the shortest path tree

34. 34 s s1s1 5 10 3 1 4 2 s3s3 s2s2 s4s4 15 8 0 5 6 8 S = {s,s 1,s 2,s 3,s 4 } When Q =  then the d() values are the distances from s The π function gives the shortest path tree

35. 35 Implementation of Dijkstra’s algorithm We need to find efficiently the vertex with minimum d() in Q We need to update d() values of vertices in Q

36. 36 Required ADT Maintain items with keys subject to Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ)

37. 37 Required ADT Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q)

38. 38 Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q) How many times we do these operations ? n = |V| n n m = |E|

39. 39 Do we know an algorithm for this ADT ? Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q) Yes! Use binary search trees, we had insert and delete and also min in the dictionary data type

40. 40 Heap (min) Heap is  A complete binary tree  For any node v, both children’s keys are larger than its key Heap-ordered tree 23 7 17 3365 19 24 2629 7940

41. 41 Representing a heap with an array  Get the elements from top to bottom, from left to right Array Representation 23 7 17 3365 19 24 2629 723172429261965334079 40 Q

42. 42 Array Representation 23 7 17 3365 19 24 2629 723172429261965334079 40 Left(i): 2i+1 Right(i): 2i+2 Parent(i):  (i-1)/2  Q

43. 43 Find the minimum 23 7 17 3365 19 24 2629 723172429261965334079 40 Return Q[0] Q

44. 44 Delete the minimum 23 7 17 3365 19 24 2629 723172429261965334079 40 Q

45. 45 Delete the minimum 2317 3365 19 24 2629 23172429261965334079 40 Q

46. 46 Delete the minimum 23 79 17 3365 19 24 2629 79231724292619653340 Q

47. 47 Delete the minimum 23 17 79 3365 19 24 2629 17237924292619653340 Q

48. 48 Delete the minimum 23 17 19 3365 79 24 2629 17231924292679653340 Q

49. 49 Delete the minimum 23 17 19 3365 79 24 2629 17231924292679653340 Q Q[0] ← Q[size(Q)-1] Heapify-down(Q,0) size(Q) ← size(Q)-1

50. 50 Heapify-down(Q,i) l ← left(i) r ← right(i) smallest ← i if l < size(Q) and Q[l] < Q[smallest] then smallest ← l if r < size(Q) and Q[r] < Q[smallest] then smallest ← r if smallest > i then Q[i] ↔ Q[smallest] Heapify-down(Q, smallest)

51. 51 Inserting an item 23 17 19 3365 79 24 2629 17231924292679653340 Q Insert(15,Q)

52. 52 Inserting an item 23 17 19 3365 79 24 2629 1723192429267965334015 40 Q Insert(15,Q) Q[size(Q)] ← 15 15 Heapify-up(Q,size(Q)) size(Q) ← size(Q) + 1

53. 53 Inserting an item 23 17 19 3365 79 24 2615 1723192415267965334029 40 Q 29

54. 54 Inserting an item 15 17 19 3365 79 24 2623 1715192423267965334029 40 Q 29

55. 55 Inserting an item 17 15 19 3365 79 24 2623 1517192423267965334029 40 Q 29

56. 56 Q Heapify-up Heapify-up(A, i) while i > 0 and A[i] < A[parent(i)] do A[i] ↔ A[parent(i)] i ← parent(i) 17 15 19 3365 79 24 2623 4029 1517192423267965334029

57. 57 Other operations Decrease-key(x,Q,Δ) Delete(x,Q) Can implement them easily using heapify Assume a pointer is given to element x (no need to find!!

58. 58 Heapsort (Williams, Floyd, 1964) Put the elements in an array Make the array into a heap Do a deletemin and put the deleted element at the last position of the array

59. 59 Put the elements in the heap 65 79 26 3323 29 24 1519 796526241915292333407 7 Q

60. 60 65 79 26 3323 29 24 1519 796526241915292333407 7 Q Make the elements into a heap

61. 61 Make the elements into a heap 65 79 26 3323 29 24 1519 796526241915292333407 7 Q Heapify-down(Q,4)

62. 62 65 79 26 3323 29 24 157 796526247152923334019 40 Q Heapify-down(Q,4)

63. 63 65 79 26 3323 29 24 157 796526247152923334019 40 Q Heapify-down(Q,3)

64. 64 65 79 26 3324 29 23 157 796526237152924334019 40 Q Heapify-down(Q,3)

65. 65 79 26 3324 29 23 157 796526237152924334019 40 Q Heapify-down(Q,2)

66. 66 65 79 15 3324 29 23 267 796515237262924334019 40 Q Heapify-down(Q,2)

67. 67 65 79 15 3324 29 23 267 796515237262924334019 40 Q Heapify-down(Q,1)

68. 68 7 79 15 3324 29 23 2665 797152365262924334019 40 Q Heapify-down(Q,1)

69. 69 7 79 15 3324 29 23 2619 797152319262924334065 40 Q Heapify-down(Q,1)

70. 70 7 79 15 3324 29 23 2619 797152319262924334065 40 Q Heapify-down(Q,0)

71. 71 79 7 15 3324 29 23 2619 779152319262924334065 40 Q Heapify-down(Q,0)

72. 72 19 7 15 3324 29 23 2679 719152379262924334065 40 Q Heapify-down(Q,0)

73. 73 19 7 15 3324 29 23 2640 719152340262924337965 79 Q Heapify-down(Q,0)

74. 74 23 7 15 3379 29 24 2619 6540 We have at most n/2 nodes heapified at height 1 How much time it takes to build the heap this way ? We have at most n/4 nodes heapified at height 2 We have at most n/8 nodes heapified at height 3

75. 75 Summary We can build the heap in linear time (we already did this analysis) We still have to deletemin the elements one by one in order to sort that will take O(nlog(n))

76. 76 Comparison to RB / 2-3 We can build the heap in linear time (we already did this analysis) If we start with unsorted elements: Can get percentiles efficiently (e.g. 10 smallest) Constants are smaller than in RB and 2-3 Great for sorting Great for priority queue (can be done also by RB)