1. Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne Fibonacci Heaps These lecture slides are adapted from CLRS, Chapter 20.

2. 2 Priority Queues make-heap Operation insert find-min delete-min union decrease-key delete 1 Binary log N 1 N 1 Binomial log N 1 Fibonacci † 1 1 log N 1 1 1 Relaxed 1 1 log N 1 1 1 Linked List 1 N N 1 1 N is-empty11111 Heaps this time † amortized

3. 3 Fibonacci Heaps Fibonacci heap history. Fredman and Tarjan (1986) n Ingenious data structure and analysis. n Original motivation: O(m + n log n) shortest path algorithm. – also led to faster algorithms for MST, weighted bipartite matching n Still ahead of its time. Fibonacci heap intuition. n Similar to binomial heaps, but less structured. n Decrease-key and union run in O(1) time. n "Lazy" unions.

4. 4 Fibonacci Heaps: Structure Fibonacci heap. n Set of min-heap ordered trees. 723 30 17 35 2646 24 H 39 41 1852 3 44 min marked

5. 5 Fibonacci Heaps: Implementation Implementation. n Represent trees using left-child, right sibling pointers and circular, doubly linked list. – can quickly splice off subtrees n Roots of trees connected with circular doubly linked list. – fast union n Pointer to root of tree with min element. – fast find-min 723 30 17 35 2646 24 H 39 41 1852 3 44 min

6. 6 Fibonacci Heaps: Potential Function Key quantities. n Degree[x] = degree of node x. n Mark[x] = mark of node x (black or gray). n t(H)= # trees. n m(H)= # marked nodes. n  (H)= t(H) + 2m(H) = potential function. 723 30 17 35 2646 24 H t(H) = 5, m(H) = 3  (H) = 11 39 41 1852 3 44 mindegree = 3

7. 7 Fibonacci Heaps: Insert Insert. n Create a new singleton tree. n Add to left of min pointer. n Update min pointer. 723 30 17 35 2646 24 H 39 41 1852 3 44 min 21 Insert 21

8. 8 Fibonacci Heaps: Insert Insert. n Create a new singleton tree. n Add to left of min pointer. n Update min pointer. 39 41 723 1852 3 30 17 35 2646 24 44 min H 21 Insert 21

9. 9 Fibonacci Heaps: Insert Insert. n Create a new singleton tree. n Add to left of min pointer. n Update min pointer. Running time. O(1) amortized n Actual cost = O(1). n Change in potential = +1. n Amortized cost = O(1). 39 41 7 1852 3 30 17 35 2646 24 44 min H 21 23 Insert 21

10. 10 Fibonacci Heaps: Union Union. n Concatenate two Fibonacci heaps. n Root lists are circular, doubly linked lists. 39 41 717 1852 3 30 23 35 2646 24 44 min H'H'' 21 min

11. 11 Fibonacci Heaps: Union Union. n Concatenate two Fibonacci heaps. n Root lists are circular, doubly linked lists. Running time. O(1) amortized n Actual cost = O(1). n Change in potential = 0. n Amortized cost = O(1). 39 41 717 1852 3 30 23 35 2646 24 44 min H'H'' 21

12. 12 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 411852 3 44 min 1723 30 7 35 2646 24

13. 13 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117231852 30 7 35 2646 24 44 current min

14. 14 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117231852 30 7 35 2646 24 44 current 0123 min

15. 15 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117231852 30 7 35 2646 24 44 current 0123 min

16. 16 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117231852 30 7 35 2646 24 44 current 0123 min

17. 17 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117231852 30 7 35 2646 24 44 current 0123 Merge 17 and 23 trees. min

18. 18 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 4117 23 1852 30 7 35 2646 24 44 current 0123 Merge 7 and 17 trees. min

19. 19 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 417 30 1852 17 35 2646 24 44 current 0123 23 Merge 7 and 24 trees. min

20. 20 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 417 30 1852 23 17 35 2646 2444 current 0123 min

21. 21 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 417 30 1852 23 17 35 2646 2444 current 0123 min

22. 22 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 417 30 1852 23 17 35 2646 2444 current 0123 min

23. 23 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 417 30 1852 23 17 35 2646 2444 current 0123 Merge 41 and 18 trees. min

24. 24 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 41 7 30 1852 23 17 35 2646 24 44 current 0123 min

25. 25 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 41 7 30 1852 23 17 35 2646 24 44 current 0123 min

26. 26 Fibonacci Heaps: Delete Min Delete min. n Delete min and concatenate its children into root list. n Consolidate trees so that no two roots have same degree. 39 41 7 30 1852 23 17 35 2646 24 44 min Stop.

27. 27 Fibonacci Heaps: Delete Min Analysis Notation. n D(n)= max degree of any node in Fibonacci heap with n nodes. n t(H)= # trees in heap H. n  (H)= t(H) + 2m(H). Actual cost. O(D(n) + t(H)) n O(D(n)) work adding min's children into root list and updating min. – at most D(n) children of min node n O(D(n) + t(H)) work consolidating trees. – work is proportional to size of root list since number of roots decreases by one after each merging –  D(n) + t(H) - 1 root nodes at beginning of consolidation Amortized cost. O(D(n)) n t(H')  D(n) + 1 since no two trees have same degree. n  (H)  D(n) + 1 - t(H).

28. 28 Fibonacci Heaps: Delete Min Analysis Is amortized cost of O(D(n)) good? n Yes, if only Insert, Delete-min, and Union operations supported. – in this case, Fibonacci heap contains only binomial trees since we only merge trees of equal root degree – this implies D(n)   log 2 N  n Yes, if we support Decrease-key in clever way. – we'll show that D(n)   log  N , where  is golden ratio –  2 = 1 +  –  = (1 +  5) / 2 = 1.618… – limiting ratio between successive Fibonacci numbers!

29. 29 Decrease key of element x to k. n Case 0: min-heap property not violated. – decrease key of x to k – change heap min pointer if necessary 24 46 17 30 23 7 88 26 21 52 39 18 41 38 Decrease 46 to 45. 72 Fibonacci Heaps: Decrease Key 45 35 min

30. 30 Decrease key of element x to k. n Case 1: parent of x is unmarked. – decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer 24 45 17 30 23 7 88 26 21 52 39 18 41 38 Decrease 45 to 15. 72 Fibonacci Heaps: Decrease Key 15 35 min

31. 31 Decrease key of element x to k. n Case 1: parent of x is unmarked. – decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer 24 15 17 30 23 7 88 26 21 52 39 18 41 38 Decrease 45 to 15. 72 24 Fibonacci Heaps: Decrease Key 35 min

32. 32 Decrease key of element x to k. n Case 1: parent of x is unmarked. – decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer 2417 30 23 7 88 26 21 52 39 18 41 38 Decrease 45 to 15. 24 Fibonacci Heaps: Decrease Key 35 min 15 72

33. 33 35 Decrease key of element x to k. n Case 2: parent of x is marked. – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list  If p[p[x]] unmarked, then mark it.  If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 24 15 17 30 23 7 88 26 21 52 39 18 41 38 Decrease 35 to 5. 72 24 Fibonacci Heaps: Decrease Key 5 min

34. 34 Decrease key of element x to k. n Case 2: parent of x is marked. – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list  If p[p[x]] unmarked, then mark it.  If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 2417 30 23 7 26 21 52 39 18 41 38 Decrease 35 to 5. 24 5 Fibonacci Heaps: Decrease Key 88 parent marked 15 72 min

35. 35 Decrease key of element x to k. n Case 2: parent of x is marked. – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list  If p[p[x]] unmarked, then mark it.  If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 24 26 17 30 23 7 21 52 39 18 41 38 Decrease 35 to 5. 88 24 5 Fibonacci Heaps: Decrease Key 15 72 parent marked min

36. 36 Decrease key of element x to k. n Case 2: parent of x is marked. – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list  If p[p[x]] unmarked, then mark it.  If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 26 17 30 23 7 21 52 39 18 41 38 Decrease 35 to 5. 88 5 Fibonacci Heaps: Decrease Key 1524 72 min

37. 37 Notation. n t(H) = # trees in heap H. n m(H) = # marked nodes in heap H. n  (H) = t(H) + 2m(H). Actual cost. O(c) n O(1) time for decrease key. n O(1) time for each of c cascading cuts, plus reinserting in root list. Amortized cost. O(1) n t(H') = t(H) + c n m(H')  m(H) - c + 2 – each cascading cut unmarks a node – last cascading cut could potentially mark a node n   c + 2(-c + 2) = 4 - c. Fibonacci Heaps: Decrease Key Analysis

38. 38 Delete node x. n Decrease key of x to - . n Delete min element in heap. Amortized cost. O(D(n)) n O(1) for decrease-key. n O(D(n)) for delete-min. n D(n) = max degree of any node in Fibonacci heap. Fibonacci Heaps: Delete

39. 39 Fibonacci Heaps: Bounding Max Degree Definition. D(N) = max degree in Fibonacci heap with N nodes. Key lemma. D(N)  log  N, where  = (1 +  5) / 2. Corollary. Delete and Delete-min take O(log N) amortized time. Lemma. Let x be a node with degree k, and let y 1,..., y k denote the children of x in the order in which they were linked to x. Then: Proof. n When y i is linked to x, y 1,..., y i-1 already linked to x,  degree(x) = i - 1  degree(y i ) = i - 1 since we only link nodes of equal degree n Since then, y i has lost at most one child – otherwise it would have been cut from x n Thus, degree(y i ) = i - 1 or i - 2

40. 40 Fibonacci Heaps: Bounding Max Degree Key lemma. In a Fibonacci heap with N nodes, the maximum degree of any node is at most log  N, where  = (1 +  5) / 2. Proof of key lemma. n For any node x, we show that size(x)   degree(x). – size(x) = # node in subtree rooted at x – taking base  logs, degree(x)  log  (size(x))  log  N. n Let s k be min size of tree rooted at any degree k node. – trivial to see that s 0 = 1, s 1 = 2 – s k monotonically increases with k n Let x* be a degree k node of size s k, and let y 1,..., y k be children in order that they were linked to x*. Assume k  2

41. 41 Fibonacci Facts Definition. The Fibonacci sequence is: n 1, 2, 3, 5, 8, 13, 21,... Slightly nonstandard definition. Fact F1. F k   k, where  = (1 +  5) / 2 = 1.618… Fact F2. Consequence. s k  F k   k. n This implies that size(x)   degree(x) for all nodes x.

42. 42 Golden Ratio Definition. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21,... Definition. The golden ratio  = (1 +  5) / 2 = 1.618… n Divide a rectangle into a square and smaller rectangle such that the smaller rectangle has the same ratio as original one. Parthenon, Athens Greece

43. 43 Fibonacci Facts

44. 44 Fibonacci Numbers and Nature Pinecone Cauliflower

45. 45 Fibonacci Proofs Fact F1. F k   k. Proof. (by induction on k) n Base cases: – F 0 = 1, F 1 = 2  . n Inductive hypotheses: – F k   k and F k+1   k+1 Fact F2. Proof. (by induction on k) n Base cases: – F 2 = 3, F 3 = 5 n Inductive hypotheses:  2 =  + 1

46. 46 On Complicated Algorithms "Once you succeed in writing the programs for [these] complicated algorithms, they usually run extremely fast. The computer doesn't need to understand the algorithm, its task is only to run the programs." R. E. Tarjan