Chair of Software Engineering Einführung in die Programmierung Introduction to Programming Prof. Dr. Bertrand Meyer Lecture 18: Topological sort Part 1: Problem and math basis Part 2: Algorithm and software solution

2 Part 1 Problem and mathematical basis

by Caran d’Ache

4 “Topological sort” From a given partial order, produce a compatible total order

5 The problem Partial order: ordering constraints between elements of a set, e.g.  “Remove the dishes before discussing politics”  “Walk to Üetliberg before lunch”  “Take your medicine before lunch”  “Finish lunch before removing dishes” Total order: sequence including all elements of set Compatible: the sequence respects all ordering constraints  Üetliberg, Medicine, Lunch, Dishes, Politics : OK  Medicine, Üetliberg, Lunch, Dishes, Politics : OK  Politics, Medicine, Lunch, Dishes, Üetliberg : not OK From a given partial order, produce a compatible total order

6 Why this is an important example  Common problem, occurring in many different areas  Interesting, non-trivial (but not too hard) algorithm  Illustrates techniques of algorithms, data structures, complexity (big-O), and other topics of last lecture  Illustrates software engineering techniques  Illustrates important mathematical concepts: binary relations and in particular order relations  It’s just beautiful! Today: problem and math basis Next time: detailed algorithm and component

7 Topological sort: example uses  From a dictionary, produce list of definitions (“glossary”) such that no word occurs prior to its definition  Produce a complete schedule for carrying out a set of tasks, subject to ordering constraints (This is done for scheduling maintenance tasks for industrial tasks, often with thousands of constraints)  Produce a version of a class with features reordered, so that no call to a feature appears before its declaration

8 Rectangles with overlap constraints B D A C E Constraints: [B, A], [D, A], [A, C], [B, D], [D, C]

9 Displaying rectangles with overlap constraints B D A C E BD EAC Constraints: [B, A], [D, A], [A, C], [B, D], [D, C] Possible display order: B D A C E

10 An example in EiffelStudio To implement x. f with dynamic binding: need routine table Routines  Classes (types)  put LINKED_LIST Routine pointer Void

11 The problem Partial order: ordering constraints between elements of a set, e.g.  “Remove the dishes before discussing politics”  “Walk to Üetliberg before lunch”  “Take your medicine before lunch”  “Finish lunch before removing dishes” Total order: sequence including all elements of set Compatible: the sequence respects all ordering constraints  Üetliberg, Medicine, Lunch, Dishes, Politics : OK  Medicine, Üetliberg, Lunch, Dishes, Politics : OK  Politics, Medicine, Lunch, Dishes, Üetliberg : not OK From a given partial order, produce a compatible total order

12 Pictured as a graph “Remove the dishes before discussing politics” “Walk to Üetliberg before lunch” “Take your medicine before lunch” “Finish lunch before removing dishes” Üetliberg Medicine Lunch Dishes Politics

13 Sometimes there is no solution  “ Introducing recursion requires that students know about stacks”  “You must discuss abstract data types before introducing stacks  “Abstract data types rely on recursion” The constraints introduce a cycle

14 end class ORDERABLE [G ] feature Overall structure (1) Given: Required: An enumeration of the elements, in an order compatible with the constraints A type G A set of elements of type G constraints : LIST [TUPLE [G, G ]] topsort : LIST [G ]... ensure compatible (Result, constraints) A set of constraints between these elements elements : LIST [G ]

15 Some mathematical background...

16 Binary relation on a set Any property that either holds or doesn’t hold between any two elements of the set On a set PERSON of persons, example relations are:  mother : a mother b holds if and only if a is the mother of b  father :  child :  sister :  sibling : (brother or sister) Notation: a r b to express that r holds of a and b. Note: relation names will appear in green

17 Example: the before relation The set of interest: Tasks = {Politics, Lunch, Medicine, Dishes, Üetliberg} The constraining relation: Dishes before Politics Üetliberg before Lunch Medicine before Lunch Lunch before Dishes “Remove the dishes before discussing politics” “Walk to Üetliberg before lunch” “Take your medicine before lunch” “Finish lunch before removing dishes”

18 Some special relations on a set X universal [X ]: holds between any two elements of X id [X ]: holds between every element of X and itself empty [X ]: holds between no elements of X

19 Relations: a more precise mathematical view We consider a relation r on a set P as: A set of pairs in P x P, containing all the pairs [x, y] such that x r y. Then x r y simply means that [x, y]  r See examples on next slides

20 A relation is a set: examples son ={[Charles, Elizabeth], [Charles, Philip], [William, Charles], [Harry, Charles]} id [ ℕ ] ={[0, 0], [1, 1], [2, 2], [3, 3], [4, 4], …} universal [ ℕ ]={[0, 0], [0, 1], [0, 2], [0, 3], [0, 4], … [[1, 0], [1, 1], [1, 2], [1, 3], [1, 4], … [2, 0], [2, 1], [2, 2], [2, 3], [2, 4], … … } = ℕ x ℕ empty [X ]=  ={ } Natural numbers Cartesian product

21 Relations illustrated universal [ ℕ ] 6 id [ ℕ ]

22 Example: the before relation The set of interest: elements = {Politics, Lunch, Medicine, Dishes, Üetliberg} The constraining relation: before={[Dishes, Politics], [Üetliberg, Lunch], [Medicine, Lunch], [Lunch, Dishes]} “Remove the dishes before discussing politics” “Walk to Üetliberg before lunch” “Take your medicine before lunch” “Finish lunch before removing dishes”

23 Using ordinary set operators spouse = wife  husband sibling = sister  brother  id [Person] sister  sibling father  ancestor universal [X ] = X x X (cartesian product) empty [X ] =  Convention: you are your own sibling

24 Possible properties of a relation (On a set X. All definitions must hold for every a, b, c...  X.) Total*: (a  b)  ((a r b)  (b r a)) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c *Definition of “total” is specific to this discussion (there is no standard definition). The other terms are standard.

25 Examples (on a set of persons) sibling Total:(a  b)  (a r b)  (b r a) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c Reflexive, symmetric, transitive sister Irreflexive Reflexive, antisymmetric family_head (a family_head b means a is the head of b’s family, with one head per family) mother Asymmetric, irreflexive

26 Total order relation (strict) A relation is a strict total order if it is:  Total  Irreflexive  Transitive Total:(a  b)  (a r b)  (b r a) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c Example: “less than” < on natural numbers 0 < 1 0 < 2, 0 < 3, 0 < 4,... 1 < 2 1 < 3 1 < 4,... 2 < 3 2 < 4,

27 Theorem A strict order is asymmetric (total)

28 Total order relation (strict) Theorem: A strict order is asymmetric total A relation is a strict total order if it is:  Total  Irreflexive  Transitive Total:(a  b)  (a r b)  (b r a) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c

29 Total order relation ( non-strict ) Relation is non-strict total order if:  Total  Reflexive  Transitive  Antisymmetric Example: “less than or equal ” ≤ on natural numbers 0 ≤ 0 0 ≤ 1, 0 ≤ 2, 0 ≤ 3, 0 ≤ 4,... 1 ≤ 1 0 ≤ 2, 0 ≤ 3, 0 ≤ 4,... 1 ≤ 2, 1 ≤ 3, 1 ≤ 4,... 2 ≤ 2 2 ≤ 3, 2 ≤ 4, Total:(a  b)  (a r b)  (b r a) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c

30 Total order relation (strict) Relation is strict total order if:  Total  Irreflexive  Transitive Total:(a  b)  (a r b)  (b r a) Reflexive:a r a Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Asymmetric:not ((a r b)  (b r a)) Transitive:(a r b)  (b r c)  a r c

31 Partial order relation (strict) [1, 2] 1 2 b a [0, 1] y x [3, 0] c [4, 2] d Example: relation between points in a plane: p q if both: x p < x q y p < y q < Relation is strict partial order if:  Total  Irreflexive  Transitive Total:(a  b)  (a r b)  (b r a) Irreflexive:not (a r a) Symmetric:a r b  b r a Antisymmetric:(a r b)  (b r a)  a = b Transitive:(a r b)  (b r c)  a r c

32 Theorems A strict order is asymmetric (total) partial A total order is a partial order (“partial” order really means possibly partial)

33 Example partial order Here the following hold: c a No link between a and c, b and c : a c nor e.g. neither [1, 2] 1 2 [0, 1] y [3, 0] [4, 2] a < b a < d c < d < < p q if both: x p < x q y p < y q < b a c d

34 Possible topological sorts 1 2 b a y c d a <b a <d c <d abcd cabd acbd

35 Topological sort understood Here the relation is: {[a, b], [a, d], [c, d]} 1 2 b a y c d We are looking for a total order relation t such that  t < One of the solutions is: a, b, c, d a < b a < d c < d <

36 Final statement of topological sort problem From a given partial order, produce a compatible total order where: A partial order p is compatible with a total order t if and only if p  t

37 From constraints to partial orders Is a relation defined by a set of constraints, such as constraints = {[Dishes, Politics], [Üetliberg, Lunch], [Medicine, Lunch], [Lunch, Dishes]} always a partial order? Üetliberg Medicine Lunch Dishes Politics

38 Powers and transitive closure of a relation r i +1 = r i ; r where ; is composition Ü M L D P r 1r 1 r 2r 2 r 3r 3 Transitive closure r + = r 1  r 2 ... always transitive

39 Reflexive transitive closure r i +1 = r i ; r where ; is composition Transitive closure r + = r 1  r 2 ... always transitive Ü M L D P r 1r 1 r 2r 2 r 3r 3 r 0r 0 Reflexive transitive closure: r  = r 0  r 1  r 2 ... always transitive and reflexive r 0 = id [X ] where X is the underlying set

40 Acyclic relation r +  id [X ] =  before + id [X] Ü M L D P A relation r on a set X is acyclic if and only if:

41 Acyclic relations and partial orders Theorems:  Any (strict) order relation is acyclic.  A relation is acyclic if and only if its transitive closure is a (strict) order. (Also: if and only if its reflexive transitive closure is a nonstrict partial order)

42 From constraints to partial orders The partial order of interest is before + before = {[Dishes, Politics], [Üetliberg, Lunch], [Medicine, Lunch], [Lunch, Dishes]} Üetliberg Medicine Lunch Dishes Politics

43 What we have seen The topological sort problem and its applications Mathematical background:  Relations as sets of pairs  Properties of relations  Order relations: partial/total, strict/nonstrict  Transitive, reflexive-transitive closures  The relation between acyclic and order relations  The basic idea of topological sort Next: how to do it for  Efficient operation: O (m + n) for m constraints & n items  Good software engineering: effective API

Chair of Software Engineering Einführung in die Programmierung Introduction to Programming Prof. Dr. Bertrand Meyer Lecture 18: Topological Sort Part 2: Algorithm and engineering

45 Back to software...

46 p q r s t u v w The basic algorithm idea topsort p q r s t u v w        

47 Overall structure (first try) Given: A type G A set of elements of type G A relation constraints on these elements Required: An enumeration of the elements in an order compatible with constraints class TOPOLOGICAL_SORTABLE [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] topologically_sorted : LINKED_LIST [G ] require no_cycle (constraints) do... ensure compatible (Result, constraints) end

48 Overall structure (improved) class TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process require no_cycle (constraints) do... ensure compatible (sorted, constraints) end Instead of a function topologically_sorted, use: A procedure process. An attribute sorted (set by process), to hold the result.

49 Non-uniqueness In general there are several possible solutions In practice topological sort uses an optimization criterion to choose between possible solutions. 1 2 b a y c d abcd cabd acbd

50 A partial order is acyclic  Must be a partial order: no cycle in the transitive closure of constraints  This means there is no circular chain of the form e 0 e 1 … e n e 0 If there is such a cycle, there exists no solution to the topological sort problem! < The relation: < < <<

51 Cycles In topological sort, we are not given the actual relation, but a relation constraints, through a set of pairs such as { [Dishes, Out], [Museum, Lunch], [Medicine, Lunch], [Lunch, Dishes]} The relation of interest is: = constraints + is acyclic if and only if constraints contains no set of pairs {[f 0, f 1 ], [f 1, f 2 ], …, [f m, f 0 ]} When such a cycle exists, there can be no total order compatible with constraints < < < Partial order Acyclic

52 Overall structure (reminder) class TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process require no_cycle (constraints) do... ensure compatible (sorted, constraints) end

53 Original assumption process require no_cycle (constraints) do... ensure compatible (sorted, constraints) end This assumes there are no cycles in the input Such an assumption is not enforceable in practice In particular: finding cycles is essentially as hard as topological sort!

54 Dealing with cycles Don’t assume anything; find cycles as byproduct of attempt to do topological sort The scheme for process becomes: “Attempt to do topological sort, accounting for possible cycles” if “Cycles found” then “Report cycles” end

55 class TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] end Overall structure (as previously improved) process require no_cycle (constraints) do... ensure compatible (sorted, constraints) end class TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] end

56 Overall structure (final) process require -- No precondition in this version do... ensure compatible (sorted, constraints) “sorted contains all elements not initially involved in a cycle” end class TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] end

57 p q r s t u v w Reminder: the basic algorithm idea topsort p q r s t u v w        

58 The basic loop scheme … loop “Find a member next of elements for which constraints contains no pair of the form [x, next]” sorted. extend (next) “Remove next from elements, and remove from constraints any pairs of the form [next, y]” end

59 The loop invariant Invariant in our first attempt: “constraints + has no cycles” Invariant in the revised architecture: “constraints + has no cycles other than any that were present originally” More generally: constraints + is a subset of the original constraints +

60 Terminology If constraints has a pair [x, y ], we say that  x is a predecessor of y  y is a successor of x

61 Algorithm scheme process do from create {...} sorted. make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next ) “Remove next from elements ” “Remove from constraints all pairs [next, y]” variant “ Size of elements” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

62 Implementing the algorithm (Number of elements: n Number of constraints: m) constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] We start with these data structures, directly reflecting input data: Example: elements = {a, b, c, d } constraints = {[a, b], [a, d ], [b, d ], [c, d ]} 1 2 b a y c d

63 Data structures 1: original elements = {a, b, c, d } constraints = {[a, b], [a, d ], [b, d ], [c, d ]} a badbd c d Efficiency: The best we can hope for: O (m + n) m constraints b c d a n elements elements constraints

64 process do from create {...} sorted. make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs of the form [next, y] ” variant “Size of elements” end if “No more elements” then “Report that topological sort is complete” else “Report cycle, in constraints and elements ” end Basic operations

65 The operations we need (n times)  Find out if there’s any element with no predecessor (and then get one)  Remove a given element from the set of elements  Remove from the set of constraints all those starting with a given element  Find out if there’s any element left

66 Efficiency The best we can hope for: O (m + n) (since we have to consider every constraint and every element at least once)

67 process do from create {...} sorted. make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs of the form [next, y] ” variant “Size of elements” end if “No more elements” then “Report that topological sort is complete” else “Report cycle, in constraints and elements ” end Basic operations n times m times n times

68 Data structures 1: original elements = {a, b, c, d } constraints = {[a, b], [a, d ], [b, d ], [c, d ]} Efficiency: The best we can hope for: Using elements and constraints as given wouldn’t allow us to reach O (m + n) ! a badbdcd m constraints b c d a n elements elements constraints

69 process do from create {...} sorted. make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs of the form [next, y] ” variant “Size of elements” end if “No more elements” then “Report that topological sort is complete” else “Report cycle, in constraints and elements ” end Basic operations n times m times n times

70 Implementing the algorithm Choose a better internal representation  Give every element a number (allows using arrays)  Represent constraints in a form adapted to what we want to do with this structure:  “Find next such that constraints has no pair of the form [y, next ]”  “Given next, remove from constraints all pairs of the form [next, y ]”

71 Algorithm scheme (without invariant and variant) process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

72 Data structure 1: representing elements elements : ARRAY [G ] -- Items subject to ordering constraints. -- (Replaces the original list) b a c d elements elements = {a, b, c, d } constraints = {[a, b], [a, d], [b, d], [c, d ]}

73 Data structure 2: representing constraints successors: ARRAY [LINKED_LIST [INTEGER]] -- Items that must appear after any given one successors elements = {a, b, c, d } constraints = {[a, b], [a, d], [b, d], [c, d ]}

74 Data structure 3: representing constraints predecessor_count: ARRAY [INTEGER] -- Number of items that must appear before a given one predecessor_count elements = {a, b, c, d } constraints = {[a, b], [a, d], [b, d], [c, d ]}

75 p q r s t u v w The basic algorithm idea topsort p q r s t u v w        

76 Finding a “candidate” (element with no predecessor) process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

77 Finding a candidate (1) Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next ] = 0 This requires an O (n) search through all indexes: bad! But wait...

78 Removing successors process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements ” end

79 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next :

80 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors targets freed

81 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors freed

82 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors

83 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors

84 Algorithm scheme process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

85 Finding a candidate (1) Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next ] = 0 We said: “Seems to require an O (n) search through all indexes, but wait...”

86 Removing successors Implement “Remove from constraints all pairs [next, y]” as a loop over the successors of next : targets := successors [next ] from targets. start until targets. after loop freed := targets. item predecessor_count [freed ] := predecessor_count [freed ] − 1 targets. forth end predecessor_count successors targets

87 Finding a candidate (2): on the spot Complement predecessor_count [freed ] := predecessor_count [freed ] − 1 by: if predecessor_count [freed ] = 0 then -- We have found a candidate! candidates. put (freed ) end

88 Data structure 4: candidates candidates : STACK [INTEGER ] -- Items with no predecessor Instead of a stack, candidates can be any dispenser structure, e.g. queue, priority queue The choice will determine which topological sort we get, when there are several possible ones candidates put item, remove

89 Algorithm scheme process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

90 Finding a candidate (2) Implement next := “A member of elements with no predecessor” if candidates is not empty, as: next := candidates. item

91 Algorithm scheme process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

92 Finding a candidate (3) Implement the test “ Every member of elements of has a predecessor” as not candidates. is_empty To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with the original number of elements.

93 Reminder: the operations we need (n times)  Find out if there’s any element with no predecessor (and then get one)  Remove a given element from the set of elements  Remove from the set of constraints all those starting with a given element  Find out if there’s any element left

94 Detecting cycles process do from create {...} sorted. make until “ Every member of elements has a predecessor” loop next := “A member of elements with no predecessor” sorted. extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements” end

95 Detecting cycles To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with the original number of elements.

96 Data structures: summary elements : ARRAY [G ] -- Items subject to ordering constraints -- (Replaces the original list) successors : ARRAY [LINKED_LIST [INTEGER]] -- Items that must appear after any given one predecessor_count : ARRAY [INTEGER] -- Number of items that must appear before -- any given one candidates : STACK [INTEGER] -- Items with no predecessor

97 Initialization Must process all elements and constraints to create these data structures This is O (m + n) So is the rest of the algorithm

98 Compiling: a useful heuristics The data structure, in the way it is given, is often not the most appropriate for specific algorithmic processing. To obtain an efficient algorithm, you may need to turn it into a specially suited form. We may call this “compiling” the data Often, the “compilation” (initialization) is as costly as the actual processing, or more, but that’s not a problem if justified by the overall cost decrease.

99 Another lesson It may be OK to duplicate information in our data structures: successors: ARRAY [LINKED_LIST [INTEGER]] -- Items that must appear after any given one. predecessor_count: ARRAY [INTEGER] -- Number of items that must appear before -- any given one. This is a simple space-time tradeoff

100 Key concepts  A very interesting algorithm, useful in many applications  Mathematical basis: binary relations  Remember binary relations & their properties  Transitive closure, Reflexive transitive closure  Algorithm: adapting the data structure is the key  “Compilation”strategy  Initialization can be as costly as processing  Algorithm not enough: need API (convenient, extendible, reusable)  This is the difference between algorithms and software engineering

101 Software engineering lessons Great algorithms are not enough We must provide a solution with a clear interface (API), easy to use Turn patterns into components