Output Sensitive Enumeration

Output Sensitive Enumeration
2. Basic Algorithms Divide and Conquer Backtracking Backtracking for Maximals Binary Partition (st-path, matching) Seeing Difficulty on Binary Partition

2-1 Branch and Bound

Brute-force Algorithm
• Brute force is acceptable if the problem is easy (small) 　 how do we do “brute force”? 　+ enumerate all candidates and output the solutions among them 　+ enlarge the solutions one by one and remove isomorphic ones 　+ scan the candidates from the smaller ones • We should have “simple algorithms” for solving easy problems • … and also “simple implementing ways”, such as, only making a subroutine (oracle) for computing a function is necessary

Divide and Conquer • Determine the value of variables one by one
• Recursive call for each value • At the bottom of recursion, an assignment is determined v1 ￢v1 v1, v2 v1, ￢v2 ￢v1, v2 ￢v1, ￢v2

Divide and Conquer • Prune the brunch when we confirm non-existence of solution in descendants  accuracy and speed are the key If the pruning is exact, and is polynomially done,  polynomial time delay v1 ￢v1 v1, v2 v1, ￢v2 ￢v1, v2 ￢v1, ￢v2

Extension Problem • Formally, to prune the branch (recursive call) we have to solve the following extension problem Extension Problem: For an enumeration problem, sets S (to be included), and X (to be excluded), determine whether there is a solution Z to the enumeration problem s.t. S⊆Z and Z ∩ X = ∅. If this problem can be solved efficiently, the branch and bound algorithm works well v1, ￢v2

(1) Enumerate Combinations
• Enum. all combinations  determine variable values recursively • Only “3. check of being a solution” is needed. 1. is not necessary • Fast if check in 1 is of high accuracy Enum1 (X :set of all determined values, i: index) 1. if no solution includes X then return 2. if i > maximum index then if X is a solution then output X 4. else 5. for each e in values which xi can take 　 call Enum1 (X∪(xi =e), i+1) 7. end for Extension problem

(2): Enumerate Patterns
• To avoid isomorphic solutions, incremental generation 　　　　　　　　　　　　　　　　　　(for graphs, matrix, sequences,…) • Only designs of 3. and 4. are necessary • Efficient if check in 2. is fast and of high accuracy Global variable: database D := φ Enum2 (X: pattern) 1. insert X to D 2. if no solution includes X then return 3. if X is a solution then output X 4. for each X’ obtained by adding an element to X if none of D is isomorphic to X’ then call Enum2 (X’)

Basic Enumeration Algorithms
• Since fundamental, construction scheme is also simple • On the other hand, not so many variations + backtracking depth-first search with lexicographic ordering + binary partition branch & bound like recursive partition algorithm + reverse search search on traversal tree defined by parent-child relation

2-2 Backtracking

Backtracking • Consider a monotone set (independent) system, for example cliques Clique: a subgraph in which any vertex pair is connected by an edge A clique is found by iteratively adding vertices, with passing through only cliques Ex. 2  6  4  8 So, starting from the empty set, and iteratively adding vertices, every time we can find a clique 1 2 3 5 4 6 7 8 9 10 11 12

Duplication • By naively adding vertices, we generate a clique several times 2  6  4  8 2  6  8  4 4  6  2  8 … Some idea to avoid duplications is necessary Define a rule; add only vertices whose indies are larger than any of the clique Then, duplication never happens  4  6  8 1 2 3 5 4 6 7 8 9 10 11 12

Why OK? add only vertices whose indies are larger than any of the clique Suppose that a clique is generated in two ways 2  4  6  8 2  6  4  8 Then, at least one of them has a pair of vertex addition, such that The previous one is larger than the latter It contradicts the addition rule 1 2 4 6 8 10 11

General Backtracking • Mainly used for independent (monotone) sets (maximals) Independent set system F : X∈F  for any X’⊆X , X'∈F 　　　( X∈F  any subset of X is a member of F) Ex) + cliques of a graph, matchings, combinations of numbers whose sum is less than b, frequent itemsets… × Not + trees of a graph, paths, cycles, … 111…1 000…0

Framework of Backtracking
• Start from the empty set, and recursively add elements • In each iteration, add only elements larger than the current maximum element (an iteration does not include those in its recursive calls) • Recursive call with the result of addition, if it is a solution • Go back after all examinations 111…1 cliques 000…0 ∅ 1,3 1,2 1,2,3 1,2,4 1,3,4 2,3,4 1 2 3 4 3,4 2,4 1,4 2,3 1,2,3,4

Pseudo Code for Backtracking
• Start from the empty set, and recursively add elements; add only elements larger than the current maximum element 1,2,3,4 Backtrack (S) 1. output S 2. for each e > tail of S 　 (the max. element in S) 3. if S∪{e} is a solution then call Backtrack (S∪{e}) 4. end for 1,2,3 1,2,4 1,3,4 2,3,4 1,2 1,3 1,4 2,3 2,4 3,4 1 2 3 4 ∅ • simple, and polynomial space • polynomial delay　(output polynomial time)

Feasible Solutions to Knapsack Problem
…folklore Problem: enumerate all subsets of a1,…,an whose sum is less than b Computation time: each iteration outputs a solution, and take O(n) time  time per solution is O(n) • Sort a1,…,an, then each recursive call can be generate in O(1) time 　an iteration O(#recursive calls) 　  O(1) time per solution FeasibleKnapsack (S) 1. output S 2. for each i > tail of S (maximum element in S) if ∑S ＋ ai < b then call FeasibleKnapsack (S∪{ai}) 4. end for

Code for Knapsack • Print all combinations of a[0],…,a[n] with summation less than b int a[n], flag[n]; sub (int i, int s){ int j; for (j=0 ; j<n ; j++) if (flag[j] = = 1) printf (“%d\n”, a[j]); // print a solution for (j=i+1 ; j<n ; j++) if (s+a[j] <= b){ // check the feasibility flag[j] = 1; sub (i, s+a[j]); flag[j] = 0; }

Simple Case • There are several problem in that we don’t need to take care the duplications Problem: for given a graph and vertex s, enumerate all paths starting from s Ex. 1  2 1  5  9 1  3  9  11  4  8  2  6 In each iteration, we generate a recursive call for each neighboring vertex 1 2 3 5 4 6 7 8 9 10 11 12

Algorithm for Paths Starting at s
…folklore enumerate all paths of G starting at s Computation time: O(n) time per iteration  O(n) time per solution PathStartingS (P, s) 1. output P 2. A := set of vertices adjacent to s 3. delete s from the graph 4. for each v in A, call PathStartingS (P∪{v}, v) 5. recover s to the graph

2-3 Maximal by Backtracking

Maximal Solutions • # of solutions increases exponentially when n or the sizes of solutions are large • If # of solutions is large, post-process is also hard  enumerate maximal so that the solution set is irredundant 　X∈F is maximal in F  for any X⊆X’, X’∈F does not hold • Maximal solutions are not neighboring to each other, efficient search is hopeless (exception; spanning trees, matroid bases) 111…1 000…0

Straightforward Method
• Backtracking enumerates all solutions, so we can enumerate all maximals by just outputting a solution only when it is maximal  it is inefficient… • If we can prune the recursive call that never output any maximal, the algorithm will be output polynomial time In general, we want to solve the following problem Maximal Extension: For partial solution S and set X of deleted elements, is there a maximal solution Z s.t., S⊆Z and Z ∩ X = ∅ ? … this problem is hard in general 111…1 000…0

Knapsack Maximals …folklore
Fortunately, maximal extension can be solved for knapsack problem Problem: enumerate all maximal subsets of a1,…,an whose sum is no greater than b • Put indices to a1,…,an in decreasing order Computation time: 　An iteration takes O(n) 　  　O(n2) time per solution MaximalKnapsack (S) 1. output S 2. for each i > tail of S and ∑S + ai +…+ an > b – ai-1 if ∑S + ai ≦b then call MaximalKnapsack (S∪{ai}) 4. end for Pruning that with only non-maximal solutions

Partial Pruning • Exact pruning is usually difficult, so we often use partial pruning; here partial means that in only partial cases we can prune the recursive calls • The pruning methods are usually based on some structures or property of the target problem So, there are less methods work in general • One of these uses element- reordering in each iteration 1,2,3,4 1,2,3 1,2,4 1,3,4 2,3,4 1,2 1,3 1,4 2,3 2,4 3,4 1 2 3 4 ∅

Maximal: Shift a Solution to the End
…folklore • Maximal enumeration admits a simple pruning algorithm (1) prune if meets a non-member (2) no brunch needed if addition of all remaining members is a member • Even if (1) is complete, exhaust search for all members is inefficient • Find a maximal solution, shift all its element to the bottom, then no need of recursive calls for the shifted elements 　 because (2) works for the elements! element ordering For small maximal solution sizes (up to 30), practically efficient

• Describe the algorithm by a pseudo code
FeedTailMaximal (P:current solution, I: undetermined elements) 1. find maximal set S among those including P and included in P∪I 2. if S is a maximal solution of the problem then output S 3. for each e∈I＼S 　　　　I := I＼{e}; call FeedTailMaximal (P∪e, I) element ordering I P

Practically very good (constant time for each maximal clique)
Specialized to Clique Tomita et. al. 2006 • For cliques, we can have bigger pruning Observation: after a recursive call with respect to P∪e, the cliques composed of P and the neighbor of e never be maximal By feeding the neighbor of e to the last, recursive calls for the fed vertices become unnecessary 1 2 3 5 4 6 7 8 9 10 11 12 Reorder this area e neighbor of e P Practically very good (constant time for each maximal clique)

2-4 Binary Partition for st-path

Binary Partition • X is a set of solutions, that is a subset (subsequence, etc.) of F satisfying a property P • Binary partition outputs the solution if solution in X is unique • Otherwise, it partitions F into two (or several) sets so that X is partitioned into non-empty sets • Do this recursively, until the solution is unique F1　　　X1 F X F2　　　X2 Ex.) 　+ paths of a graph connecting vertex s and vertex t (st-paths) 　+ perfect matchings of a bipartite graph 　+ spanning trees of a graph 　+ connected components of a graph

Time Complexity • Binary partition always partitions a problem or outputs a solution  #iteration is bounded by 2N • The partition process is polynomial time, (determine how to partition, and check empty or not)  the algorithm is output polynomial time • • • • • •

Time Complexity • If the height of the tree is polynomial in n,
it is polynomial delay (to go up (go back) from the leaf to the root, O(height) time is needed) • If the partition process needs polynomial space, the algorithm is polynomial space • • • • • •

Binary Partition of st-paths
Read&Tarjan ’75 …modified by U Problem: enumerate all st-paths in G = (V,E) Partition: choose an edge e incident to s, and partition into + enumeration of st-paths including e + enumeration of st-paths not including e so that both problems are non-empty (it is also called pivoting and e is called pivot, or pivoting edge) Child Problems: st-paths including e: remove all edges incident to s except e 　st-paths not including e: remove e

Child Problems on st-paths
st-paths including e: remove all edges incident to s (and move s to the next vertex)  denote G-s 　st-paths not including e: remove e  denote G-e Computation time: 　one iteration = O(|E|) 　 s t s t s t s t

Choosing Valid Edge • If we choose a bad edge, the subproblems will be empty; + “including e” is empty, if t is not reachable via e  remove the component including e + “not including e” is empty, if e is the only edge reachable to t  move s to the next vertex , and remove e • After at most |E| repetitions, we can always find a valid edge s t

Time Complexity • Test of the validity of the edge takes O(|V|) time at most O(|E|) repetitions • An iteration takes O(|E||E|) time • Since #iterations < 2N, time per solution is O(|E||E|) • Since the height of the recursion tree is O(|V|), the delay O(|V||E|2) s t

Pseudo Code for st-paths
Enum_st-path (G, s, t, S) 1. if s = t then output S, return 2. choose an edge e = (s,v) 3. if no vt-path in G-s then remove e, go to 1. 4. if no st-path in G-e then remove e, S := S+s, s := v, go to 1. 5. call Enum_st-path (G-s, v, t, S) 6. call Enum_st-path (G-e, s, t, S) s t s t

Better Algorithm • How long does it take (graph reform) to find a valid edge? • Find a path P from s to t • Choose an edge e = (s,v) incident to s and not in P + t is not reachable via e  delete the visited edges  O(#delete edges) + only one edge (in P) is incident to s  move s to v, and remove e  O(1) • Computation time is O(#delete edges), until we find a valid edge, i.e., O(|E|) v s t

Pseudo Program Code • flag[] :=0 in initialization, path is the current solution int mark[m], path[n]; enum_path (int s, int i){ if (s = t){ output path[0],…,path[i]; return } • find an st-path, f (=(s,v)) := the edges in the path incident to s • mark[f]:= 1 (put mark) while (1){ • choose an edge e=(s, v) s.t. mark[e] = 0 • mark[e] := 1 • if (no such edge e exist){ path[i] := v; i++; s := v } else if ( t is reachable from v via only unmarked edges and not through s ){ break } } call enum_path (s, i); path[i] := v; call enum_path (v, i+1); • set mark[e]:= 0 for edges e marked in this iteration

2-5 Binary Partition General Scheme

A Simple Description BinaryPartition (E, S, X) 1. while E ≠ S∪X
2. choose e∈E＼(S∪X) 3. solve extension problem for S∪e, X 4. solve extension problem for S, X∪e 5. if yes for both problems call BinaryPartition (E, S∪e, X) call BinaryPartition (E, S, X∪e) return 6. else if yes for 3 then S = S∪e 7. else X = X∪e 8. end while 9. if E = S∪X then output S, return • Binary partition divides the problem into two We often partition by a variable/vertex/edge +enumerate all including x, and all not including x In this setting, we can generalize the algorithm as the right O(|E|T(Ext)) time for each

Using Certificate BinaryPartition (E, S, X) 1. while E ≠ S∪X
2. choose e∈E＼(S∪X) 3. solve extension problem for S∪e, X 4. solve extension problem for S, X∪e 5. if yes for both problems call BinaryPartition (E, S∪e, X) call BinaryPartition (E, S, X∪e) return 6. else if yes for 3 then S = S∪e 7. else X = X∪e 8. end while 9. if E = S∪X then output S, return • When the extension problem gives a solution C as a certificate, we can use it to efficiently choose the pivot • If there is another solution to be enumerated, at least one element of C acts as a pivot  another solution doesn’t include at least one of C O(|C|T(Ext)) time for each

With Strong Extension • If we can solve “another solution problem”, which is to find a solution different from a given solution, we can do better  Any edge in the symmetric difference can be a pivot O(T(AnoS)) time for each BinaryPartition (E, S, X, T) 1. output T 2. find a solution T’ ≠ T s.t. S ⊆ T’, T’∩X = ∅ 3. if such T’ does not exists return 4. choose e∈ T’ △ T　(w.l.o.g. assume e∈ T) 5. call BinaryPartition (E, S∪e, X, T) 6. call BinaryPartition (E, S, X∪e, T’)

2-5 Binary Partition Perfect Matching

Bipartite Perfect Matching
• A matching is an edge set such that no two edges in the set have their endpoints on the same vertex • A matching is perfect if it covers all the vertices (any vertex is incident to an edge of the matching) • For given a bipartite graph G=(V,E), its matching is called a bipartite matching • We want to enumerate all perfect matchings in G

• For an edge e of the graph, the set of perfect matchings not including e is the set of perfect matchings in G＼e (obtained by removing e) • For an edge e of the graph, the set of perfect matchings including e is the set of perfect matchings in the graph obtained by removing all edges adjacent to e • A perfect matching in a bipartite graph can be found in O(|V|1/2|E|) time • Combining these, we can obtain an output linear time algorithm of O(|V|1/2|E|2) time

• For bipartite perfect matching, we can solve the another solution problem in O(|E|) time • The symmetric difference of any two perfect matching is composed only of cycles In the cycle edges of one matching and edges of the other appear alternatively ↑ such cycle is called an alternating cycle • On the other hand, for a matching, a cycle is called alternating if matching edges and non-matching edges appear in the cycle alternatively

Existence of Another Solution
• If there is another solution, there is always an alternating cycle for the perfect matching By exchanging the edges along an alternating cycle, we can have another perfect matching • Then, how to find an alternating cycle? • Orient edges from left to right for matching edges, and the opposite to the others … then directed cycles and alternating cycles correspond one to one and thus can be found in O(|E|) time

2-6 Seeing Difficulty of Binary Partition Algorithms

Why Difficulty • It is of course important to study on developing efficient binary search algorithms and, also seeking good problems, admit poly-time algorithms • However, for given problems, usually this direction is hard we cannot find any good algorithm  then, we naturally want to know “why it is difficult” • We don’t have any NP-completeness like complexity tool dualization hard is such a kind, but restricted • Another way is to state that simple algorithm never work

Subproblem in the Same Analogy
• We often partition the problem into solutions including e, and solution not including e (e is called pivot) Easy case: both subproblem can be formulated in the same analogy as the original problem ex) s-t path enumeration  s’-t path enumeration ex) knapsack of set X  knapsack of set X-a Hard case: one is not formulated in the same way + enumerate maximal cliques including v (the same in the induced subgraph of N(v)) + enumerate maximal cliques not including v (not formulated as clique enumeration) F1　　　X1 F X F2　　　X2

Extension Problem • In general, when we choose a vertex/element and partition the problem by inclusion/exclusion, the problem we have to solve is an extension problem Extension Problem: For an enumeration problem, sets S (to be included), and X (to be excluded), determine whether there is a solution Z to the enumeration problem s.t. S⊆Z and Z ∩ X = ∅. Even though the problem is hard in general, there might be polynomial time binary partition algorithm v1, ￢v2

Extension of st-path • Consider the extension problem on st-path
Extension Problem for st-path: For a graph G= (V, E), vertices s and t, vertex sets S and X, determine whether there is an st-path passing through all edges in S but none in X Actually, it is known to be NP-complete (even in the case X=∅)  Hamiltonian path problem can be reduced However, by carefully choosing the pivoting edge, we can have polynomial time delay v1, ￢v2

Extension of Maximal Clique
• Consider the extension problem on maximal clique Extension Problem for maximal clique: For a graph G= (V, E), vertex sets S and X, determine whether there is a maximal clique Z s.t. S⊆Z and Z ∩ X = ∅. Actually, it is known to be NP-complete  Straightforward binary partition is difficult to design Different from st-path, it is hard to see some properties that enables us to find some pivot vertex makes the extension problem easy v1, ￢v2

References st-path, cycle
D. Eppstein, Finding the k Shortest Paths, FOCS94, (1994) D. B. Johnson, Finding All the Elementary Circuits of a Directed Graph, SIAM J. Comp. 4, (1975) R. C. Read and R. E. Tarjan, Bounds on Backtrack Algorithms for Listing Cycles, Paths, and Spanning Trees, Networks 5, (1975)

References Clique E. A. Akkoyunlu, The Enumeration of Maximal Cliques of Large Graphs, SIAM J. Comp. 2,1-6 (1973) D. S. Johnson, M. Yannakakis, and C. H. Papadimitriou, On Generating All Maximal Independent Sets, Info. Proc. Lett. 27, (1988) T. Kashiwabara, S. Masuda, K. Nakajima and T. Fujisawa, Generation of Maximum Independent Sets of a Bipartite Graph and Maximum Cliques of a Circular-Arc Graph, J. Algo. 13, (1992) E. Tomita, A. Tanaka, H. Takahashi, The Worst-case Time Complexity for Generating all Maximal Cliques and computational experiments", Theoretical Computer Science 363, (2006)

References Perfect Matching
K. Fukuda and T. Matsui, Finding All the Perfect Matchings in Bipartite Graphs, Appl. Math. Lett. 7, (1994). K. Fukuda and T. Matsui, Finding All the Minimum Cost Perfect Matchings in Bipartite Graphs, Networks 22, (1992) C. R. Chegireddy and H. W. Hamacher, Algorithms for Finding K-best Perfect Matchings, Discrete Appl. Math. 18, (1987) T. Uno, Algorithms for Enumerating All Perfect, Maximum and Maximal Matchings in Bipartite Graphs, ISAAC97, LNCS 1350, (1997) T. Uno, A Fast Algorithm for Enumerating Bipartite Perfect Matchings, ISAAC2001, LNCS 2223, (2001) T. Uno, A Fast Algorithm for Enumerating Non-Bipartite Maximal Matchings, J. National Institute of Informatics 3, (2001)

Exercises 2

Backtrack 2-1. Explain why the algorithm PathStartingS does not produce a path passing through the same vertex twice 2-2. Explain why the deletion of a vertex in step 3 of the algorithm PathStartingS does not affect the other iterations 2-3. Actually, the algorithm PathStartingS takes only constant time for each iteration, except for the outputting process. Prove this. 2-4. Give some example of independent set system (monotone sets) 2-5. Give some example of problems in which backtracking algorithm doesn’t have to care the duplications

Backtrack 2-6. Design a backtracking algorithm for the following problem: For given a sequence of numbers a1,…,an, enumerate all its subsequence such that any two consecutive numbers ai and aj satisfies ai < aj. 2-7. Design a backtracking algorithm for the following problem: For given a set of points in a plane, a non-crossing graph is a graph whose vertex set is the point set, and its edge set is a set of segments whose ends are on the points, such that no two segments intersects except for their ends. Enumerate all non-crossing trees for given a point set

Backtrack 2-8. Design a backtracking algorithm for the following problem: For given a set of vectors x1,…,xn of composed of positive integers, enumerate all sets of vectors such that their sum is no greater than given vector b. (subset X s.t., Σx∈ X ≤ b) 2-9. Design a backtracking algorithm for the following problem: For given a set of rectangles and a square, enumerate all possible locations of the subset of the rectangles s.t. no two rectangles overlap. The left-up corner of each rectangle has to be placed at an integer grid point, and the edges of rectangles has to axis parallel (so, 90 degree rotation is allowed).

Backtrack Design a backtrack algorithm for the following problem, and analyze its time complexity For given a graph, enumerate all matchings of the graph Design a backtrack algorithm for the following, and analyze its time complexity For given a sequence of letters, enumerate all its subsequences that form palindromes, i.e., forming a,b,c,…,d,d,…,c,b,a

Binary Partition Design a binary partition algorithm for the following problem, and analyze its time complexity For given a set of points in a plane, enumerate all non-crossing spanning trees Design a binary partition algorithm for the following problem, and analyze its time complexity For given a set of points in a plane, and two points s and t, enumerate all non-crossing s-t paths (simple paths whose ends are s and t)

Binary Partition For two perfect matchings M and M’ of a bipartite graph G, the symmetric difference between M and M’ is composed of disjoint cycles. Further, the symmetric difference between M and an alternating cycle in which edges of M and edges not in M appear alternatively results a perfect matching different from M. Design a binary partition algorithm by using this fact. Design a binary partition algorithm for the following problem, and analyze its time complexity For a given partial order, a chain is a sequence of elements e1,…,em s.t., ei < ei+1 holds for any i. Enumerate all maximal chains that are included in no other chain

Exercises Design a binary partition algorithm for the following, and analyze its time complexity For a connected graph, a minimal cut is a partition of vertices such that the subgraphs induced by each group is connected. For given two vertices s and t, enumerate all minimal cuts s.t. one component includes s and not t Design a binary partition algorithm for the following problem, and analyze its time complexity For given a graph such that each edge is colored, enumerate all matchings of the graph s.t. any two edges have different colors

Exercises Prove the NP-completeness of the extension problem of maximal clique Prove the NP-completeness of the extension problem of st-path Prove the NP-completeness of the extension problem of minimal dominating set (equivalent to minimal set covering, hypergraph transversal, dualization of monotone Boolean formula)

Output Sensitive Enumeration

Similar presentations

Presentation on theme: "Output Sensitive Enumeration"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Output Sensitive Enumeration

Similar presentations

Presentation on theme: "Output Sensitive Enumeration"— Presentation transcript:

Similar presentations

About project

Feedback