1. Saucy3: Fast Symmetry Discovery in Graphs Hadi Katebi Karem A. Sakallah Igor L. Markov The University of Michigan

2. 2 Outline Graph symmetry Graph symmetry Implicit representation of permutation sets: Ordered Partition Pairs (OPPs) Implicit representation of permutation sets: Ordered Partition Pairs (OPPs) Basic permutation search tree Basic permutation search tree Pruning via partition refinement: Pruning via partition refinement: –Non-isomorphic OPP pruning –Matching OPP pruning Group-theoretic pruning: Group-theoretic pruning: –Coset pruning –Orbit pruning Algorithm trace Algorithm trace Experimental results Experimental results Conclusions Conclusions

3. 3 6 1 3 2 4 7 8 5 Graph Symmetry 6 1 3 2 4 7 8 5 (1 4)(2 3) 4 1 4 2 3 1 6 4 2 3 1 7 8 5 is a symmetry! The set of edges is unchanged. (1 2 3 4)(5 6) is not a symmetry! 2 4 3 1 5 2 4 3 1 65 2 4 3 1 7 8 6 5 2 4 3 1 7 8 6 The set of edges is different.

4. 4 Problem Statement Given a graph G Given a graph G –with n vertices –and a partition  of its vertices (colors), –with unknown set of symmetries Sym(G) , Find a set of symmetries S  Sym(G)  Find a set of symmetries S  Sym(G)  –such that S generates Sym(G)  –and |S| ≤ n - 1

5. 5 Graph Symmetry Tools Nauty (McKay, ’81) Nauty (McKay, ’81) –Blazed the trail –Tuned to quickly find the symmetries of large sets of small graphs Saucy (Darga et al, DAC ’04) Saucy (Darga et al, DAC ’04) –Graph symmetry can be fast for large yet sparse graphs –> 1000x speedup over nauty for graphs with tens of thousands of vertices Bliss (Junttila & Kaski, ’07) Bliss (Junttila & Kaski, ’07) –Efficient canonical labeling of sparse graphs –Some improvements on Saucy Traces (Piperno, ’08) Traces (Piperno, ’08) Primarily canonical labeling tools; symmetries produced as a byproduct

6. 6 Permutations Permutation: bijection from V to V Permutation: bijection from V to V –Tabular representation –Cycle notation Graph Symmetry: permutation that preserves edge relation Graph Symmetry: permutation that preserves edge relation Permutation Composition: Permutation Composition: Symmetric group on m-element set T: S m (T ) Symmetric group on m-element set T: S m (T ) –|S m (T )| = m !

7. 7 Ordered Partitions Unit OP: m = 1 Unit OP: m = 1 Discrete OP: m = n Discrete OP: m = n

8. 8 Ordered Partition Pair (OPP) Isomorphic OPP: m = k and Isomorphic OPP: m = k and Non-isomorphic OPP: m ≠ k or Non-isomorphic OPP: m ≠ k or Matching OPP: isomorphic and Matching OPP: isomorphic and Unit OPP: top and bottom ordered partitions are unit Unit OPP: top and bottom ordered partitions are unit Discrete OPP: top and bottom ordered partitions are discrete Discrete OPP: top and bottom ordered partitions are discrete

9. 9 Implicit Representation of Permutations Sets using OPPs Discrete OPP (single permutation)

10. 10 Implicit Representation of Permutations Sets using OPPs Unit OPP (m ! permutations)

11. 11 Implicit Representation of Permutations Sets using OPPs Isomorphic OPP

12. 12 Implicit Representation of Permutations Sets using OPPs Matching OPP

13. 13 Implicit Representation of Permutations Sets using OPPs Non-isomorphic OPP

14. 14 Basic Search for Symmetries 1 3 2 4 6 7 8 5 6 7 8 6767 8888 8686 7 6 8 X 7 8 6 X 6666 7878 6 8 7 7777 6 7 8 6868 7676 8 6 7 X 7777 8 7 6 X

15. 15 Vertex Partition Refinement For each vertex v, compute a neighbor-count tuple For each vertex v, compute a neighbor-count tuple Partition the vertices based on these tuples Partition the vertices based on these tuples Repeat until the partition stabilizes Repeat until the partition stabilizes Try to distinguish vertices that are not symmetric Try to distinguish vertices that are not symmetric R 6 1 3 2 4 7 8 5 (3)(3) (3)(3) (3)(3) (3)(3) (3)(3) (3)(3) (3)(3) (7)(7) 5 (2,1)(2,1) (2,1)(2,1) (2,1)(2,1) (2,1)(2,1) (2,1)(2,1) (2,1)(2,1) (2,1)(2,1)

16. 16 Search-Tree Pruning via Vertex Refinement 6 7 8 7777 6 7 8 7878 6 8 7 R

17. 17 Isomorphic Refinement 6 1 3 2 4 7 8 5 6 1 3 2 4 7 8 5 12121 2 R 2 34 1 4 3 TopBottom

18. 18 Non-Isomorphic Refinement 6 1 3 2 4 7 8 5 6 1 3 2 4 7 8 5 16161 6 R 2 34 7 8  TopBottom

19. 19 Group Generators 8 vertices: 8! = 40320 permutations 8 vertices: 8! = 40320 permutations 48 symmetries 8 for square and 6 for triangle 48 symmetries 8 for square and 6 for triangle Basic enumeration is inefficient Basic enumeration is inefficient Fundamental concept: symmetry group can be represented implicitly by an exponentially smaller set of generators Fundamental concept: symmetry group can be represented implicitly by an exponentially smaller set of generators 6 1 3 2 4 7 8 5 identity (1 2 4 3) (1 4)(2 3) (1 3 4 2) (1 2)(3 4) (1 3)(2 4) (1 4) (2 3) identity (6 7 8) (6 8 7) (6 7) (6 8) (7 8)

20. 20 Group Generators 6 1 3 2 4 7 8 5 Generators:  1 = (1 2)(3 4)  2 = (2 3)  3 = (6 7)  4 = (6 8) (1 3 4 2)(7 8) =  2  1  3  4  3 is a symmetry   4  3 = (6 8) ◦ (6 7) = (6 7 8)    3  4  3 = (6 7) ◦ (6 7 8) = (7 8)    1  3  4  3 = (1 2)(3 4) ◦ (7 8) = (1 2)(3 4)(7 8)  2  1  3  4  3 = (2 3) ◦ (1 2)(3 4)(7 8) = (1 3 4 2)(7 8)

21. 21 Orbit Partition Initial partition: Initial partition:{{1,2,3,4,5,6,7,8}} Orbit partition: Orbit partition:{{1,2,3,4},{5},{6,7,8}} After degree refinement: After degree refinement:{{1,2,3,4,6,7,8},{5}} 6 1 3 2 4 7 8 5 Symmetry group induces an equivalence relation on vertices: Symmetry group induces an equivalence relation on vertices: the orbit partition Refinement provides an approximation of the orbit partition Refinement provides an approximation of the orbit partition Orbit partition: Orbit partition: –Built up incrementally from discovered symmetries –Used to prune search for redundant symmetries

22. 22 Cosets A subgroup H of a group G partitions it into cosets A subgroup H of a group G partitions it into cosets Each coset has the same number of elements as H Each coset has the same number of elements as H G can be generated by composing a single representative from each coset with H G can be generated by composing a single representative from each coset with H Used to prune search for redundant symmetries Used to prune search for redundant symmetries

23. 23 Structure of Permutation Search Tree

24. 24 Algorithm Outline Phase 1: Recursive subgroup decomposition Phase 1: Recursive subgroup decomposition Phase 2: Search for coset representatives … surprisingly like SAT solving! Phase 2: Search for coset representatives … surprisingly like SAT solving! Four pruning mechanisms: Four pruning mechanisms: –Group-theoretic Coset pruning: stop after coset representative is found Coset pruning: stop after coset representative is found Orbit pruning: avoid looking for coset representative Orbit pruning: avoid looking for coset representative –Algorithmic (due to OPP data structure): Matching OPP pruning: identify candidate permutation before reaching leaves Matching OPP pruning: identify candidate permutation before reaching leaves Non-isomorphic OPP pruning: detect absence of coset representative in current subtree Non-isomorphic OPP pruning: detect absence of coset representative in current subtree

25. 25 Saucy 2.1 Search Tree ≈ 6 1 3 2 4 7 8 5 {7,8} {6,7,8} {2,3}{6,7,8} {1,2,3,4}{6,7,8} Orbit Partition (7 8) id (6 7) // ≈ x (2 3) = (1 2)(3 4) = ≈ Coset pruning // Orbit Pruning = Matching OPP x Non-isomorphic OPP 1111 1212 13141314 1616 17181718 2222 2323 6666 6767 6868 2121 2424 7777 7878 7676 7878 R R R

26. 26 Experimental Evaluation 1183 SAT 2009 competition benchmarks 1183 SAT 2009 competition benchmarks –Application –Crafted –Random Saucy on all 1183 Saucy on all 1183 Shatter on 47 most difficult benchmarks Shatter on 47 most difficult benchmarks Experiments on SUN workstation Experiments on SUN workstation –3GHz Intel Dual-Core CPU –6MB cache –8GB RAM –64-bit Redhat Linux

27. 27 Saucy Time vs. Graph Vertices

28. 28 Saucy Time vs. Graph Vertices Time out = 500 sec. Time out = 500 sec. Sacuy finished on all but 18 Sacuy finished on all but 18 –connum: 6 (solved by varying branching heuristics) –equilarge: 3 –mod2-rand3bip: 9 Crafted category is the most challenging Crafted category is the most challenging Weak trend towards larger run times for larger graphs Weak trend towards larger run times for larger graphs Saucy is really fast (runtime < 1 sec.) on 93% (1101) of all benchmarks Saucy is really fast (runtime < 1 sec.) on 93% (1101) of all benchmarks

29. 29 Saucy Group Order vs. Testcase

30. 30 Saucy Group Order vs. Testcase 323 benchmarks exhibited non-trivial symmetries 323 benchmarks exhibited non-trivial symmetries Random category: Random category: –606 had no symmetry –4 had one symmetry Crafted category: Crafted category: –175 out of 263 (66%) had symmetry –18 timed out Application category: Application category: –144 out of 292 (50%) had symmetry

31. 31 Saucy Group Order vs. #generators

32. 32 Saucy Group Order vs. #generators Guarantee to produce no more than n - 1 generators for n-vertex graph Guarantee to produce no more than n - 1 generators for n-vertex graph The number of reported generators is significantly less than n - 1 The number of reported generators is significantly less than n - 1

33. 33 Shatter +SBP Variables

34. 34 Shatter +SBP Clauses

35. 35 Shatter Variables and Clauses Shatter on 47 benchmarks Shatter on 47 benchmarks –Unsolved benchmarks or benchmarks with run time > 1000 sec. Application: 13 Application: 13 Crafted: 34 Crafted: 34 # added SBP clauses # added SBP clauses –Less than 4% for 29 benchmarks –Ranged from 25% to 133% for 18 benchmarks # added SBP variables # added SBP variables –Less than 1% for 23 benchmarks –Ranged from 9% to an order of magnitude for 24 benchmarks

36. 36 Shatter (Symmetry-Breaking) Flow Use shatter to generate SBPs Use shatter to generate SBPs Add SBPs to the original CNF formula Add SBPs to the original CNF formula Pass the augmented CNF formula to the SAT solver Pass the augmented CNF formula to the SAT solver Statistical data: Statistical data: –We used a re-ordering script to Reorder variables Reorder variables Reorder clauses Reorder clauses –20 re-ordered versions of each benchmark 10 for the original benchmarks 10 for the original benchmarks 10 for the SBP augmented benchmarks 10 for the SBP augmented benchmarks Time-outs Time-outs –Crafted: 5000 sec. –Application: 10000 sec.

37. 37 SAT Solver Run Time 13 out of 47 benchmarked finished within time-outs 13 out of 47 benchmarked finished within time-outs

38. 38 SAT Solver Run Time SBP augmented versions led to fewer time-outs SBP augmented versions led to fewer time-outs All but 3 benchmarks were solved faster All but 3 benchmarks were solved faster Four benchmarks which were reported to be unsolvable in SAT 2009 competition were solved with the addition of SBPs Four benchmarks which were reported to be unsolvable in SAT 2009 competition were solved with the addition of SBPs

39. 39 Conclusions and Future Work For SAT: symmetry discovery is practically free For SAT: symmetry discovery is practically free Static symmetry breaking Static symmetry breaking –Many CNF instances possess no or little symmetry –CNF instances with a lot of symmetry may or may not benefit from static symmetry breaking Future work: Future work: –SAT-inspired algorithmic enhancements: Branching heuristics Branching heuristics Learning Learning –Dynamic symmetry breaking: Integrating symmetry breaking within the SAT solver Integrating symmetry breaking within the SAT solver Uncovering hidden/conditional symmetries Uncovering hidden/conditional symmetries