1. BIT-VECTOR REWRITING WITH AUTOMATIC RULE GENERATION Alexander Nadel Intel Haifa, Israel CAV 2014 Vienna, Austria July 22 th, 2014

2. Agenda Introduction to Bit-Vector Solving and Rewriting Automatic Rewriting-based Preprocessing Experimental Results 2

3. Bit-Vector (BV) Language Variables and constants are bit-vectors of user-defined width: width 1 (Bool): 0 width 8: [01011110] width 30: [010111100100011101000011001101] Operators: Core: =, , if-then-else Bit-wise: ~, &, |, ^, … Arithmetic: +, -, *, /, … Comparison:,  Bit-propagating: concat, extract, >, … Assertions: a Boolean literal must hold 3

4. 4 BV Formula Example a = 1 Assertions Widths: w(a) = 1; w(v  {b,c,d,e,f,g,h,i,j,k,l}) = 4 cbfj xnor d = a + e + g - h / i -1=[1111 ] & k 0=[0000 ] << l op1 op2 Input DAG op1 op2 BV solving is widely used (in SW and HW validation; scheduling; number theory; automatic configuration; …)

5. BV Solver Architecture 5 Preprocessing SATDPLL(BV) EagerLazy

6. BV Solver Architecture 6 Preprocessing Rewriting: preprocessor’s bread and button SATDPLL(BV) EagerLazy

7. xnor 7 Rewriting Example DAG Rewriting Rules Premise Conclusion y+0y 0+zz 1*zz y*1y y*2y<<1 y<<0y y [w] <<(n  w) 0 y | -1 y & -1y -1 & zz hundreds of rules in practice! cbfj = a + e + g - h / i -1=[1111 ] & k 0=[0000 ] << l op1 op2 Input DAG op1 op2 a = 1 Assertions d

8. 8 Rewriting Example DAG Rewriting Rules Premise Conclusion y+0y 0+zz 1*zz y*1y y*2y<<1 y<<0y y [w] <<(n  w) 0 y | -1 y & -1y -1 & zz hundreds of rules in practice! op1 op2 a = 1 Assertions cbfj = a + e + g - h / i -1=[1111 ] & k 0=[0000 ] << l op1 op2 Input DAG op1 op2 d xnor

9. 9 Rewriting Example DAG Rewriting Rules Premise Conclusion y+0y 0+zz 1*zz y*1y y*2y<<1 y<<0y y [w] <<(n  w) 0 y | -1 y & -1y -1 & zz hundreds of rules in practice! a = 1 Assertions cbfj = a + e + g - h / i -1=[1111 ] & k op1 op2 Input DAG d xnor

10. 10 Rewriting Example DAG Rewriting Rules Premise Conclusion y+0y 0+zz 1*zz y*1y y*2y<<1 y<<0y y [w] <<(n  w) 0 y | -1 y & -1y -1 & zz hundreds of rules in practice! a = 1 Assertions cbf = a + e + g - h / i -1=[1111 ] & k op1 op2 Input DAG j op1 d xnor

11. 11 Rewriting Example DAG Rewriting Rules Premise Conclusion y+0y 0+zz 1*zz y*1y y*2y<<1 y<<0y y [w] <<(n  w) 0 y | -1 y & -1y -1 & zz hundreds of rules in practice! a = 1 Assertions cbf = a + e + g - h / i op1 op2 Input DAG d j op1 xnor

12. 12 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

13. 13 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

14. 14 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

15. 15 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

16. 16 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

17. 17 Common Approach Our Approach Our approach: comments When are the rules created? OfflineRun-time Algorithm starts with an empty set of rules How are the rules generated? ManuallyAutomatically A SAT-based procedure Where are the rules stored? Hard-codedHash table Are the rules instance-specific NoYes -Generates rules only for operators & widths appearing in the instance -Won’t miss width-specific rules, e.g.: y+2 = y–2 for width 2 Rewriting framework DAG-based rewriting 0-saturation -0-saturation: -equivalence & constant propagation, we extend from propositional to BV reasoning -applied after the DAG is ready  our approach can be applied after the common approach

18. Next: the New Algorithm in Action 18

19. d = b xnor c a = b == c g = e + d h = f - g e = f + d i = j / h Triplets (a triplet: x = y op z) Generating Input to 0-saturation 19 Input to 0-saturation w(a) = 1; w(v  {b,c,d,e,f,g,h,i,j}) = 4 cbfj xnor d = a + e + g - h \ i Input DAG (potentially after rewriting) a = 1 Assertions op2 op1 w(a) = 1; w(v  {b,c,d,e,f,g,h,i,j}) = 4

20. a = b == c 20 Evaluation QueueBindings d = b xnor c g = e + d h = f - g e = f + d i = j / h a = 11 = b == c

21. 21 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h Triplets to be evaluated Evaluation of a triplet may yield new bindings or a contradiction. An evaluated triplet either moves to the “active” state or becomes a tautology and removed Active triplets Evaluated non-tautological triplets become active Any triplet one of whose variables is bound (replaced by another) is moved back to the evaluation queue Active triplets comprise the output of 0- saturation Bindings Equivalences entailed by the formula

22. 22 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h Rules Automatically generated rules for evaluating triplets Empty in the beginning Instance-specific rules are generated on-the-fly A rule: premise  conclusion per concrete operation and width Premise: a condition that must hold for the triplet to apply the rule Conclusion is either: A set of bindings A contradiction Empty

23. 23 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h

24. 24 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h

25. 25 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h Given a triplet x = y op z, look for a rule if: any v  {x,y,z} is a rewriting constant or a rewriting function of u  v  {x,y,z} Premise Detection: Rewriting constants 0 = [00…000] -1 = [11…111] 1 = [00…001] -2 = [11…110] 2 = [00…010] Rewriting functions Equivalence: f 1 (v) = v Unary minus: f 6 (v) = −v Negation: f 7 (v) = −v − 1 = ∼ v Neighbors: f 2 (v) = v − 1 = ∼ −v f 3 (v) = v − 2 = ∼ − ∼ −v f 4 (v) = v + 1 = − ∼ v f 5 (v) = v + 2 = − ∼ − ∼ v Unary minus neighbors: f 8 (v) = −v − 2 = ∼ − ∼ v f 9 (v) = −v + 1 = − ∼ −v f 10 (v) = −v + 2 = − ∼ − ∼ −v

26. 26 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h 1.Premise detection is successful: x is the rewriting constant 1! 2.Do any rules with this premise exist? 1.Yes  apply the rule 2.No  generate a conclusion

27. 27 1 = y == z Check if at least one of v  {x,y,z} is a rewriting constant or a rewriting function of u  v  {x,y,z} and generate the corresponding bindings Conclusion Generation Example Generate a CNF corresponding to the formula: y=0? Add clauses for y  0 with a selector s Solve all the clauses under the assumption s with incremental SAT Satisfiable: no rule can be learnt w(x) = 1; w(y)=w(z)=4

28. 28 1 = y == z y=z? Add clauses for y  z with a selector: Solve all the clauses under the assumption s with incremental SAT Unsatisfiable; rule learnt: x=1  y=z for operation ==; width 4 Conclusion Generation Example Generate a CNF corresponding to the formula: w(x) = 1; w(y)=w(z)=4 Check if at least one of v  {x,y,z} is a rewriting constant or a rewriting function of u  v  {x,y,z} and generate the corresponding bindings

29. 29 d = b xnor c 1 = b == c g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h OpWPrem.Concl. ==4x=1y=z b = c

30. 30 d = b xnor b g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h b = c OpWPrem.Concl. ==4x=1y=z

31. 31 d = b xnor b g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h b = c OpWPrem.Concl. ==4x=1y=z

32. 32 d = b xnor b g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h 1.Premise detection is successful: y is the rewriting function f 1 (v)=v of z! 2.Do any rules with this premise exist? No  generate a conclusion with incremental SAT b = c OpWPrem.Concl. ==4x=1y=z

33. 33 d = b xnor b g = e + d h = f - g Evaluation QueueActive TripletsBindingsRules e = f + d i = j / h d=-1 b = c OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1

34. 34 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules e = f - 1 i = j / h d=-1 b = c OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1

35. 35 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules e = f - 1 i = j / h d=-1 b = c OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1

36. 36 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules e = f - 1 i = j / h 1.Premise detection is successful: z is the rewriting constant 1 1.Do any rules with this premise exist? No  generate a conclusion with incremental SAT d=-1 b = c OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1

37. OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y 37 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules e = f - 1 i = j / h d=-1 b = c ~e=-f

38. 38 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

39. 39 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

40. 40 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f 1.Premise detection is successful: z is the rewriting constant 1 1.Do any rules with this premise exist? Yes  apply the rule OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

41. 41 g = e - 1 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f ~g=-e OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

42. 42 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f ~g=-e OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

43. 43 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f ~g=-e OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

44. 44 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f ~g=-e 1.Premise detection is successful: -~-~g = -~-(-e) = -~e = -(-f) = f, thus g = ~-~-f = f-2 Discovered in O(1) based on bindings 2.Do any rules with this premise exist? No  generate a conclusion with incremental SAT OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y

45. OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2 45 h = f - g Evaluation QueueActive TripletsBindingsRules i = j / h d=-1 b = c ~e=-f ~g=-e h=2

46. 46 Evaluation QueueActive TripletsBindingsRules i = j / 2 d=-1 b = c ~e=-f ~g=-e h=2 OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2

47. 47 Evaluation QueueActive TripletsBindingsRules i = j / 2 d=-1 b = c ~e=-f ~g=-e h=2 OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2

48. 48 Evaluation QueueActive TripletsBindingsRules i = j / 2 d=-1 b = c ~e=-f ~g=-e h=2 1.Premise detection is successful: z=2 2.Do any rules with this premise exist? No  generate a conclusion using incremental SAT solving OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2

49. OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2 /4z=2empty 49 Evaluation QueueActive TripletsBindingsRules i = j / 2 d=-1 b = c ~e=-f ~g=-e h=2 1.Premise detection is successful: z=2 2.Do any rules with this premise exist? No  generate a conclusion with incremental SAT

50. 1.0-saturation is completed 2.In eager BV solving, the active triplets are bit- blasted to CNF and solved with SAT 50 Evaluation QueueActive TripletsBindingsRules i = j / 2 d=-1 b = c ~e=-f ~g=-e h=2 OpWPrem.Concl. ==4x=1y=z xnor 4y=zx=-1 -4z=1x=y-1  ~x=-y -4z=y-2x=2 /4z=2empty

51. Experimental Results Solvers: Hazel: Intel’s eager BV solver Base Hazel: Hazel without the new approach New Hazel: Hazel with the new approach Boolector (version 1.6.0) Mathsat (version 5.2.10; SMT’11 config.) Benchmarks: 23 families of the ASP category of SMT-LIB 11 difficult instances from Intel’s HW validation flow Time-out: 20 min. for SMT-LIB 40 min. for industrial instances 51

52. 52 New Hazel outperforms base Hazel on 20/23 ASP families ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100

53. 53 ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100 New Hazel outperforms base Hazel significantly on 13/23 ASP families: it either solves more instances or is at least 2x faster

54. 54 ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100 New Hazel outperforms all the other solvers on 14/23 ASP families

55. 55 ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100 New Hazel outperforms all the other solvers significantly on 10/23 ASP families: it either solves more instances or is at least 2x faster

56. Experimental Results over 23 ASP Families: Observations Summary Few rules with non-empty conclusion are generated, but are applied very frequently 2-12 rules are generated on average per family Rules are applied almost 200,000 times on average 0-saturation run-time overhead is negligible 1% on average overall <5% is the worst result per family 56

57. New Hazel vs. Base Hazel vs. Boolector on Industrial Benchmarks 57 Benchmark New HazelBase HazelBoolector TimeTO?TimeTO?TimeTO? 1 6.05207.828041.2660 2 53.147065.196019.2690 3 49.919052.063053.2870 4 54.891066.64024001 5 113.371046.974016.7970 6 118.799087.061011.10 7 99.97091.125016.9450 8 172.486025.953020.1250 9 174.942024001 1 10 23.3530240013.5040 11 648.70802400184.8850 Grand Total1515.63807642.8435067.1782

58. 58

59. Interesting Rules (that is, Rules with a Non-Empty Conclusion) Number per ASP Family 59

60. Interesting Rules Application Number per ASP Family 60

61. 0-saturation Average Run-time per ASP Family (Including Rule Generation) as Function of Speed-Up of New Hazel vs. Base Hazel 61

62. Percentage of CNF Clauses per ASP Family in New Hazel w.r.t Base Hazel 62

63. BV Solving BV theory: a powerful and flexible language which encompasses C language operators Wide-spread usage: SW validation HW validation Number theory Combinatorial problems Automated configuration Scheduling … Many solvers: Boolector, STP, CVC, Z3, Mathsat, Intel’s Hazel, … 63

64. 64 ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100 New Hazel outperforms Mathsat on 21/23 ASP families (while base Hazel outperforms Mathsat on 18/23 ASP families)

65. 65 ASP Family New HazelBase HazelBoolectorMathsat #TimeTO'sTimeTO'sTimeTO'sTimeTO's DisjunctiveScheduling 45150480043582225232 Solitaire 23220012760023582512480218 Labyrinth 106996010342812000101200010 EdgeMatching 2953450250731930693233239124 WeightBoundedDominatingSet 29259102562516555022022810 Sudoku 8322905918472136 6 GraphColouring 1233621495242771156744 GraphPartitioning 72050251411418029392 Fastfood 1711520427223869064873 HamiltonianPath 292694136783271040363 Sokoban 298330266208720157008 HierarchicalClustering 125601230116502670 15Puzzle 1516803590477026540 Hanoi 15101020001345015840 GeneralizedSlitherlink 2922003520122605060 ChannelRouting 81201801170600 KnightTour 3128011798167013471 BlockedNQueens 29117375162135102303052122 SchurNumbers 2966433693831053080614 WireRouting 198567587125633001651112 TravellingSalesperson 2934800293480029644903333926 ConnectedDominatingSet 214990382029406120 MazeGeneration 293543125180879026100 New Hazel outperforms Boolector on 14/23 ASP families (while base Hazel outperforms Boolector on 8/23 ASP families only)