# Computing the Density of States of Boolean Formulas Stefano Ermon, Carla Gomes, and Bart Selman Cornell University, September 2010.

## Presentation on theme: "Computing the Density of States of Boolean Formulas Stefano Ermon, Carla Gomes, and Bart Selman Cornell University, September 2010."— Presentation transcript:

Computing the Density of States of Boolean Formulas Stefano Ermon, Carla Gomes, and Bart Selman Cornell University, September 2010

From 100 variables, 200 constraints (early 90’s) to over 1,000,000 vars. and 5,000,000 clauses in 20 years. Applications: Hardware and Software Verification, Planning, Scheduling, Optimal Control, Protocol Design, Routing, Multi-agent systems, E-Commerce (E-auctions and electronic trading agents), etc. Motivation: Significant progress in SAT SAT: Given a Boolean formula Φ in CNF, Φ=C1ΛC2 Λ…ΛCm does Φ have a satisfying assignment?

Extending SAT technology Model counting problem (number of distinct satisfying assignments): probabilistic inference problems multi-agent / adversarial reasoning (bounded) [Roth ‘96, Littman et. al. ‘01, Sang et. al. ‘04, Darwiche ‘05, Domingos ‘06] MAX-SAT and Weighted MAX-SAT: find a truth assignment that maximizes the number of satisfied clauses or the sum of their weights beyond decision (NP) [Hansen at al. ’90] hard and soft constraints [Heras et al. ’08, Cohen et al. ’06] How can we combine both challenges?

Density of states Given a Boolean formula Φ in CNF, Φ=C 1 ΛC 2 Λ…ΛC m with m clauses the density of states is a function that gives the number of truth assignments that violate exactly i clauses, for i =0,..,m n(0) = number of assignments that violate 0 clauses (models) n(1) = number of assignments that violate exactly 1 clause

Density of states: a challenging problem Generalizes SAT  Decision problem: Φ is satisfiable if and only if n(0)>0 Generalizes MAX-SAT  MAX-SAT is the minimum i such that n(i)>0 Generalizes #SAT  Number of models = n(0)

Statistical physics More generally, the density of states (DOS) gives the number of microstates with energy E  Microstates = truth assignments  Energy = number of violated clauses  Ground states = maximally satisfying assignments Compact, very informative characterization of a physical system  Macroscopic thermodynamic quantities (free energy, internal energy,..)  Partition function, phase transitions,..

Motivation DOS provides a finer characterization of the structure of a combinatorial search space Statistical physics and CSPs:  Insights on problem structure, hardness, new algorithms, Survey Propagation [Montanari et. al. ‘07, Monasson et. al. ‘96, Mézard ’02, Parisi ‘02] By defining different energy functions, it can be naturally used for probabilistic style inference (e.g. Markov Logic, [Domingos ‘06] )

Talk Outline Prior work A novel sampling strategy: MCMC-FlatSat Empirical Validation  Small formulas with ground truth  Synthetic formulas  Random 3-SAT  Large structured instances  Model counting Conclusions

Density of states: prior work Exact Method: Enumeration (exponential) Approximate  Uniform Sampling [Belaidouni et. al. ‘02] Sample density from K random truth assignments Impractical, unlikely to hit rare assignments (e.g. solutions)  Metropolis Sampling [Rose et. al. ‘96] In theory, the density can be extracted from the Boltzmann distribution Impractical, difficult choice of the temperature and slow mixing times [Wei et. al. ‘04 ] How can we improve the sampling strategy?

The flat histogram idea Idea: Set up a Markov Chain that visits all energy levels equally often [F. Wang and Landau ’02, J. Wang et. al. ’99, De Oliveira et. al. ’96] e.g. an equal amount of time at the set of truth assignments with 0 unsat clauses, 1 unsat clause,... How? Flip a variable, accept new state σ’ with probability Always accepts “rarer” states (when n(E’) { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/14/4258343/slides/slide_10.jpg", "name": "The flat histogram idea Idea: Set up a Markov Chain that visits all energy levels equally often [F.", "description": "Wang and Landau ’02, J. Wang et. al. ’99, De Oliveira et. al. ’96] e.g. an equal amount of time at the set of truth assignments with 0 unsat clauses, 1 unsat clause,... How. Flip a variable, accept new state σ’ with probability Always accepts rarer states (when n(E’)

Example 1 1 1 1 1 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 3/10 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 3/10 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1 1 1 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1 11 1 1 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1 1 1 1 1 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1 1 1 1 1 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1/10 1 1/3 1/10 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example 1/10 1 1/3 1/10 RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10 Goal: visit all energy levels (colors) equally often

Example …and finally…

Example RED ↔ 0 unsat clauses, n(0)=1 GREEN ↔ 1 unsat clauses, n(1)=3 BLUE ↔ 2 unsat clauses, n(2)=10

Flat histogram: Intuition Detailed balance holds with those transition probabilities Properly biased towards “rare” states But there are few “rare” states The total time spent in each type of state is the same (flat visit histogram). (Red, Blue, Green) Note: in contrast, Simulated Annealing concentrates sampling around low energy states (more greedy!)

Flat histogram But… how can we even run the Markov Chain? Acceptance probability: The density n() is unknown and is precisely what we want to compute!

Adaptive sampling Start with an initial guess g (our estimate of the true density n) Random walk:  Use g for guidance (acceptance probability)  Chain will initially not sample uniformly across energy levels  Each step, adjust g using a modification factor F  Keep track of the visit histogram H When we see a flat H, we must have the right density!

The modification factor, F F controls the tradeoff between convergence rate and accuracy Use large modification factors F at the beginning to get  rough estimates  fast convergence Keep reducing F to get finer estimates Analogous to an annealing process

MCMC-FlatSat Inner Loop Initialization Inner Loop : adaptive sampling until the visit histogram is flat (g becomes our new guess for n)

MCMC-FlatSat Outer Loop Reduce modification factor and repeat inner loop until g≈n

Outline of empirical validation Empirical validation on combinatorial problems  Convergence  Efficiency (number of samples vs search space size)  Accuracy We study:  Small formulas with ground truth  Large synthetic formulas  Random 3-SAT  Large structured instances  Model counting

Formulas with known ground truth Instances from MAXSAT2007 competition (Ramsey, Spin Glass, Max Clique) Direct enumeration is possible (n<=28), so we can compare our estimate with ground truth Metrics:  KL divergence:  Relative error per point

Spin glass instance, 27 variables, 162 clauses Needs ~ 8 *10 6 flips << search space size (2 27 ≈1.3 *10 8 ) Energy (# unsat clauses) Log-density Rel. error (%) Energy 3.1 % maximum relative error n – ground truth g – estimate X

Instancevariablesclauses KL- divergenceMax rel. errorEntropy ramk3n7.ra021703.9 E-052.39 % 2.45 ramk3n8.ra0281261.1 E-055.1 % 3.93 johnson8-2-4.clq284204.5 E-055.5 % 2.90 T3pm3-5555.spn271621.3 E-053.15 % 3.34 Ramsey Max Clique Energy (# unsat clauses) Log-density Energy (# unsat clauses) Log-density n – ground truth g – estimate X ngng X

Synthetic formulas Ground truth for larger formulas?  Construct synthetic formulas Formulas for which we derive a closed form solution for the density Result on the composition (logical conjunction) of independent formulas (do not share variables) The density of F is the convolution of the density of Φ with itself l times

InstancevariablesclausesKL-divergenceMax relative error (%)Entropy UnifConv50_100501001.19 E-053.05 % 7.3 PigHoleConv4102007501.26 E-072.2 % 33.7 Convolution of uniform densities Convolution of pigeon hole formulas Energy (# unsat clauses) Log-density Energy (# unsat clauses) Log-density Needs ~ 2*10 7 flips << state space size (2 50 ≈10 15 ) n – ground truth g – estimate X ngng X

Random k-SAT formulas Well known phase transitions for the satisfiability property in terms of the ratio α (Φ is satisfiable ↔ n Φ (0)>0 ) Analytic result on the average density given a truth assignment, the probability of having a clause that is violated is 1/2 k for random k-SAT

n=50 variables (average over 1000 instances) Needs ~ 10 8 flips << state space size (2 50 ≈10 15 ) Average Densities Log-density Ratio clauses to var. α

Large structured instances No ground truth known Consistency checks  Number of models, when exact model counting is feasible (# models=n(0))  Method of the moments: Sample K assignments at random, compute their energies (unsat clauses) Compute sample moments (e.g. average energy, 2 nd order moment of energy,..) Compare with moments obtained using the estimated density g

Large structured instances Instancevariablesclausesg(0) # models Ms(1)M(1)Ms(2)M(2) brock400 2.clq 40118800297.0 88365.988372.3 Spinglass5.pm 12575000187.4 35249.235247.1 MANN a27.clq 42169000422.4 178709178703 bw large.a 459467511995.2995.3996349996634

Logistic instance Planning problem:  n=459 variables  m=4675 clauses Huge search space (2 459 truth assignments), but MCMC-FlatSat returns within hours Remarkable precision:  Finds the only existing model g(0)=n(0)=1!  The mode of the estimated distribution is e 300 times larger than the number of models g(0)  counts the needles and the haystack! (the moments method indicates g is accurate) Energy (# unsat clauses) Log-density

Model counting Comparison with state-of-the-art model counters  SampleCount [Gomes et. al. ‘06]  SampleMiniSAT [Gogate et. al. ‘07] MCMCFlatSat is very accurate Timings are competitive when ratio clauses to variables is not too large  DOS provides guidance  Information on what is not a model  Overhead because it provides more information

Model counting comparison Instancevariablesclauses Exact # models SampleCountSampleMiniSATMCMC-FlatSat Models Time (s)Models Time (s)Models Time (s) 2bitmax252766 2.10×10 29 >2.40×10 28 29 2.08×10 29 345 1.96×10 29 1863 wff-3-3.5150525 1.40×10 14 >1.60×10 13 145 1.60×10 13 240 1.34×10 14 393 wff-3.1.5100150 1.80×10 21 >1.00×10 20 240 1.58×10 21 128 1.83×10 21 21 wff-4-5.0100500 >8.00×10 15 120 1.09×10 17 191 8.64×10 16 189 ls8-norm3011603 5.40×10 11 >3.10×10 10 1140 2.22×10 11 168 5.93×10 11 2693

Conclusions Computing the density of states is a hard problem (encompasses SAT, MAX-SAT, #SAT) MCMCFlatSat: sampling strategy adapted from physics for combinatorial spaces that adaptively explores the space while collecting statistics  Extremely accurate, very efficient (few samples)  Provides a compact, rich description of the search space. New insights about structure and local search Very general method: any property can be used for search space partitioning.  Many applications to counting and inference problems

Extra slides

Future work SAT-specific improvements  Energy saturation  Energy barriers (and related normalization issues)  Walksat heuristics (with Metropolis-Hastings updates) Direct application to inference in Markov Logic Formal proof of convergence / counterexamples Application to other counting problems in combinatorial spaces

Runtime for random 3-SAT Search space 2^50 Flips 10^8 ≈ 2^26

Related work on random 3-SAT Lots of work on the i=0 (i.e. SAT/UNSAT) case [Gent et. al. ‘94] Previous experimental work for i>0 [Zhang ‘01]  Different definition: “no more than i unsat clauses” versus “exactly i unsat clauses”  Location on the phase transitions  Apparently, same location Analytic results [Achlioptas et al. ‘05] We can see two phase transitions: assignments are unlikely to violate a large number of clauses

Other phase transition 50 variables, 1000 instances

Other structured instances

Histogram flatness (formal) Flatness condition of the visit histogram H is a necessary condition for convergence  If g is equal to the true density n, the detailed balance is satisfied by upon convergence, the steady state probability is proportional to the reciprocal of the density of the corresponding energy level

Histogram flatness (formal) Flatness condition of the visit histogram H is a necessary condition for convergence If g=n, the visit histogram H will be flat i.e. energy levels visited equally often Problem: the density n() is unknown and is precisely what we want to compute!

Satifiability (SAT): Given a formula in propositional calculus, does it have a model, i.e., is there an assignment to its variables making it true? (a   b   c) AND (b   c) AND (a  c) SAT: prototypical hard combinatorial search and reasoning problem. Problem is NP-Complete. (Cook 1971) Propositional Satisfiability (SAT)

Prior work - Metropolis Approximate [Rose et. al., ‘96 ]  Metropolis sampling at temperature T  Energy = number of violated clauses  Measure the empirical probability P(E) of energy levels, rescale to get the density n(E)  Slow mixing times [Wei et. al., ‘04 ]  Difficult choice of the temperature

Theoretical Convergence Stated as a conjecture in Lee,Okabe and Landau, Convergence and Refinement of the Wang-Landau Algorithm, Computer Physics Communications, 2006 Proof of convergence in Atchade and Liu,The Wang-Landau Algorithm for Monte Carlo computation in general state spaces, Statistica Sinica, 2009

Closed forms k-SAT, m clauses such that each variable appears exactly in one clause, the DOS is where is the fraction of assignments of the variables in a single clause not satisfying it (e.g. in 3-SAT, only 1 over 8 assignments does not satisfy a clause)

Implementation details Use of log-densities New assignment generated by flipping a variable uniformly at random Flatness condition (within 10% of the max) Normalization  The density is obtained only up to a constant factor  We use the normalization constraint F0=1.5, reduce by Flatness is checked every 1000 moves

Random k-SAT formulas k-SAT: each clause has at most k literals Random k-SAT: formulas Φ generated with  n variables  m clauses produced independently by randomly choosing a set of k variables from n available negating each one with probability 0.5  α is the ratio clauses to variables (m/n) Let be the probability measure associated with this generative model for Φ

Composition Logical conjunction of formulas Φ that do not share variables The density of F is the convolution of the density of Φ with itself l times Analogous to the probability density of the sum of l independent random variables

Closed forms Starting with a formula Φ with uniform density (e.g. ) we construct Sum of n s-sided dices has closed form probability distribution:

Download ppt "Computing the Density of States of Boolean Formulas Stefano Ermon, Carla Gomes, and Bart Selman Cornell University, September 2010."

Similar presentations