Survey Propagation

Outline Survey Propagation: an algorithm for satisfiability 1 – Warning Propagation – Belief Propagation – Survey Propagation Survey Propagation Revisited 2 – SP’s relation to BP – Covers 1 Braunstein, A., Mézard, M., and Zecchina, R Survey propagation: An algorithm for satisfiability. Random Struct. Algorithms 27, 2 (Sep. 2005), Kroc, L.; Sabharwal, A.; and Selman, B Survey propagation revisited. In UAl 2007.

Survey Propagation and Message Passing New threshold defining two regions of solutions, easy SAT and hard SAT – Easy SAT region has many, highly-clustered solutions, and any two solutions are connected by a few number of steps – Hard SAT region has many clusters of solutions, with fewer overall solutions than easy SAT SP is an efficient way of computing solutions in the Hard SAT region

New Model - Factor Graph

Message Passing Treat the Factor Graph as a network Depending on the current state of the network, messages will be passed between the nodes These messages will later be used to determine the final value of a variable

Cavity Fields Used to determine the value of a given message, and varies depending on the actual algorithm in use Calculated by removing a function node from consideration

Warning Propagation Messages are from the set {0,1} Passed from Function nodes to Variable Nodes – Product of cavity fields Initially randomly generate messages Calculate new warnings for edges until we finish the time, or we hit a fixed point for all edges – New warnings are calculated from the cavity fields (∑ b Є V+(j)\a u b→j ) – (∑ b Є V-(j)\a u b→j )

WP Example Local Field Contradiction Numbers The whole algorithm is an iterative process involving multiple WP executions, simplifying the graph using local fields

Belief Propagation Finds the total number of solutions, as well as the fraction of assignments with variable X i is true Two types of messages: – From Function node to Variable node Given a value for X i, the probability that it satisfies the function clause – From Variable node to Function node Probability that variable i takes value X i in the absence of function clause a

Satisfiable and Unsatisfiable Sets

BP Example Blue is the cavity field of the variable node, red is the message from function to variable. Feed these values into another function to compute probabilities of being true

WP and BP Work 100% of the time when the factor graph is a tree Can be used as heuristics when it is not a tree, but no guarantee on solution Goal: Create a more efficient scheme for the general case

Basics of Survey Propagation Generally the same as BP, but deals better with clusters Details of the message change – A message sent from function to variable is the probability that a warning is sent to the variable node – A warning is sent if all other variable nodes connected to the function do not satisfy the function. Note: Behaves exactly like WP on tree factor graphs

Clustering of Configurations SAT Assignments The problem with BP’s approximations is that they do not hold globally, but do hold if you restrict the computation to a specific cluster. SP fixes this.

SP Results with Don’t Care State

Testing Data

Results New way to solve SAT using cavity fields and message passing Works well on generic input, as well as the tree structure Has not been rigorously examined, but is based off of similar work

Why does SP Work? As of 2008, SP is the only method of solving SAT problems with 1,000,000 variables (and larger) Does so in near linear time (even in the hardest region) SP behaves like backtrack searches, except it almost never has to backtrack

SP’s relation to BP BP works well when the number of SAT clusters is low SP seems to work well ALL the time Work in 2005 shows that SP is equivalent to BP search over a new object, covers This is a generalization of the clusters described previously

Covers σ Є {0,1,*}ⁿ is a cover of F if: 1.Every clause has at least one satisfying literal, or two *-ed literals 2.No unsupported variables are assigned 0 or 1 X = Y = Z = (x V ¬y V ¬z) Λ (¬x V y V ¬z) Λ (¬x V ¬y V z)

Computing Covers New Factor Graphs:

Solving SAT using Covers Let P(F) be a cover of F: The factor graph of P(F) can bee seen as a Bayesian Network, so marginals can be computed using Bayes’ Theorem These marginals, when fed to the BP algorithm, computes a solution for the original SAT problem (and results in the SP algorithm). The SP algorithm has been shown to compute marginals over these covers.

SP, BP and Marginals Magnetization of a Variable: The difference between the marginal being positive vs. negative

Results The reason why SP works so well seems to be related to these covers These covers seem to expose hidden properties of a SAT instance, such as backbone variables

Interesting Aside SAT and Decision SAT are self-reducible Finding covers is NOT self-reducible (x V ¬y V ¬z) Λ (¬x V y V ¬z) Λ (¬x V ¬y V z) (y V ¬z) Λ (¬y V z) (¬z) x = 1?Oracle: yes y = 0?Oracle: yes (1,0,0) a cover of the original? No!