Paul Beame University of Washington

Paul Beame University of Washington
The Limits of Proof Paul Beame University of Washington

Formal Proofs Gottlob Frege David Hilbert
“Begriffsschrift, a Formula Language, Modeled Upon That of Arithmetic, for Pure Thought” (1884) Invention of formally quantified variables “Grundgesetze der Arithmetik” (1893) “Basic Laws” David Hilbert “Grundlagen der Geometrie” (1899) “Foundations” - proper axioms/proofs that Euclid never had Math Congress (1900) – 23 problems “It shall be possible to establish the correctness of the solution by means of a finite number of steps based upon a finite number of hypotheses which are implied in the statement of the problem and which must always be exactly formulated.” Hilbert’s Program (1920s) – Mechanize math via formal proofs ∀𝒙 ∃𝒚 Euclid’s elements was early model of proof but our modern notion of formal proof is primarily due to these two. Unfortunately, Frege is mostly known for a negative story. “There is no ignoramibus in Mathematics”

Formal Proofs Bertrand Russell & Alfred North Whitehead Paradox (1901)
Principia Mathematica ( ) Used a formulation of the “barber paradox” about a barber who shaves everyone who doesn’t shave themselves and applied it to the “set of all sets” Principia tried to do for arithmetic what Hilbert had done for geometry but even with 3 volumes didn’t get very far.

Limits of Proof I Kurt Gödel Alan Turing
“On Formally Undecidable Propositions in Principia Mathematica and Related Systems I” (1931) First Incompleteness Theorem: Any consistent formal system S that can express basic arithmetic properties of natural numbers has statements that can neither be proved nor disproved in S. Alan Turing “On Computable Numbers, with an application to the Entscheidungsproblem” (1936) Undecidability of the Halting Problem: There is no algorithm to determine whether or not (other) arbitrary algorithms halt

Limits of Proof I Kurt Gödel Alan Turing
“On Formally Undecidable Propositions in Principia Mathematica and Related Systems I” (1931) First Incompleteness Theorem: Any consistent formal system S that can express basic arithmetic properties of natural numbers has statements that can neither be proved nor disproved in S. Alan Turing “On Computable Numbers, with an application to the Entscheidungsproblem” (1936) Undecidability of the Halting Problem: There is no system of proof that can prove whether or not arbitrary algorithms halt

Infinities in these limits to proof
Gödel’s Incompleteness Natural numbers ℕ={𝟎,𝟏,𝟐,𝟑,…,𝟏𝟎,…, 𝟏𝟎 𝟏𝟎 ,…, 𝟏𝟎 𝟏𝟎 𝟏𝟎 ,…, 𝟏𝟎 𝟏𝟎 𝟏𝟎 𝟏𝟎 ,…} Turing’s Undecidability of the Halting Problem Not halting ≡ running forever No impossibility for either if we don’t care about Numbers larger than 𝟏𝟎 𝟏𝟎 𝟏𝟎 or Running times larger than 𝟏𝟎 𝟏𝟎 𝟏𝟎

A finite universe and a finite lifetime
Planck length ≈ 1.6 x m ≈ 6.4 × 10−34 in Can store only a few bits (≈ 6) in a Planck length cube Our universe is ≈ 13.8 billion years old and (even with hyperinflation) ≈ 93.2 billion light years across So…volume of universe < < 2670 cubic Planck lengths age of universe < 1027 < 290 nanoseconds For comparison… 1 petabyte = 250 bytes = 253 bits year ≈ 255 nanoseconds

Computing in a finite world
Physical realizations of digital computers inspired concerns about efficiency of algorithms How quickly do they run given 𝒏 input bits? Time ≈ # steps Good: 𝟑𝒏+𝟓, 𝟐𝒏 log 𝟐 𝒏, 𝟒 𝒏 𝟐 , 𝒏 𝟑 +𝟑 𝒏 𝟐 +𝟐 - bounded by a polynomial function of the input size 𝒏: 𝒏 𝑶 𝟏 for short Bad: 𝟐 𝒏 , 𝟏.𝟏 𝒏 , 𝟐 𝒏 , … 1960s: Many problems have (surprisingly) good algorithms but others seem only to have bad ones… A lot of the initial concern by people like von Neuman was that computers that ran too long would break down.

Complexity class P Algorithm 𝑨 runs in polynomial time iff its # of steps on 𝒏-bit inputs grows as 𝒏 𝑶 𝟏 For simplicity, focus on decision problems and algorithms whose answer is true/false, yes/no 𝐏 = set of all decision problems with polynomial time algorithms This has a consequence for our understanding of our finite world.

Reasoning about finite worlds
Without loss of generality Can describe their states and actions on them in finite sequences of bits Propositional (Boolean) logic Variables 𝒙 𝟏 , 𝒙 𝟐 ,…, 𝒙 𝒏 each of which can be True/False, 1/0 represent bits of the state or actions or time Connectives AND ∧ OR ∨ NOT ¬ IMPLIES → Formulas e.g. ¬𝒙⟶ 𝒚∧𝒛 ∧ 𝒙→¬ 𝒚∨𝒛 George Boole “The Laws of Thought” (1854)

Boolean formula properties
Boolean formulas express constraints on the world (Finite) world ≡ possible assignment of true/false values to the Boolean variables Properties of Boolean formulas: Satisfiable: true in some worlds e.g. 𝒙∧ ¬𝒙∨𝒚 Tautology: true in all worlds, e.g. 𝒙∧ ¬𝒙∨𝒚 →𝒚 Unsatisfiable: false in all worlds e.g. 𝒙∧¬𝒙 , ¬((𝒙∧ ¬𝒙∨𝒚 →𝒚) Let’s focus on the first of these properties

Satisfiability (SAT) Problem
Given a Boolean formula 𝑭 as input, is 𝑭 satisfiable? If 𝑭 has 𝒏 Boolean variables, we can try all 𝟐 𝒏 truth assignments to those variables. Is SAT in 𝐏? That’s open A related problem with a polynomial-time algorithm 𝑽 Given a Boolean formula 𝑭 and a truth assignment 𝒕, does 𝒕 make 𝑭 true? 𝑭 is SAT ⇔ ∃𝒕. 𝒕 makes 𝑭 true ⇔ ∃𝒕. 𝑽(𝑭,𝒕) is true 𝒕 is a proof that 𝑭 is SAT. 𝒕 is also short 𝒕 ≤ 𝑭 𝑶 𝟏 The general class of problems with this property is called NP

NP and NP-completeness
𝐍𝐏 – nondeterministic polynomial-time Decision problems 𝑫 with a polynomial-time (verification) algorithm 𝑽 𝑫 s.t. 𝒙 is a YES for 𝑫 iff there is a short “proof” 𝒚 such that 𝑽 𝑫 (𝒙,𝒚) outputs true Thm: SAT is “NP-complete”, a hardest problem in NP If SAT is in P then P=NP Stephen A. Cook “The Complexity of Theorem-Proving Procedures” (1971) Leonid Levin “Universal Sequential Search Problems” (1973) NP generalizes the property of SAT we saw to arbitrary decision problems. Neither Cook nor Levin was aware of the following back-story

Boolean formula properties
Boolean formulas express constraints on the world (Finite) world ≡ possible assignment of true/false values to the Boolean variables Properties of Boolean formulas: Satisfiable: true in some worlds In NP, NP-complete e.g. 𝒙∧ ¬𝒙∨𝒚 In P? Tautology: true in all worlds, ?? e.g. 𝒙∧ ¬𝒙∨𝒚 →𝒚 Unsatisfiable: false in all worlds ?? e.g. 𝒙∧¬𝒙 , ¬((𝒙∧ ¬𝒙∨𝒚 →𝒚) Let’s focus on tautology and unsatisfiability which are the typical notion of proof

What constitutes a proof?
Conventional textbook proofs: Axioms: e.g. Excluded Middle 𝑨∨¬𝑨 Inference Rules: e.g. Modus Ponens 𝑨, 𝑨→𝑩 ∴𝑩 Proof: A sequence of lines 𝑭 𝟏 , 𝑭 𝟐 ,…, 𝑭 𝒕 each of which either follows from an axiom, or follows from previous lines via an inference rule. Proves 𝑭 𝒕 Truth tables are also proofs: 𝒙 𝒚 ¬𝒙 ¬𝒙∨𝒚 𝒙∧(¬𝒙∨𝒚) 𝒙∧ ¬𝒙∨𝒚 →𝒚 F T

What constitutes a proof?
Need a system for expressing proofs such that proofs are “easy to check”: Proof system ≡ polynomial-time verification algorithm 𝑽 taking 𝑭 and a supposed proof 𝑷 that is: Sound: If 𝑽(𝑭,𝑷) outputs “true” then 𝑭 is a tautology* Complete: For every tautology* 𝑭 there is a 𝑷 such that 𝑽(𝑭,𝑷) outputs “true”. This is a lot like the definition of NP Except no requirement that 𝑷 is short. Cook, Reckhow (1975) “The Relative Complexity of Propositional Proof Systems” *or unsatisfiable using ¬𝑭 instead of 𝑭

Proof Complexity The complexity of a proof system is a function 𝑪:ℕ→ℕ s.t. A proof of 𝑭 exists ⇔ a proof of length ≤𝑪 𝑭 exists e.g. Truth tables have complexity ≥ 𝟐 𝒏 If 𝑪(𝒏) could be polynomial in 𝒏 then unsatisfiability* is in NP If that is impossible for all proof systems for unsatisfiability* then P ≠ NP Would imply complement of SAT is not in NP, but complement of any problem in P is also in P, since SAT∊ NP we have SAT ∉ P Cook, Reckhow (1975) “The Relative Complexity of Propositional Proof Systems” *or tautology using ¬𝑭 instead of 𝑭

Proof Complexity Proof complexity bounds nondeterministic algorithms
It only measures how big proofs have to be Finding these proofs may be much harder! Many proof systems, e.g. Any deterministic algorithm 𝑨 for SAT running in time 𝑻(𝒏) ⇒ proof system for unsatisfiability* with complexity ≈ 𝑻(𝒏) The transcript of the execution of 𝑨 on input 𝑭 that fails to find a satisfying assignment is a proof that none exists. Proof complexity program: Prove that specific proof systems of increasing strength are not polynomially bounded Steps towards proving P ≠ NP (and NP ≠ coNP) Understanding these proof systems may also be useful in practice How do we compare the strength of proof systems? *or tautology using ¬𝑭 instead of 𝑭

Comparing Proof Systems [Cook-Reckhow 1975]
Proof system 𝑽′ simulates proof system 𝑽 ⇔ proofs in 𝑽 can be efficiently converted to proofs in 𝑽’ So proofs in 𝑽’ can be at most polynomially longer than in 𝑽 Proof systems 𝑽 and 𝑽’ are equivalent iff each polynomially simulates the other

Frege Systems [Cook-Reckhow 1975]
Inference systems given by a finite set of axioms, e.g. excluded middle 𝑨∨¬𝑨 inference rules, e.g. modus ponens 𝑨, 𝑨→𝑩 | ∴𝑩 Axioms and inference rules are schemas applied under any consistent substitution of formulas e.g., by excluded middle can conclude (𝒙𝒚)  (𝒙𝒚) A Frege system is any such system that is general enough to make all possible inferences (after some # of steps) Theorem [Cook-Reckhow 1975] All Frege systems are equivalent We can begin by simplifying what we need to prove.

CNF Formulas A Boolean formula is a CNF iff it is in this form: ¬𝒙 𝟏 ∨ 𝒙 𝟑 ∧ 𝒙 𝟐 ∨ ¬𝒙 𝟑 ∨ 𝒙 𝟒 ∨ 𝒙 𝟓 ∧…∧ 𝒙 𝟏 ∨¬ 𝒙 𝟐 ∨ 𝒙 𝟒 Tseitin (1968): Linear time transformation converting any Boolean formula 𝑭 into a CNF formula 𝑭’ s.t 𝑭′ is unsatisfiable ⇔ 𝑭 is unsatisfiable. So… can assume w.l.o.g. that input formula is a CNF and goal is to prove that it is unsatisfiable Used as part of proof that 3SAT is NP-complete literals clauses CNF = “Conjunctive Normal Form”

CNF Formulas and Circuits
Boolean circuits Directed acyclic graphs Nodes labeled by input variables, or connectives (gates) Generalize Boolean formulas Can efficiently simulate any polytime computation ∨ 𝒘 𝒕 𝒗 𝒖 𝒑 𝒒 𝒓 𝒔 𝒙 𝒚 𝒛 𝒆 𝒅 ∨ ∧ ∧ ∨ ∧ ∨ ∨ ∨ ∧ 𝒙 𝟏 𝒙 𝟐 𝒙 𝟑 𝒙 𝟒 𝒙 𝟓 Tseitin transformation: Add an extra variable to represent value of each gate CNF is ∧ of clauses saying each gate correct: e.g. ¬𝒘∨𝒖∨𝒗 , (¬𝒖∨𝒘) , ¬𝒗∨𝒘 plus clause saying output gate is true: 𝒕 This property of Boolean circuits is a key behind the ability of SAT to simulate all of NP. CNF formulas are much simpler to represent than general formulas, what about with proofs.

Resolution proofs Resolution proofs [Robinson 1965]:
Refutation system for CNF input proves unsatisfiability Every proof line is a clause. Each line is either: a clause of the input CNF, or follows from two prior clauses via the resolution rule 𝑨 ∨ 𝒙, 𝑩 ∨ ¬𝒙 ∴ 𝑨 ∨ 𝑩 Goal: Derive the empty clause ⊥ Sound and complete; useful in practice.

Resolution and Formal Methods in CS
Since 2000, formal methods for analyzing software and hardware, based on SAT Solvers have revolutionized the field General Idea: Write CNF formula expressing the constraints on any bug Any satisfying assignment is a bug Correctness of system ≡ CNF formula is unsatisfiable. Modern SAT solvers are Conflict-Directed Clause-Learning (CDCL) extensions of DPLL algorithms. CDCL solvers often very successful despite CNF formulas having 10,000s-100,000s of variables 1,000,000s of clauses

Resolution and Formal Methods in CS
DPLL: Davis, Putnam (1960) Proof method for first-order logic based on refutation search for CNF formulas Davis, Logeman, Loveland (1962) Version of DP (1960) using backtracking search for satisfying assignments for CNF formulas instead of refutation search Theorem: For any UNSAT formula 𝑭, the trace of DPLL search on 𝑭 ≡ resolution refutation of 𝑭 in which no derived clause is reused CDCL extensions: beginning in late 1990s, early 2000s Add new learned clauses to formula at every backtrack Theorem [B-Kautz-Sabharwal 2004, Pipatsrisawat-Darwiche 2008]: For any UNSAT formula 𝑭, the trace of a CDCL search on 𝑭 ≋ resolution refutation of 𝑭 Also many other items like restarts.

The Limits of Resolution
The Pigeonhole Principle CNF formula 𝑷𝑯 𝑷 𝒏 𝒎 for any 𝒎>𝒏 Expresses “Exists a 1-1 map from 𝒎 to [𝒏]” 𝒙 𝒊𝒋 : pigeon 𝒊 maps to hole 𝒋 Clauses: 𝒙 𝒊𝟏 ∨…∨ 𝒙 𝒊𝒏 ¬ 𝒙 𝒊𝒋 ∨¬ 𝒙 𝒊 𝒋 ′ ¬ 𝒙 𝒊𝒋 ∨¬ 𝒙 𝒊 ′ 𝒋 for 𝒊≠ 𝒊 ′ ∈ 𝒎 𝒋≠ 𝒋 ′ ∈[𝒏]

Theorem [Haken 1984] : Every Resolution refutation of 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 requires size ≥𝟐 𝒄𝒏 for some 𝒄>𝟎. “Resolution can’t count” Razborov (2003), building on [Raz 2000], showed that for any value of 𝒎>𝒏, 𝑷𝑯 𝑷 𝒏 𝒎 requires Resolution refutation size 𝟐 𝛀 𝒏 𝟏/𝟑 . Note: In practice, SAT solvers can’t refute 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 when 𝒏≥𝟐𝟎. Not just one hard problem for Resolution

Random 𝒌-CNF formulas for 𝒌≥𝟑 Choose 𝒎=𝚫𝒏 uniformly random clauses of size 𝒌. When 𝚫 is big enough this is almost surely unsatisfiable. Almost surely every Resolution refutation requires size 𝟐 𝛀(𝒏 𝚫 𝟏 (𝒌−𝟏) ) . [Chvatal-Szemeredi 1988, B-Karp-Pitassi-Saks 1998, Ben-Sasson Wigderson 2001] General lower bound technique: [Ben-Sasson Wigderson 2001] Show that every Resolution refutation requires at least one clause of size 𝒘 (the width of the refutation) If original CNF formula only has clauses of length ≤𝒘/𝟐 then every Resolution refutation requires size 𝟐 𝛀 𝒘 𝟐 𝒏

Formulas of larger depth than CNF?
CNF formulas in the input is no loss but during the proofs it seems to be CNF formulas are big ∧’s of big ∨’s of literals If 𝒅 alternations of big ∧’s and big ∨’s for constant 𝒅 then we get “𝑨 𝑪 𝟎 formulas of depth 𝒅”. Theorem [Ajtai 1988, Pitassi-B-Impagliazzo, Krajicek-Pudlak-Woods 1992]: Any Frege refutation of 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 only using 𝑨 𝑪 𝟎 formulas of depth ≤𝒅 requires size ≥𝟐 𝒏 𝜺 𝒅 , for some 𝜺 𝒅 ≥𝟎. Fact [Buss 1987]: Frege can refute 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 in 𝒏 𝑶 𝟏 size Open: Refuting random 𝒌-CNF formulas using 𝑨 𝑪 𝟎 -Frege?

More powerful logical proof systems
In Frege systems the only variables allowed are the original variables One cannot define new ones the only schemas are the axioms and inference rules Even when one has proved that they are tautologies If one allows either Extension (even of Resolution) by the definition of new variables, or Substitution of arbitrary formulas into any proven formula then one gets Extended-Frege ≡ proofs in which the lines are circuits, not just formulas Can even define proof systems with quantified proof lines e.g., Zermelo-Frankel set theory with Axiom of Choice

Some Proof System Relationships
ZFC Extended Frege Simulation Frege Simulation + Exponential Separation TC0-Frege TC0-Frege: weakening of Frege proofs ≡ each proof line a constant depth neural net TC0-Frege can efficiently refute 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 AC0-Frege No nontrivial lower bounds known Some strong lower bounds known Resolution Many strong lower bounds known DPLL Truth Tables

Thinking outside the logic box
There are many other NP-hard problems that we can use to express unsatisfiability. Proof complexity has explored a rich variety of techniques using expressions and theorems from: Algebra Simultaneous polynomial equations Nullstellensatz, Polynomial Calculus proofs Optimization 01-Integer Programming Cutting Planes, Sherali-Adams, Sum-of-Squares, Lovasz-Schrijver, Positivstellensatz calculus proofs In turn, the proof complexity results have fed back into our understanding of many other NP-hard problems.

Thinking outside the logic box: Algebra
CNF constraints are easy to express Given: ¬𝒙 𝟏 ∨ 𝒙 𝟑 ∧…∧ 𝒙 𝟏 ∨¬ 𝒙 𝟐 ∨ 𝒙 𝟒 Algebra: polynomial equalities over any field: Clauses: 𝒙 𝟏 𝟏− 𝒙 𝟑 =𝟎,…, 𝟏− 𝒙 𝟏 𝒙 𝟐 𝟏− 𝒙 𝟒 =𝟎 Values in 0,1 : 𝒙 𝟏 𝟐 − 𝒙 𝟏 =𝟎, …, 𝒙 𝟒 𝟐 − 𝒙 𝟒 =𝟎 Or, using two algebraic variables per logical variable: Clauses: 𝒙 𝟏 𝒙 𝟑 ′ =𝟎, …, 𝒙 𝟏 ′ 𝒙 𝟐 𝒙 𝟒 ′ =𝟎 Complements: 𝒙 𝟏 + 𝒙 𝟏 ′ =𝟏, …, 𝒙 𝟒 + 𝒙 𝟒 ′ =𝟏 Values in 0,1 : 𝒙 𝟏 𝟐 − 𝒙 𝟏 =𝟎, …, 𝒙 𝟒 𝟐 − 𝒙 𝟒 =𝟎

Algebraic reasoning Hilbert’s Nullstellensatz: System of polynomials 𝒇 𝟏 𝒙 𝟏 ,…, 𝒙 𝒏 =𝟎, …, 𝒇 𝒎 𝒙 𝟏 ,…, 𝒙 𝒏 =𝟎 over field 𝔽 has no solution in any extension field of 𝔽 ⇔ there exist polynomials 𝒈 𝟏 , …, 𝒈 𝒎 over field 𝔽 such that 𝒊=𝟏 𝒎 𝒇 𝒊 𝒈 𝒊 ≡𝟏 Nullstellensatz proof system over 𝔽: Refutation is the sequence 𝒈 𝟏 , …, 𝒈 𝒎 for 𝒇 𝟏 , …, 𝒇 𝒎 polynomial translation of CNF 𝒙 𝒊 𝟐 − 𝒙 𝒊 =𝟎 ensures any solution in 0,1 polynomials can be made multilinear (and so degree ≤𝒏 ) Natural measures: degree, # of monomials in 𝒈 𝒊 One go further with this [B-Impagliazzo-Krajicek-Pitassi-Pudlak 1994]

Algebraic reasoning Polynomial Calculus (PC) proof system over 𝔽:
Dynamic version of Nullstellensatz refutations Refutation lines, each a polynomial 𝒇( 𝒙 𝟏 ,…, 𝒙 𝒏 )=𝟎 Each line is either An input polynomial equation from the CNF translation 𝒂𝒇+𝒃𝒈=𝟎 for polynomials 𝒇=𝟎 and 𝒈=𝟎 on previous lines, 𝒂,𝒃∈𝔽 𝒙 𝒊 𝒇=𝟎 for a polynomial 𝒇=𝟎 from a previous line, 𝒊∈[𝒏] Goal 𝟏=𝟎 Size= total # of monomials, degree = max degree of any line PC Refutations of degree ≤𝒅 can be found in time 𝒏 𝑶 𝒅 by the Gröbner basis algorithm/iterated linear algebra [Clegg-Edmonds-Impagliazzo 1996]

Polynomial Calculus simulates Resolution
Given (𝒙∨¬𝒑∨𝒕) and (¬𝒙∨𝒒∨¬𝒔∨𝒕) infer (¬𝒑∨𝒒∨¬𝒔∨𝒕) Polynomial Calculus: 𝒙 ′ 𝒑𝒕′ Given 𝒙 𝒒 ′ 𝒔 𝒕 ′ Given 𝒙 ′ 𝒑 𝒒 ′ 𝒔𝒕′ from 1 𝒙𝒑 𝒒 ′ 𝒔𝒕′ from 2 𝒙+ 𝒙 ′ 𝒑 𝒒 ′ 𝒔𝒕′ from 3 + 4 𝟏−(𝒙+ 𝒙 ′ ) Given 𝟏− 𝒙+ 𝒙 ′ 𝒑 𝒒 ′ 𝒔𝒕′ from 6 𝒑 𝒒 ′ 𝒔𝒕′ from 5 + 7 This is typical of how these simulations work

The Limits of Polynomial Calculus
Lower bound technique: [Clegg-Edmonds-Impagliazzo 1996] Show that every refutation requires degree ≥𝒅 If original CNF formula only has clauses of length ≤𝒅/𝟐 then every PC refutation requires size 𝟐 𝛀 𝒅 𝟐 𝒏 Both Pigeonhole principle 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 and random 𝒌-CNF formulas require 𝟐 𝛀 𝒏 PC refutations [Razborov 1996], [Impagliazzo-Pudlak-Sgall 1999], [Ben-Sasson Impagliazzo 1999], [Alekhnovich-Razborov 2003] Choice of field 𝔽 can make a big difference: PC (and even Nullstellensatz) in characteristic 𝒑 can efficiently prove properties that are exponential in characteristic 𝒒 [Buss-Grigoriev-Impagliazzo-Pitassi 2001]

Thinking outside the logic box: Optimization
Given: ¬𝒙 𝟏 ∨ 𝒙 𝟑 ∧…∧ 𝒙 𝟏 ∨¬ 𝒙 𝟐 ∨ 𝒙 𝟒 Optimization: Inequalities over the reals Clauses: 𝟏− 𝒙 𝟏 + 𝒙 𝟑 ≥𝟏,…, 𝒙 𝟏 + 𝟏− 𝒙 𝟐 + 𝒙 𝟒 ≥𝟏 Values in [0,1]: 𝒙 𝟏 ≥𝟎, 𝒙 𝟏 ≤𝟏,…, 𝒙 𝟒 ≥𝟎, 𝒙 𝟒 ≤𝟏 Derive: −𝟏≥𝟎 So far: just linear programming (LP) Need to use fact that values are in {0,1} Not the whole real interval [0,1] Two options: Cutting Planes proofs Higher degree inequalities using 𝒙 𝟐 −𝒙=𝟎

Cutting Planes proofs [Gomory 1959], [Chvatal 1993]
Lines have integer coefficients and RHS Addition: Multiplication by positive integer: Division by positive integer: 𝒂 𝟏 𝒙 𝟏 +…+ 𝒂 𝒏 𝒙 𝒏 ≥𝑨 𝒃 𝟏 𝒙 𝟏 +…+ 𝒃 𝒏 𝒙 𝒏 ≥𝑩 __________________________________________________________________________________________________________________ ∴ 𝒂 𝟏 + 𝒃 𝟏 𝒙 𝟏 +…+ 𝒂 𝒏 + 𝒃 𝒏 𝒙 𝒏 ≥𝑨+𝑩 𝒂 𝟏 𝒙 𝟏 +…+ 𝒂 𝒏 𝒙 𝒏 ≥𝑨 ________________________________________________________________________ ∴ 𝒄 𝒂 𝟏 𝒙 𝟏 +…+𝒄 𝒂 𝒏 𝒙 𝒏 ≥𝒄𝑨 Why cutting planes in the name? 𝒄𝒂 𝟏 𝒙 𝟏 +…+ 𝒄𝒂 𝒏 𝒙 𝒏 ≥𝑩 __________________________________________________________________________ ∴ 𝒂 𝟏 𝒙 𝟏 +…+ 𝒂 𝒏 𝒙 𝒏 ≥ 𝑩 𝒄 e.g. 𝟐𝒙+𝟐𝒚+𝟐𝒛≥𝟑 implies 𝒙+𝒚+𝒛≥𝟐

Cutting Planes Geometry

Higher Degree: Sherali-Adams
Use translated clause constraints 𝒇 𝟏 ≥𝟎,…, 𝒇 𝒎 ≥𝟎 plus 𝒙 𝒊 ≥𝟎, 𝒙 𝒊 ′ ≥𝟎, 𝟏− 𝒙 𝒊 − 𝒙 𝒊 ′ =𝟎, 𝒙 𝒊 𝟐 − 𝒙 𝒊 =𝟎 Sherali-Adams proofs [Sherali-Adams 1990] Refutation is set of 𝒄 𝑨,𝑩,𝒋 ≥𝟎 plus sequence of polynomials 𝒈 𝒊 , 𝒉 𝒊 such that 𝒋 ( 𝑨,𝑩 𝒄 𝑨,𝑩,𝒋 𝒊∈𝑨 𝒙 𝒊 𝒊∈𝑩 𝒙 𝒊 ′ )⋅𝒇 𝒋 + 𝒊 𝒈 𝒊 𝟏− 𝒙 𝒊 − 𝒙 𝒊 ′ + 𝒊 𝒉 𝒊 𝒙 𝒊 𝟐 − 𝒙 𝒊 =−𝟏 Degree 𝒅 = maximum degree of any polynomial on LHS Size = # of monomials on LHS which is ≤ 𝒏 𝑶 𝒅 . Also 𝒏 𝑶 𝒅 algorithm. Note that the polynomial multiplying each f_j is always nonnegative

Positivstellensatz 𝒋 ( 𝒌 𝒑 𝒋,𝒌 𝟐 )⋅ 𝒇 𝒋 + 𝒌 𝒑 𝟎,𝒌 𝟐 + 𝒊 𝒈 𝒊 𝒉 𝒊 ≡𝒇
Positivstellensatz: System of linear inequalities and polynomial equations 𝒇 𝟏 𝒙 𝟏 ,…, 𝒙 𝒏 ≥𝟎, …, 𝒇 𝒎 𝒙 𝟏 ,…, 𝒙 𝒏 ≥𝟎, 𝒉 𝟏 𝒙 𝟏 ,…, 𝒙 𝒏 =𝟎,…, 𝒉 𝒎 ′ 𝒙 𝟏 ,…, 𝒙 𝒏 =𝟎 over a compact set 𝑲⊂ ℝ 𝒏 implies that polynomial 𝒇 𝒙 𝟏 ,…, 𝒙 𝒏 ≥𝟎 on 𝑲 ⇔ there exist polynomials 𝒑 𝒋,𝒌 and 𝒈 𝒊 such that 𝒋 ( 𝒌 𝒑 𝒋,𝒌 𝟐 )⋅ 𝒇 𝒋 + 𝒌 𝒑 𝟎,𝒌 𝟐 + 𝒊 𝒈 𝒊 𝒉 𝒊 ≡𝒇 Generalizes Nullstellensatz over ℝ Simplified special case of Krivine-Stengle Positivstellensatz [Krivine 1964], [Stengle 1974] due to Putinar [Putinar 1993]

Higher Degree: Sum of Squares
Use translated clause constraints 𝒇 𝟏 ≥𝟎,…, 𝒇 𝒎 ≥𝟎 plus 𝒙 𝒊 𝟐 − 𝒙 𝒊 =𝟎 Sum-of-Squares (SOS) proofs [Parrilo 2000], [Lasserre 2001], [Grigoriev-Vorobjov 2001] Refutation is sequence of polynomials 𝒑 𝒋,𝒌 , 𝒈 𝒊 , s.t. 𝒋 ( 𝒌 𝒑 𝒋,𝒌 𝟐 ) ⋅𝒇 𝒋 𝒌 𝒑 𝟎,𝒌 𝟐 𝒊 𝒈 𝒊 𝒙 𝒊 𝟐 − 𝒙 𝒊 =−𝟏 Degree 𝒅 = maximum degree of any polynomial on LHS. Size = # of monomials on LHS which is ≤ 𝒏 𝑶 𝒅 Also 𝒏 𝑶 𝒅 algorithm to find refutations using semi-definite programming. (equivalent systems under different names Positivstellensatz, Lasserre)

Optimization and refutation
These proof systems can start with constraints that don’t come from CNF formulas For optimization problems Low degree SOS can derive the constraints given by semi- definite programming algorithms e.g., Goemans-Williamson algorithm for MAXCUT can be captured by degree 2 SOS. SOS degree lower bounds imply lower bounds on many of our best optimization algorithms

Extension Complexity [Yannakakis 1991]
Many problems have natural expression as optimization over a 01-polytope w/ exponential # of facets input instance given by the objective function Extension approach Add extra variables to make # of facets smaller Use polytime Linear Programming algorithm LP extension complexity = min # of facets of any extension Theorem [Kothari-Meka-Raghavendra 2016]: LP extension complexity for approximating any constraint satisfaction problem (CSP), is 𝒏 𝛀 𝒅 where 𝒅 is the Sherali-Adams degree required for the approximation. Corollary: LP Extension complexity for MAXCUT 𝟏 𝟐 +𝝐 approx or MAX-3SAT 𝟕 𝟖 +𝝐 approx is 𝟐 𝒏 𝛀 𝟏 .

SDP Extension Complexity [Gouveia-Parrilo-Thomas 2011], [Fiorini-Massar-Pokutta-Tiwari-de Wolf 2012]
SDP Extension approach Start with same 01-polytope formulation but lift to a semi- definite programming (SDP) formulation w/ extra variables Use polytime SDP algorithm SDP extension complexity = min # of SDP constraints of any extension Theorem [Lee-Raghavendra-Steurer 2015]: For any CSP, the SDP extension complexity is 𝒏 𝑶 𝟏 iff it has SOS degree 𝑶(𝟏). Theorem [Lee-Raghavendra-Steurer 2015]: SDP extension complexity of TSP, Clique is 𝟐 𝒏 𝛀 𝟏 . Theorem [Grigoriev 2001]: SOS degree of inferring 𝒊=𝟏 𝟐𝒏+𝟏 𝒙 𝒊 ≤𝒏 from 𝒊=𝟏 𝟐𝒏+𝟏 𝟐𝒙 𝒊 ≤𝟐𝒏+𝟏 is 𝛀 𝒏 . Proof relies on:

Dynamism Sherali-Adams and SOS are static proof systems like Nullstellensatz Polynomial calculus is dynamic But SOS can efficiently simulate it over ℝ [Berkholz 2017] We also have dynamic systems for inequalities (e.g. Lovasz-Schrijver, Positivstellensatz calculus) that can use rules such as the following: Given 𝒇≥𝟎 and 𝒈≥𝟎 can infer 𝒇+𝒈≥𝟎, 𝒇⋅𝒈≥𝟎, 𝒄𝒇≥𝟎 for positive 𝒄 𝒑 𝟐 ≥𝟎 for any polynomial 𝒑. Goal: infer −𝟏≥𝟎 Theorem: [B-Pitassi-Segerlind 2006,Lee-Shraibman 2008, Chattopadhyay-Ada 2008,Göös-Pitassi-Watson 2012] For a dynamic system with degree ≤𝒅≤ 𝐥𝐨𝐠 𝟏−𝝐 𝒏 polynomial inequalities that can’t reuse derived formulas, “𝒅+𝟏 lifts of Tseitin formulas” require size 𝟐 𝒏 𝛀 𝟏

Proof System Relationships
ZFC Extended Frege Simulation Frege Simulation + Exponential Separation Positivstellensatz Calculus TC0-Frege SOS/Lasserre Cutting Planes AC0-Frege Sherali-Adams Polynomial Calculusℝ Resolution Nullstellensatzℝ Incomparable DPLL 1-way separation Truth Tables

Positivstellensatz Calculus
Complexity of 𝑷𝑯 𝑷 𝒏 𝒏+𝟏 ZFC Extended Frege Frege Easy Positivstellensatz Calculus TC0-Frege SOS/Lasserre Cutting Planes AC0-Frege Sherali-Adams Polynomial Calculusℝ Resolution Nullstellensatzℝ Exponentially hard DPLL Truth Tables

Complexity of random 𝑘-CNF
ZFC Not known to be easy for any proof system! Extended Frege Frege Random 3XOR 𝑶(log⁡𝒏)-CNF Positivstellensatz Calculus TC0-Frege SOS/Lasserre Cutting Planes AC0-Frege Sherali-Adams Polynomial Calculusℝ Resolution Nullstellensatzℝ DPLL Exponentially hard Truth Tables

The Power of Proof Complexity
Proof complexity captures many of our best approaches towards solve hard problems Practical approaches for formal methods and verification Methods such as LP, SDP for optimization problems Rich field of study with many open problems

Thanks!

Paul Beame University of Washington

Similar presentations

Presentation on theme: "Paul Beame University of Washington"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Paul Beame University of Washington

Similar presentations

Presentation on theme: "Paul Beame University of Washington"— Presentation transcript:

Similar presentations

About project

Feedback