Boaz Barak – Microsoft Research Partially based on joint work with Jonathan Kelner (MIT) and David Steurer (Cornell) Sum of Squares Proofs and The Quest Towards Optimal Algorithms SURGEON GENERAL’S WARNING: Talk contains only “pseudo theorems”

The space of algorithms is very rich - a different algorithm for every problem

A pipe dream… One algorithm for all problems? Answer 1: Yes for non-interesting reasons. Proof: Enumerate over all Turing machines and try them all.. Answer 2: Maybe there is a deeper reason.. Few unifying principles underlie many algorithms. Convexity, matroid structure, submodularity, algebraic identities,..

Hope: An optimal meta algorithm: Why care? A pipe dream… One algorithm for all problems? Unified approach to classifying easy vs. hard problems Hardness*/algorithmic results are often very challenging. Proving specific algorithm fails much easier than ruling out all algorithms. Understanding why problem easy/hard rather than laundry-list of results.

A generic “meta algorithm” for polynomial optimization [Shor’87,Nesterov’00,Parillo’00,Lasserre’01] Algorithmic version of works related to Hilbert’s 17 th problem [Artin 27,Krivine64,Stengle74] Used in many applications including quantum information theory, control theory, automated theorem proving, game theory, and more… Can it be an optimal meta algorithm? Image credit: Chakraborty et al The Sum of Squares Algorithm Generalizes many known algorithms. Empirically seems to work well. Theoretical analysis lags far behind. See also Laurent’s talk

This talk: Dual views of SOS alg: Positivstellensatz/SOS proofs “Pseudo distributions” Example application: Dictionary Learning / Sparse Coding Relation to Khot’s Unique Games Conjecture Image credits: Georghiades et al, Mairal et al, Singer, Kindler et al

Polynomial Equations Extremely general framework: All poly’s of degree 1: Linear programming Single quadratic: Least squares / eigenvalue problem Poly’s of degree >1: Captures a great many problems, some NP hard Can similarly encode many other problems, including SAT, 3COL, Max-Cut, etc…

Empirically often converges much faster, still not very well understood. Some evidence and intuition, still not at the level of a precise conjecture. Next: What is a “proof” and what is a “pseudo-solution” SOS Algorithm – high level view:

SOS Proofs An SOS proof deduces polynomial inequalities using the following rules: SOS proofs surprisingly powerful: Capture many standard tools such as Cauchy-Schwarz, Hölder, etc.. and even more advanced notions (e.g., Isoperimetric results on Boolean cube) Only examples of assertions that robustly* require large degree to prove are non-constructive, shown using probabilistic method.

What is a “pseudo-solution”? Next: 1) Detour: defining statistical knowledge via distributions 2) Definition of pseudo solutions. SOS Algorithm – high level view:

Detour: Knowledge as a distribution As you learn more information, you adjust your distribution accordingly.

Detour: Knowledge as a distribution As you learn more information, you adjust your distribution accordingly. i.e., all information is known, but we are computationally bounded and can’t make all possible logical inferences from it. Can we do the same for computational knowledge?

Computational knowledge as a pseudo-distribution Notes:

An SOS proof deduces polynomial inequalities using the following rules:

This talk: Dual views of SOS alg: Positivstellensatz/SOS proofs “Pseudo distributions” Example application: Dictionary Learning / Sparse Coding Relation to Khot’s Unique Games Conjecture Image credits: Georghiades et al, Mairal et al, Singer, Kindler et al

Find the “right” representation of observed data LOTS of work: important primitive in Machine Learning, Vision, Neuroscience... Example Application: Dictionary Learning / Sparse Coding [Olhausen-Field ’96] [Mairal-Bach-Ponce-Sapiro ’10]

Example Application: Dictionary Learning / Sparse Coding (have no control over local maxima) Proof of (*) uses low degree SOS arguments Can show (*) using Hölder-type inequalities.

This talk: Dual views of SOS alg: Positivstellensatz/SOS proofs “Pseudo distributions” Example application: Dictionary Learning / Sparse Coding Relation to Khot’s Unique Games Conjecture Image credits: Georghiades et al, Mairal et al, Singer, Kindler et al

Unique Games Conjecture Conjecture [Khot ‘02] : Certain problem known as “Unique Games” is NP hard. Many implications to complexity: Implies hardness results problems across many domains including constraint satisfaction, cut and routing, scheduling, algebra and more. If true then yields optimality of many classical algorithms (e.g. Grothendieck, Cheeger-Alon-Milman, Geomans-Williamson, etc..). Fascinating connections to areas including probability, geometry, metric embeddings, social choice theory, etc.. E.g. Khot/Arora/O’Donnell’s talks ; surveys [Khot, ‘10 ‘10 ‘14], [Trevisan ‘12] vs.

Main reasons to believe UGC: Can’t refute it: Don’t know of an algorithm that solves it. Want it to be true: Gives very clean picture of complexity landscape. Algorithmic attacks on UGC: “Eigenspace enumeration”: Brute force search in top eigenspace of adj. matrix SOS Algorithm: Generalizes both Solves random instances … [Arora-Khot-Kolla-Steurer-Tulsiani-Vishnoi’05] but not all instances [Khot-Vishnoi’04] Solves KV instance [Kolla’10] [Arora-B-Steurer-’10]..and every instance in subexp time [B-Gopalan-Håstad-Meka-Raghavendra-Steurer’12]..but not much better know an algorithm, don’t have proof Solves KV+BGHMRS instances [B-Brandao-Harrow-Kelner-Steurer-Zhou ‘12] [B-Kelner-Steurer ‘14].. candidate algorithm to solve all instances sort of

Conclusions Maybe talk on refuting UGC via SOS at ICM 2018?