On the Unique Games Conjecture Subhash Khot NYU Courant CCC, June 10, 2010

Approximation Algorithms A C-approximation algorithm for an NP-complete problem computes (C > 1), for problem instance I, solution A(I) s.t. Minimization problems : A(I)  C  OPT(I) Maximization problems : A(I)  OPT(I) / C

PCP Theorem [B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92] Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * Satisfiable (i.e. OPT = 1) or * No assignment satisfies more than 99% clauses (i.e. OPT  0.99). i.e. MAX-3SAT is 1.01 hard to approximate.

(In)approximability : Towards Tight Hardness Results [Hastad’96] Clique n 1-  [Hastad’97] MAX-3SAT 8/7 -  [Feige’98] Set Cover (1-  ) ln n [Dinur’05] Combinatorial Proof of PCP Theorem !

Open Problems in (In) Approximability –Vertex Cover (1.36 vs. 2) [DinurSafra’02] –Coloring 3-colorable graphs (5 vs. n 3/14 ) [ KhannaLinialSafra’93, BlumKarger’97 ] –Sparsest Cut (1+ε vs. (logn) 1/2 ) [AMS’07, AroraRaoVazirani’04 ] –Max Cut (17/16 vs 1/0.878… ) [ Håstad’97, GoemansWilliamson’94] ………………………..

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction) Supporting Evidence

Example of Unique Game (2CSP) OPT = max fraction of equations that can be satisfied by any assignment. x 1 - x 3 = 2 (mod k) x 5 - x 2 = -1 (mod k) x 2 - x 1 = k-7 (mod k) ………….

Unique Game 2CSP w/ Permutation Constraints variable k labels Here k=4 constraints 

Unique Game 2CSP w/ Permutation Constraints variable k labels Here k=4 Permutations or matchings  : [k]  [k]

OPT(G) = 6/7 Find a labeling that satisfies max # constraints Unique Game

Unique Games Considered before …… [Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G). How hard is approximating OPT(G) ? Observation : Easy to decide whether OPT(G) = 1.

Unique Games Conjecture For any , , there is integer k( ,  ), s.t. it is NP-hard to tell whether a Unique Game with k = k( ,  ) labels has OPT  1-  or OPT   i.e. Gap-Unique Game (1- ,  ) is NP-hard. Gap Projection Game (1,  ) is NP-hard. [ PCP Theorem + Raz’s Parallel Repetition Theorem ].

Supporting Evidence [UGC] Gap-Unique Game (1-ε,  ) is NP-hard. [Feige Reichman’04] Gap-Unique Game (C ,  ) is NP-hard. However C  --> 0 as C --> ∞. [KV’05] SDP relaxation for UG has “integrality gap” (1- ,  ). [KV’05] UGC based predictions were proven correct. Specifically, metric embedding lower bounds. [Wishful thinking] “There is structure in CS/math”.

Small Set Expansion Conjecture [Raghavendra Steurer’ 10] Φ (S ) = Edge expansion of set S. For every ε > 0, there exists δ > 0, such that, it is NP-hard to tell whether in a graph G(V,E), - There is a set S, |S| = δ |V|, Φ (S) ≤ ε. - For every set S, |S| ≈ δ |V|, Φ (S) ≥ 1- ε. [Raghavendra Steurer’ 10] SSE Conjecture  Unique Games Conjecture.

S = Optimal labeling. |S| = 1/k |G ’|. Φ(S) = 1- OPT(G). Unique Game and Small Set Expansion |G’| = n k. Unique Game G with n variables, k labels

Linear Equations Over Reals [K Moshkovitz’10] Homogeneous 3LIN(R): x 1 – x x 5 = 0. ∙∙∙∙∙∙∙∙ eq: x i +.5 x j - x k = 0. Theorem: It is NP-hard to tell if : There is a “non-trivial” solution that satisfies 1-ε fraction of equations. Any “non-trivial” solution fails on a constant fraction of equations with error Ω(√ε). 3LIN(R) to 2LIN(R) reduction ? 2LIN(R) ≈ Sparsest Cut

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction)

Generic Reduction from Unique Game [BGS’95 (Long Code), Hastad’97 (Fourier), UGC, ……]

Gadget: {-1,1} k PCP Reduction k labels Unique Game Instance MAX-CUT Instance OPT(UG) > 1-ε    sized cut. OPT(UG) < δ  No cut with size arccos (1-2  ) /  Match Goemans-Williamson’s SDP rounding Algorithm 1/0.878… Hardness

Gadget : Dictatorships (Long Codes) Weighted graph, total edge weight = 1. Picking random edge : x  R {-1,1} k y <-- flip every co-ordinate of x with probability  (   0.8) Noise-sensitivity graph. x {-1,1} k y - Consider f: {-1, 1} k  {-1,1}, i.e. Cuts. - Encode label i Є {1,2,…., k} by dictatorship function f(x) = x i.

Gadget: Cut that “commits” to co-ordintae i Fraction of edges cut = Pr (x,y) [x i  y i ] =  Observation : These are the maximum cuts. x i = 1 x i = -1

Gadget : Cuts not committing to a co-ordinate How large can be cuts with no influential co-ordinate ? Random Cut : ½ Majority Cut :  > arccos (1-2  ) /  > ½ [KKMO’04, MOO’05] Majority Is Stablest (Under Noise) Any cut with no influential co-ordinate has size at most arccos (1-2  ) / . Influence (i, f) = Pr x [ f(x)  f(x+e i ) ]

Integrality Gap Given : Maximization Problem + SDP relaxation. For every problem instance G, SDP(G)  OPT(G) Integrality Gap = Sup G SDP(G) / OPT(G)

[Raghavendra’ 08] Duality between Algorithms and Hardness. For every CSP, write a natural SDP relaxation. Integrality gap = β. Implies β-approximation. Theorem: Every instance with gap β’ < β can be used to construct a gadget and prove UGC-based β’- hardness result ! SDPs are optimal algorithm for CSPs.

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction)

Inapproximability and Fourier Analysis f : {-1,+1} k  {-1,+1}, balanced. Sparsest Cut [KV’05, CKKRS’05] [KKL’88] f has a co-ordinate with influence Ω(log k /k). [Bourgain’02] If NS ε (f) << √ε, then f depends essentially on exp(1/ε 2 ) co-ordinates. MAX-CUT [KKMO’04] Majority Is Stablest [MOO’05] If f has no influential co-ordinate, then NS ε (f) ≥ NS ε (Majority) - o(1).

Inapproximability and Fourier f : {-1,+1} k  {-1,+1}, balanced. Vertex Cover [DinurSafra’02, K Regev’03, K Bansal’09] [Friedgut’98] If total influence is k, then f depends essentially on exp(k) co-ordintaes. [MOO’05] It Ain’t Over Till It’s Over If f has no influential co-ordinate, then on almost every subcube of {-1, +1} k of dimension k/100, f = 1 and f = -1 with constant probability.

Inapproximability and Fourier f : {-1,+1} k  {-1,+1}, balanced. MAX-k-CSP [Samorodintsky Trevisan ’06] If f has no influential co-ordinate, then f has low Gowers’ Uniformity norm. Open: f: [q] k  [q], q ≥ 3, no influential co-ordinate. f balanced. Is Plurality Stablest ? What is the maximum Fourier mass at the first level ?

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction)

Disproving UGC means.. For small enough (constant) , given a UG with optimum 1- , algorithm that finds a labeling satisfying (say) 50% constraints, irrespective of k = #labels.

Algorithmic Results Algorithm that finds a labeling satisfying f( , k, n) fraction of constraints. [K’02] 1-  1/5 k 2 [Trevisan’05] 1-  1/3 log 1/3 n [Gupta Talwar’05] 1-  log n [CMM’05] 1/k , 1-  1/2 log 1/2 k [CMM’06] 1-  log 1/2 k log 1/2 n [AKKSTV’08, Kolla’10] UG on “mild” expander graphs. [ABS’10] Exp ( n  ) time algorithm. None of these disproves UGC. However …

If the UGC is true, then : k >> 2 1/ε. Graph of constraints cannot even be a “mild” expander. UG is easy on random graphs. Reduction from 3SAT must blow up the size by n 1/ε. Conjecture does not hold for sub-constant ε, i.e. below 1/log n.

SDP Relaxation of Unique Games [FL’92] OPT(G) = 1- ε  SDP(G) ≥ 1- ε. For i = 1, …, k,  u i, v i  ≥ 1- ε, up to permutation of indices. Orthonormal Bases for R k u 1, u 2, …, u k v 1, v 2, …, v k variables k labels Matchings [k]  [k] u v

[K’02, CMM’05] Rounding Algorithm u1u1 ukuk u2u2 vkvk v2v2 v1v1 r r Pick the label closest to r. Label(u) = Label(v) = 2. Pr [ Label(u) = Label(v) ] > 1 -  1/5 k 2 [K’02]. Pr [ Label(u) = Label(v) ] > 1-  1/2 log 1/2 k [CMM’05].  Labeling satisfies 1-  1/2 log 1/2 k fraction of constraints in expected sense. Random r u v

[Trevisan’05] Algorithm [Leighton Rao’88] Delete 1% of edges so that all connected components have diameter O(log n). Algorithm to solve UG on low diameter graph. Graph of variables and constraints

[AKKSTV’08 Algorithm] Algorithm that works on a UG instance s.t. 1-ε satisfiable and, Every balanced cut in the graph cuts at least Ω ( √ε ) fraction of edges. SDP-based. “Mild” expansion  Almost all SDP vector tuples are nearly identical  Yields a good labeling.

S = Optimal labeling. |S| = 1/k |G ’|. Φ(S) = 1- OPT(G). Unique Game and Small Set Expansion Label extended Graph |G’| = n k. UG G with n variables, k labels

[Arora Barak Steurer’10 Algorithm] [Kolla’10, Naor’10] Algo. runs in time exp(n ε ) on UG that is 1-ε satisfiable. Good solution to UG  Small non-expanding set S in G’. Small non-expanding set in label-extended graph G’ Either corresponds to a good UG solution (useful) Or is a non-expanding set in G (fake). Iteratively remove all fake sets from G, sacrificing at most 1% edges.

[Arora Barak Steurer’10 Algorithm] [Kolla’10, Naor’10] Main Lemma (Algorithmic) : If every set of size n 1-ε expands by Ω(ε 2 ), then the number of eigenvalues exceeding 1-ε is n O(ε). The UG solution is found in the linear span of eigenvectors with eigenvalues ≥ 1-ε. [Kolla’10] Run-time exp ( n O(ε) ).

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction)

(Gaussian) Isoperimetry. [MOO’05] Majority Is Stablest reduces via Invariance Principle, to a geometric question: P: R n  {-1,+1} be a partition of Gaussian space into two sets of equal measure. NS ε (P) = Pr [ P(x) ≠ P(y) ], Cor (x,y) = 1-2ε. Which P minimizes the noise-sensitivity? [Borell’85] NS ε (P) ≥ NS ε ( HALF-SPACE THRU ORIGIN ).

(Gaussian) Isoperimetry Open: q ≥ 3. More Invariance. [IM’10] MAX-q-CUT Problem. Plurality is Stablest Conjecture. Partition R n into q equal parts. (Geometric): Standard Simplex Conjecture. [K Naor’08] Kernel Clustering Problem. Maximizing Fourier Mass at First Level. (Geometric): Propeller Conjecture.

Integrality Gap [Feige Schechtman’01] [Goemans Williamson’92] 1/ Integrality gap for MAX-CUT. SDP with “triangle inequality constraints” ? ω(1) Integrality gap for Sparsest Cut? UGC  NP-hardness  These integrality gaps exist.

[KV’05] Integrality Gap for Unique Games SDP Unique Game G with OPT(G) = o(1) SDP(G) = 1-o(1) Orthonormal Bases for R k u 1, u 2, …, u k v 1, v 2, …, v k variables k labels Matchings [k]  [k] u v

Integrality Gap for MAX-CUT with Triangle Inequality {-1,1} k u 1, u 2, …, u k  u 1  u 2  u 3 ………  u k-1  u k PCP Reduction OPT(G) = o(1) No large cut Good MAX-CUT SDP solution

MAX-CUT and Sparsest Cut I.G. [KV’05] MAX-CUT gap matching Goemans-Williamson even with triangle inequality constraints. [KV’05, KrauthgamerRabani’05, DKSV’06]  ( loglog n) integrality gap for Sparsest Cut SDP.  An n-point “negative type” metric that needs distortion  ( loglog n) to embed into L 1.  Refutation of [Goemans Linial’97, ARV’04] conjectures. [KS’09, RS’09] Similar gaps for SDP + Sherali-Adams LP. Negative type metric that is L 1 embeddable locally but not globally.

Open Problems Integrality gaps for the Lasserre SDP Relaxation? Lasserre Relaxation could potentially disprove UGC. Sparsest Cut (NEG versus L 1 Metrics) : [ARV’04, AroraLeeNaor’05] O(√log n). [LeeNaor’06, CheegerKleiner’06, CKNaor’09]. Ω(log c n), c = ½?

Unique Games Conjecture [K’02] Connections to: Inapproximability (UGC  several problems inapproximable) Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) Geometry (Isoperimetry, Metric geometry, Integrality gaps) Algorithms (Attempts to disprove UGC) Parallel Repetition (Gap amplification, Foam construction)

Gap Amplification Prove UGC in two steps (?): Prove “mild” hardness, i.e. GapUG (1-ε’, 1-ε’’ ) is hard. Amplify gap via parallel repetition to GapUG (1-ε, δ). Note however that even proving “mild” hardness is a huge challenge.

Strong Parallel Repetition ? OPT(G) = 1-ε. [Raz’98] OPT(G m ) ≤ (1-ε 32 ) m/log k. 2P1R Games [Holenstein’07] OPT(G m ) ≤ (1-ε 3 ) m/log k. 2P1R Games [Rao’08] OPT(G m ) ≤ (1-ε 2 ) m. Projection Games (UG). GapUG (1-ε, δ ) is NP-hard iff GapUG (1-ε, 1 - √ε C(ε) ) is NP-hard where C(ε) –> ∞ as ε –> 0. [Raz’08] The rate (1-ε 2 ) m cannot be improved further.

Raz’s Example  Optimal Foam Problem: Tiling R d using a “shape” of unit volume and minimum surface area. Great for tiling: Surface area = 2d Not good for tiling: Surface area ≈ √d [Kindler O’Donnell Rao Wigderson ’08, Alon Klartag’09] There exists a tiling shape with unit volume and surface area O(√d ) !

Conclusion (Dis)Prove Unique Games Conjecture. Intermediate between P and NP-complete? Prove hardness results bypassing UGC. TSP, Steiner Tree, Scheduling Problems ? More techniques, connections, results …