Subexponential Algorithms for Unique Games and Related Problems Barriers II Workshop, Princeton, August 2010 David Steurer MSR New England Sanjeev Arora Princeton University & CCI Boaz Barak Princeton & MSR New England U

Introduction Small-Set Expansion Unique Games

U NIQUE G AMES Input:list of constraints of form x i – x j = c ij mod k Goal:satisfy as many constraints as possible Input:U NIQUE G AMES instance with k << log n (say) Goal: Distinguish two cases YES: more than 1 - ² of constraints satisfiable NO: less than ² of constraints satisfiable Khot’s Unique Games Conjecture (UGC) For every ² > 0, the following is NP-hard: UG( ² )

Implications of UGC For many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations) Examples: V ERTEX C OVER [Khot Regev’03], M AX C UT [KhotKindlerMosselO’Donnell’04, MosselO’DonnellOleszkiewicz’05], any M AX C SP [Raghavendra’08], …

Implications of UGC For many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations) Examples: V ERTEX C OVER [Khot Regev’03], M AX C UT [KhotKindlerMosselO’Donnell’04, MosselO’DonnellOleszkiewicz’05], any M AX C SP [Raghavendra’08], …

Unique Games Barrier Example: ( ® GW + ² )-approximation for M AX C UT at least as hard as UG( ² ’) U NIQUE G AMES is common barrier for improving current algorithms of many basic problems Reductions show that beating current algorithms for these problems is harder than U NIQUE G AMES ® GW = 0.878… Goemans–Williamson bound for Max Cut Implications of UGC For many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations) Examples: V ERTEX C OVER [KR’03], M AX C UT [KKMO,’04 MOO’05], any M AX C SP [Raghavendra’08], …

Consequences for UGC (*) Analog of UGC with subconstant ² (say ² = 1/log log n) is false (contrast: subconstant hardness for L ABEL C OVER [Moshkovitz-Raz’08]) Subexponential Algorithm for Unique Games In particular: UG( ² 3 ) has exp(n ² )-time algorithm Given a U NIQUE G AMES instance with alphabet size k such that 1 - ² of constraints are satisfiable, can satisfy 1 - √ ² / ¯ 3 of constraints in time exp(k n ¯ ) NP-hardness reduction for UG( ² ) requires blow-up n poly(1/ ² )  rules out certain classes of reductions for proving UGC (*) assuming 3 S AT does not have subexponential algorithms

poly(n)exp(n) Concrete Complexity Landscape 2-S AT M AX 3-S AT (7/8) M AX C UT ( ® GW ) * assuming Exponential Time Hypothesis [Impagliazzo-Paturi-Zane’01] ( 3-S AT has no exp(o(n)) algorithm ) 3-S AT (*) Factoring Graph Isomorphism exp(n 1/2 )exp(n 1/3 )exp(n ² ) UG( ² 3 ) M AX 3-S AT (7/8+ ² ) L ABEL C OVER ( ² ) [Moshkovitz-Raz’08 + Håstad’97] If UGC true, U NIQUE G AMES is first CSP with intermediate complexity M AX C UT ( ® GW + ² )? UGC-based hardness does not rule out subexponential algorithms,  Possibility: exp(n ² )-time algorithm for M AX C UT ( ® GW + ² ) U NIQUE G AMES much easier than L ABEL C OVER Implications of U NIQUE G AMES algorithm (*)

Introduction Small-Set Expansion Unique Games

d-regular graph G d vertex set S Graph Expansion expansion(S) = # edges leaving S d |S| volume(S ) = |S| |V|

d-regular graph G d vertex set S Graph Expansion expansion(S) = # edges leaving S d |S| volume(S ) = |S| |V|

S expansion(S) = # edges leaving S d |S| Graph Expansion volume(S ) = |S| |V| S MALL -S ET E XPANSION Goal:find S with volume(S) < ± so as to minimize expansion(S) Input:d-regular graph G, parameter ± > 0 Important concept in many contexts: derandomization, network routing, coding theory, Markov chains, differential geometry, group theory close connection to U NIQUE G AMES [Raghavendra-S.’10]

S expansion(S) = # edges leaving S d |S| Graph Expansion volume(S ) = |S| |V| Important concept in many contexts: derandomization, network routing, coding theory, Markov chains, differential geometry, group theory Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± )

Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± )

1.few large eigenvalues 2.many large eigenvalues Distinguish two cases: large eigenvalues ¸ i > 1 - ´ Eigenvalues of the random walk matrix G: 1 = ¸ 1 ¸ … ¸ ¸ m ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 (pseudorandom graph) (structured graph?) Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± ) ´ À ² (best: ´ = 100 ², simpler: ´ = ² 0.75 )

1 - ´ Eigenvalues of the random walk matrix G: 1 = ¸ 1 ¸ … ¸ ¸ m ¸ Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± ) ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 large eigenvalues ¸ i > 1 - ´

Case 1: Few large eigenvalues: (inspired by [Kolla–Tulsiani ’07] and [Kolla’10]) 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Expander Mixing Lemma: indicator vector of S lives almost completely in span of top m eigenvectors Eigenvalues of the random walk matrix G:  can find set S’ close to S in time exp(m) Enumerate this space in time exp(m) Case 2: Many large eigenvalues (m > n ¯ / ± )  can find small non-expanding set S’ around that vertex Will show: 9 vertex whose neighborhoods grow very slowly Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± )

Case 1: Few large eigenvalues: 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G:

Case 1: Few large eigenvalues Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10]) Eigenvalues of the random walk matrix G: Suffices to show: indicator vector of S is ² / ´ -close to U |S ¢ S’| < ² / ´ |S [ S’| For every set S with expansion(S) < ², can find S’ that is ² / ´ -close to S in time exp(m) Algorithm: Let U be span of top-m eigenvectors For every vector u in ² -net of unit ball of U, output all level sets of u U 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1

Case 1: Few large eigenvalues Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10]) Eigenvalues of the random walk matrix G: Suffices to show: indicator vector of S is ² / ´ -close to U (generalization of “easy direction” of Cheeger’s inequality) h x, G x i > 1 - ² because expansion(S) < ² x = normalized indicator vector of S 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 For every set S with expansion(S) < ², can find S’ that is ² / ´ -close to S in time exp(m)

Case 1: Few large eigenvalues Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10]) Eigenvalues of the random walk matrix G: (generalization of “easy direction” of Cheeger’s inequality) h x, G x i = h u, G u i + h w, G w i < h u,u i + (1 - ´ ) h w,w i = 1 - ´ h w,w i Suppose x = u + w for u in U and w orthogonal to U h x, G x i > 1 - ² because expansion(S) < ² x = normalized indicator vector of S 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Suffices to show: indicator vector of S is ² / ´ -close to U For every set S with expansion(S) < ², can find S’ that is ² / ´ -close to S in time exp(m)

Eigenvalues of the random walk matrix G: 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1

Case 2: Many large eigenvalues (m > n ¯ / ± ) Eigenvalues of the random walk matrix G: 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1

If m > 1, then 9 S with volume(S) < ½ and expansion(S) < √ ´ and we can find S in poly(n)-time. Compare: “hard direction” of Cheeger’s inequality Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion “higher eigenvalue Cheeger bound”

Heuristic: balls tend to be the least expanding sets in graphs How can we find small non-expanding sets? Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion

Suffices to show: 9 vertex i such that volume( Ball(i, t) ) < ± for t = ( ¯ / ´ ) log(n) Volume growth vs. small-set expansion  volume growth < 1+( ´ / ¯ ) in intermediate step  expansion( Ball(i,s) ) < ( ´ / ¯ ) for some s < t Suppose volume( Ball(i, t) ) < ± for t = ( ¯ / ´ ) log(n) Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion

Volume growth vs. small-set expansion How can we relate eigenvalues and volume growth? Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion Suffices to show: 9 vertex i such that volume( Ball(i, t) ) < ± for t = ( ¯ / ´ ) log(n)

Suffices to show: 9 vertex i such that volume( Ball(i, t) ) < ± for t = ( ¯ / ´ ) log(n) Heuristic: collision probability ¼ 1/|support| Collision probability decay || G t e i || 2 > 1/( ± n) collision probability of t-step random walk from i Proof follows from local variant of Cheeger’s inequality (e.g. Dimitriou–Impagliazzo’98) Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion Volume growth vs. small-set expansion

Number of large eigenvalues vs. collision probability decay Collision probability decay vs. small-set expansion Suffices to show: 9 vertex i such that || G t e i || 2 > 1/( ± n) for t = ( ¯ / ´ ) log(n) =  i h e i, G 2t e i i = Trace(G 2t )> m ¢ (1 - ´ ) 2t > 1/ ±  i ||G t e i || 2 Case 2: Many large eigenvalues (m > n ¯ / ± ) 1 = ¸ 1 ¸ …………………… ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G: If m > n ¯ / ±, then 9 S with volume(S) < ± and expansion(S) < √ ´ / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion

Case 1: Few large eigenvalues: 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ´ ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 Eigenvalues of the random walk matrix G:  can find set S’ ² / ´ -close to S in time exp(m) Subspace enumeration: discretize span of top m eigenvectors Case 2: Many large eigenvalues (m > n ¯ / ± )  can find small non-expanding set S’ around that vertex Trace bound: 9 vertex where local random walk mixes slowly U Subexponential Algorithm for S MALL -S ET E XPANSION If there exists S with volume(S) < ± and expansion(S) < ², we can find S’ with volume(S’) < 2 ± and expansion(S’) < √ ² / ¯ in time exp(n ¯ / ± )

Introduction Small-Set Expansion Unique Games

U NIQUE G AMES Input:list of constraints of form x i – x j = c ij mod k Goal:satisfy as many constraints as possible Constraint Graph G variable  vertex constraint  edge i j x i – x j = c ij mod k Subexponential Algorithm for Unique Games In particular: UG( ² 3 ) has exp(n ² )-time algorithm Given a U NIQUE G AMES instance with alphabet size k such that 1 - ² of constraints are satisfiable, can satisfy 1 - √ ² / ¯ 3 of constraints in time exp(k n ¯ )

can solve UG( ² 3 ) in time exp(m) Few large eigenvalues + strong L 1 bounds on top-m eigenvectors large eigenvalues Eigenvalues of the random walk matrix of constraint graph G: 1 = ¸ 1 ¸ … ¸ ¸ m > 1 - ² ¸ ¸ m+1 ¸ … ¸ ¸ n ¸ -1 [Arora–Khot–Kolla– S.–Tulsiani–Vishnoi’08] Expanding constraint graph (m=1) can solve UG( ² ) in time poly(n) [Kolla’10] U NIQUE G AMES on pseudorandom instances is easy Algorithms for special instances of Unique Games

Strategy partition general instance into pseudorandom instances by changing only a small fraction of edges [Trevisan’05] Algorithms for general instances of Unique Games general instance pseudorandom instances decomposition few constraints between parts [Arora–Impagliazzo– –Matthews–S.’10] here: Leighton–Rao decomposition tree, constant depth, using higher eigenvalue Cheeger bound here: at most n ¯ eigenvalues > 1- ² here: at most √ ² / ¯ 3 fraction

Subexponential Algorithm for Unique Games few large eigenvalues: at most n ¯ eigenvalues >1- ² Few-large-eigenvalues decomposition Every regular graph G can be partitioned into components with few large eigenvalues by removing √ ² / ¯ 3 fraction of edges Unique Games with few large eigenvalues If every component of G has few large eigenvalues, can solve UG( ² 2 ) in time exp(n ¯ )

few large eigenvalues: at most n ¯ eigenvalues >1- ² Unique Games with few large eigenvalues If every component of G has few large eigenvalues, can solve UG( ² 2 ) in time exp(n ¯ )

few large eigenvalues: at most n ¯ eigenvalues >1- ² Unique Games with few large eigenvalues If every component of G has few large eigenvalues, can solve UG( ² 2 ) in time exp(n ¯ ) label-extended graph G* i j x i – x j = c ij mod k constraint graph G cloud j cloud i (i,a) » (j,b) if a-b = c ij mod k assignment satisfying 1- ² 2 of constraints set with volume = 1/k and expansion < ² 2 Subspace enumeration: can enumerate all nonexpanding sets if G* has few large eigenvalues When does G* have here few large eigenvalues?

Unique Games with few large eigenvalues label-extended graph G* i j x i – x j = c ij mod k constraint graph G cloud j cloud i (i,a) » (j,b) if a-b = c ij mod k When does G* have here few large eigenvalues? if G is an expander [Kolla–Tulsiani’07] if G has few large eigenvalues and eigenvectors are well-spread [Kolla’10] if G has few’ large’ eigenvalues (this work, by comparing collision probabilities) If every component of G has few large eigenvalues, can solve UG( ² 2 ) in time exp(n ¯ ) few large eigenvalues: at most n ¯ eigenvalues >1- ²

Recall: (with slightly different parameters) Analysis: For every subdivision S, charge its expansion to vertices in S K If component K has many large eigenvalues, then 9 S in K with |S| < |K| 1- ¯ and expansion(S) < √ ² / ¯ and we can find S in poly(n)-time Number of large eigenvalues vs. small-set expansion Algorithm: Subdivide components until all components have few large eigenvalues  no vertex is charged more than log(1/ ¯ )/ ¯ times If vertex is charged t times, its component has size < n (1- ¯ ) t few large eigenvalues: at most n ¯ eigenvalues larger 1- ² Few-large-eigenvalues decomposition Every regular graph G can be partitioned into components with few large eigenvalues by removing √ ² / ¯ 3 fraction of edges

Open Questions Example: C-approximation for S PARSEST C UT in time exp(n 1/C ) How many large eigenvalues can a small-set expander have? Is Boolean noise graph the worst case? (polylog(n) large eigenvalues) Thank you!Questions? More Subexponential Algorithms Similar approximation for M ULTI C UT and d-T O P ROVER G AMES Better approximations for M AX C UT and V ERTEX C OVER on small-set expanders What else can be done in subexponential time? Towards better-than-subexponential algorithms for U NIQUE G AMES Better approximations for M AX C UT or V ERTEX C OVER on general instances?