Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Semidefinite Program (SDP): find X s.t. A 1 ² X · 0 A 2 ² X · 0  A m ² X · 0 X º 0 Tr(X) = 1 All eigenvalues are non-negative ´ 9 v 1, v 2, …, v n s.t. X ij = v i ¢ v j A ² B = Tr(AB) =  ij A ij B ij Density matrix

Max Cut [GW’95] Graph Coloring [KMS’98, ACC’06] (a Ç : b) Æ ( : a Ç c) Æ (a Ç b) Æ ( : c Ç b) Constraint Satisfaction [ACMM’05, R ’08, RS ‘09,..] SDP Balanced Partitioning [ARV’04] Control Theory

Algorithms for SDP Ellipsoid method [GLS’81]: O(n 8 ) iterations O(n 8 ) iterations Interior point methods [NN’90, A’95]: O(√n ¢ m 3 ) time O(√n ¢ m 3 ) time Lagrangian Relaxation [KL’95], [AHK’05]: Reduction to eigenvectors Reduction to eigenvectors poly(1/  ) dependence on , limits applicability poly(1/  ) dependence on , limits applicability A 1 ² X · 0  A n ² X · 0  i w i A i ² X · 0 Combinatorial, Primal-Dual algorithms (analogous to flow algorithms via LP) Combinatorial, Primal-Dual algorithms (analogous to flow algorithms via LP)

 Positive semidefiniteness hard to maintain  Rounding algorithms exploit geometric structure (e.g. negative-type metrics)  Matrix operations inefficient to implement

A general scheme that yields fast, combinatorial, primal-dual approximation algorithms for various combinatorial optimization problems using SDP relaxations

Our Results: Primal-Dual algorithmsProblem Previous best: O(√log n) apx Our algorithm: O(√log n) apx Our algorithm: O(log n) apx Undirected Sparsest Cut Õ(n 2 ) [AHK’04] Õ(n 2 ) Õ(m + n 1.33 ) Undirected Balanced Sep. Õ(n 2 ) [AHK’04] Õ(n 2 ) Õ(m + n 1.33 ) Directed Sparsest Cut Õ(n 9.5 ) Õ(m n 2 ) Õ(m 1.33 ) Directed Balanced Sep. Õ(n 9.5 ) Õ(m n 2 ) Õ(m 1.33 ) Min UnCut Õ(n 9.5 ) Õ(n 3 ) --- Min 2CNF Deletion Õ(n 9.5 ) Õ(nm n 3 ) --- Õ(m + n 1.33 ) Õ(m + n 1.33 ) [S’09] Unknown how to achieve using LP!

 Also yields Õ(m) time algorithm for approximating MAXCUT SDP  Gives first near-linear implementation of Goemans-Williamson approximation algorithm  Previous best: Õ(nm) [KL’95]

a 1 ¢ x · 0 a 2 ¢ x · 0  a m ¢ x · 0 x ¸ 0  i x i = 1 Multiplicative Weights Initialize: x 1 = (1/n, …, 1/n) Update: x t+1 = x t £ exp(-  a i t )/  t  t = normalization factor Violation Checker Find i t s.t. a i t ¢ x t >  xtxt aitait Thm: Finds  -feasible x in O(  2 log(n)/  2 ) iterations.  = max ij |a ij | (“width”) Convex combination of constraints allowed:  i y i a i ¢ x t > , where y i ¸ 0,  i y i = 1. Hence Primal-Dual. Analysis uses  t as potential function Coordinate-wise a 1 ¢ x ·  a 2 ¢ x ·   a m ¢ x ·  x ¸ 0  i x i = 1

A 1 ² X · 0 A 2 ² X · 0  A m ² X · 0 X º 0 Tr(X) = 1 Matrix MW Initialize: X 1 = (1/n) I Update: X t+1 = exp(-   t s=1 A i s )/  t  t = normalization factor Violation Checker Find i t s.t. A i t ² X t >  XtXt AitAit Thm: Finds  -feasible X in O(  2 log(n)/  2 ) iterations.  = max i k A i k (“width”) Convex combination of constraints allowed:  i y i A i ² X t > , where y i ¸ 0,  i y i = 1. Hence Primal-Dual. Analysis uses  t as potential function

The Matrix Exponential Matrix exponential: exp(A) = I + A + A 2 /2! + A 3 /3! + … Always PSD: exp(A) º 0 Always PSD: exp(A) º 0 Tricky beast: Tricky beast: exp(A+B) = exp(A) exp(B) Computation: Computation: O(n 3 ) time O(n 3 ) time Computing exp(A)v: Õ(m) time (arises as soln of ODE) Computing exp(A)v: Õ(m) time (arises as soln of ODE) Golden-Thompson inequality: Tr(exp(A+B)) · Tr(exp(A)exp(B)) Matrix MW Initialize: X 1 = (1/n) I Update: X t+1 = exp(-   t s=1 A i s )/  t  t = normalization factor

Input Quadratic Program SDP: Vector variables Reduction SDP Solver Rounding Algorithm SDP Opt: X X ij = v i ¢ v j v1v1 vnvn v2v2 Relaxation Primal-Dual SDP

Reduction Relaxation Rounding Algorithm v1v1 vnvn v2v2 Matrix MW X y 1,…, y m Primal-Dual SDP SDP: Vector variables Input X ij = v i ¢ v j Quadratic Program

 Cut (S, S’) is c-balanced if |S|, |S’| ¸ cn  Min c-Balanced Separator: c-balanced cut of min capacity  Numerous applications:  Divide-and-conquer algs  Markov chains  Geometric embeddings  Clustering  Layout problems …… G = (V, E) S S’

¼ k v i – v j k 2 = 0 if i, j on same side, = 1 otherwise SDP min  i,j 2 E ¼ k v i – v j k 2 8 i: k v i k 2 = 1  i, j ¼ k v i – v j k 2 ¸ c(1-c)n 2 8 ijk: k v i –v j k 2 + k v j –v k k 2 ¸ k v i –v k k 2 v i = -1v i = 1 G = (V, E) S S’

SDP (C1)  i,j 2 E ¼ k v i – v j k 2 <  (C2) 8 i: k v i k 2 = 1 (C3)  i, j ¼ k v i – v j k 2 ¸ c(1-c)n 2 (C4) 8 ijk: k v i –v j k 2 + k v j –v k k 2 ¸ k v i –v k k 2 To find violated constraints, given X (thus, v i ):  Check (C2) and (C3)  To check (C1) and (C4): find multicommodity flow s.t. 1. Total flow from any node is at most d = Õ(  /n) 2. Total flow on any edge is at most 1 3.  i,j f ij k v i – v j k 2 ¸ 4  f ij = total flow between i, j Bounds width  Binary search parameter Either (C1) or (C4) violated

X ij = v i ¢ v j v1v1 vnvn v2v2 d d d d d d d To find flow s.t.  ij f ij k v i – v j k  ¸ 4  Thm: This yields either: 1.a flow s.t.  ij f ij k v i – v j k 2 ¸ 4  2.a cut of value O(log(n)  ) k v i k 2 = 1 ¼  ij ¼ k v i – v j k 2 ¸ c(1-c)n 2 Max flow

X ij = v i ¢ v j v1v1 vnvn v2v2 Thm: This yields either: 1.a flow s.t.  ij f ij k v i – v j k 2 ¸ 4  2.a cut of value O(√log(n)  ) d d d d d d d d Multicommodity flow To find flow s.t.  ij f ij k v i – v j k  ¸ 4  k v i k 2 = 1 ¼  ij ¼ k v i – v j k 2 ¸ c(1-c)n 2

Derandomization Deterministic Expander Graphs Constructions [K’07, WX ’07] Deterministic Approximation via Covering SDPs [K ’07, WX ’07] Quantum Computing QIP = PSPACE [JJUW ’10] and related results [JUW ’09, JW ’09] Learnability of Quantum States [A ’06, K ’07] Machine Learning Online Variance Minimization [WK ’06] Online PCA [WK ’06] Other SDP based Approximation Algorithmss Near Linear-Time Approximation Algorithms for 2-CSPS [S ’10] Near-Linear Time Balanced Graph Partitioning [OV ’11]

 Make SDP solver efficient in practice  Sampling, rescaling techniques might be useful  ``Combinatorialize’’ interior-point methods?  Near linear-time algs for approx max flow?