Geometry and Expansion: A survey of some results Sanjeev Arora Princeton ( touches upon: S. A., Satish Rao, Umesh Vazirani, STOC04; S. A., Elad Hazan, and Satyen Kale, FOCS04; S. A., James Lee, and Assaf Naor, STOC05 & JAMS08 S.A., S. Kale STOC papers that are not mine)

Outline: Graph partitioning problems: intro and history New approximation via expander flows. New approximation algorithm via semidefinite programming (+ analysis using Structure Theorem) [A., Rao, Vazirani] Outline of proof of S. T. Uses of S. T. in geometric embeddings Open problems

Sparsest Cut / Edge Expansion S S G = (V, E) c- balanced separator Both NP-hard G) = min S µ V | E(S, S c )| |S| |S| < |V|/2 c (G) = min S µ V | E(S, S c )| |S| c |V| < |S| < |V|/2

Why these problems are important Analysis of random walks, PRAM simulation, packet routing, clustering, VLSI layout etc. Underlie many divide-and-conquer graph algorithms (surveyed by Shmoys95) Discrete analog of isoperimetry; useful in Riemannian geometry (via 2 nd eigenvalue of Laplacian ( Cheeger70 ) Graph-theoretic parameters of inherent interest (cf. Lipton-Tarjan planar separator theorem)

Previous approximation algorithms 1)Eigenvalue approaches ( Cheeger70, Alon85, Alon-Milman85 ) Only yield factor n approximation. 2c(G) ¸ (G) ¸ c(G) 2 /2 2) O(log n) - approximation via LP (multicommodity flows ) ( Leighton-Rao88 ) Approximate max-flow mincut theorems Region-growing argument ( Linial, London, Rabinovich94, AR94 ) 3) Embeddings of finite metric spaces into l 1 Geometric approach; more general result (but still O(log n) approximation)

New results of [ARV04] 1.O( ) -approximation to sparsest cut and conductance 2.O( )-pseudoapproximation to c-balanced separator (algorithm outputs a c-balanced separator, c < c) 3.Existence of expander flows in every graph (approximate certificates of expansion) log n Disparate approaches from previous slide get unified Subsequent work: [AHK05],[AK07],[S09]: O(m + n 1.5 ) time!

The three main characters ExpansionIsoperimetry (continuous analog of expansion) Geometry (and geometric embeddings of finite metric spaces)

Identifying sparse cuts via traffic flows

Approach 1: traffic congestion identifies sparse cuts [SM87]: Stress a network by passing traffic flow through it. Look at congested edges to identify sparse cuts [LR88] O(log n) approximation to sparsest cut. Route 1 unit of traffic between every pair of nodes [ARV04] Traffic flow is like embedding a weighted graph. w ij = amount of traffic from i to j Solve a math program to find the right flow pattern ([AHK05] Do it in O(n 2 ) time)

Expander traffic flows [ARV04] G = (V, E) S S A D-regular flow graph s.t. 8 S w(S, S c ) = ( D |S|) Weighted Graph w satisfies (*) iff L (w) = (1) [Cheeger] (*) Our Thm: If G has expansion, then a D-regular expander flow exists in it where D= (certifies expansion = (D) )

Formal statement : 9 0 >0 s.t. foll. LP is feasible for D = (G) log n f p ¸ 0 8 paths p in G 8i j p 2 P ij f p = D (degree) P ij = paths whose endpoints are i, j 8S µ V i 2 S j 2 S c p 2 P ij f p ¸ 0 D |S| (demand graph is an expander) 8e 2 E p 3 e f p · 1 (capacity) WHY IS THIS FEASIBLE???

Feasibility Criterion for LP on prev. slide (via Farkass Lemma) Existence of such i, j proved in [ARV04]. When fail to find such i, j, we find a cut of small expansion

Overall approximation algorithm via flows Try to solve above LP to find D-regular expander flow If succeed, have verified that expansion is D/10. If fail, then use [ARV04] ideas to find a cut of capacity Note: Before finding this cut already had D/2-regular flow

Next: The SDP-based approach to Graph partitioning (ARV04)

c-balanced separator c (G) = min S µ V | E(S, S c )| |S| c |V| < |S| < |V|/2 S S Assign {+1, -1} to v 1, v 2, …, v n to minimize (i, j) 2 E |v i –v j | 2 /4 Subject to i < j |v i –v j | 2 /4 ¸ c(1-c)n 2 +1 |v i –v j | 2 /4 =1 Semidefinite relaxation for Find unit vectors in < n |v i –v j | 2 + |v j –v k | 2 ¸ |v i –v k | 2 8 i, j, k Triangle inequality cut semimetric |v i –v j | 2 =0

Unit l 2 2 space Unit vectors v 1, v 2,… v n 2 < d |v i –v j | 2 + |v j –v k | 2 ¸ |v i –v k | 2 8 i, j, k ViVi VkVk VjVj non obtuse ! Example: Hypercube {-1, 1} k |u – v| 2 = i |u i – v i | 2 = 2 i |u i – v i | = 2 |u – v| 1 In fact, l 2 and l 1 are subcases of l 2 2

Structure Theorem for l 2 2 spaces [ARV04] Subsets S and T are -separated if for every v i 2 S, v j 2 T |v i –v j | 2 ¸ ¸ Thm: If i< j |v i –v j | 2 = (n 2 ) then 9 S, T of size (n) that are -separated for = ( 1 ) <d<d log n

Main thm ) O( )-approximation log n v 1, v 2,…, v n 2 < d is optimum SDP soln; SDP opt = (i, j) 2 E |v i –v j | 2 S, T : –separated sets of size (n) Do BFS from S until you hit T. Take the level of the BFS tree with the fewest edges and output the cut (R, R c ) defined by this level (i, j) 2 E |v i –v j | 2 ¸ |E(R, R c )| £ ) |E(R, R c )| · SDP opt / · O( SDP opt ) log n S d(S, i) i j d(S, j)

Other new -approximation algorithms MIN-2-CNF deletion and several graph deletion problems. [Agarwal, Charikar, Makarychev, Makarychev04]. Weighted version of S.T. MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao04] General SPARSEST CUT [A., Lee, Naor 04] Min-ratio VERTEX SEPARATORS and Balanced VERTEX SEPARATORS [ Feige, Hajiaghayi, Lee, 04] log n All use the Structure Theorem (+ other ideas)

Outline: Graph partitioning problems: intro and history New approximation algorithm via semidefinite programming (+ analysis using Structure Theorem) [A., Rao, Vazirani] Outline of proof of S. T. Uses of S. T. in geometric embeddings Introduction to expander flows and O(n 2 ) time algorithms Open problems S T (Algorithm to produce -separated sets S, T, of size (n) )

Algorithm to produce two –separated sets <d<d u SuSu TuTu 0.01 d Easy: S u and T u likely to have size (n) If S u, T u still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted d Stretched pair: v i, v j such that |v i –v j | 2 · and | h v i –v j, u i | ¸ 0.01 Obs: Deleted pairs are stretched and they form a matching. Delete any v i 2 S u, v j 2 T u s.t. |v i –v j | 2 <. (till no such pair remains)

Naïve analysis of random projection fails <d<d v u ?? 1 d 1 d e -t 2 /2 Stretched pair: |v i –v j | 2 | > 0.01 d standard deviations E[# of stretched pairs] = n 2 exp(- ) À n = O( 1 )

ViVi Ball (v i, ) u VjVj 0.01 d Proof by contradiction: Suppose matching of (n) size exists with probability (1)… ….stretched pairs are almost everywhere you look! Idea: Put stretched pairs together; derive very improbable event

Walks in unit l 2 2 space Unit vectors v 1, v 2,… v n 2 < d |v i –v j | 2 + |v j –v k | 2 ¸ |v i –v k | 2 8 i, j, k ViVi VkVk VjVj Angles are non obtuse Taking r steps of length s only takes you squared distance rs 2 (i.e. distance r s) ss ss

Proof by contradiction (contd.) s s s s r steps of length s ) squared distance rs 2 (distance r s) Stretched pair: |v i –v j | 2 ¸ 0.01 Claim: 9walk on stretched edges u d 0.01 d d …. ¸ r 0.01 d |v final –v 0 | · r Projection = r £ standard deviation VERY UNLIKELY IF r large enough ) Walk impossible (CONTRADICTION) Why walk is possible: delicate argument; measure concentration

Outline: Graph partitioning problems: intro and history New approximation algorithm via semidefinite programming (+ analysis using Structure Theorem) [A., Rao, Vazirani] Outline of proof of S. T. Geometric embeddings of metric spaces Open problems

Finite metric space (X, d) x y d(x,y) < k (with l 2 norm) f distortion of f is minimum C>1 such that d( x, y) · |f(x ) – f( y)| 2 · C d( x, y) 8 x, y Thm (Bourgain85): For every n-point metric space, a map exists with distortion O(log n) [LLR94]: Efficient algorithm to find the map; Proof that O(log n) cannot be improved in general Qs: Improve O(log n) for X = l 2 2 (say) or l 1 ? f(x) f(y)

Embeddings and Cuts (LLR94, AR94) Recall: Cut semi-metric 1 0 Fact: Metric (X, d) embeds isometrically in l 1 iff it can be written as a positive combination of cut semimetrics Embedding l 2 2 into l 1 gives a way to produce cuts from SDP solution

Status report of this area l 1 into l 2 log 0.5 n [Enflo69] l 2 2 into l [Zatloukal04] Superconstant [Khot, Vishnoi04] (logn) 0.01 [Cheeger,Kleine r, 08] l 2 2 into l 2 log 0.5 n [Enflo69] Best lowerbound Best upperbound Exactly the integrality gap of SDP for general SPARSEST CUT [LLR94, AR94] log n [Bourgain85] log 0.75 n [Chawla,Gupta,Racke 04] log 0.5 n log log n [A., Lee, Naor04] Uses fourier techniques developed for PCPs! Disproves Goemans-Linial conjecture Uses new metric differentiation techniques

Upperbounds: Frechets recipe to embed metric space (X, d) into R k Pick k suitable subsets A 1, A 2, …, A k of X Map x 2 X to (d(x, A 1 ), d(x, A 2 ), …, d(x, A k )) AiAi x In recent embeddings, A i s are chosen using S.T.and Measured descent idea of [Krauthgamer, Lee, Naor, and Mendel04] Note: d(x, A 1 ) – d(y, A 1 ) · d(x, y)

Embedding lowerbounds (Khot-Vishnoi05) Explicit unit- l 2 2 space (X, d) that requires distortion log log log n into l 1 Main observation: Need good handle on cut structure of X Use hypercube as building block ! Cut´ Boolean Function Number of cut edges = average sensitivity (Fourier analysis a la KKL, Friedgut, Hastad, Bourgain etc. ) isoperimetric theorems)

OPEN PROBLEMS Better approximation factor than O( )? (log log n lowerbound assuming UGC ) Better distortion bound for embedding l 2 2 into l 1 ?.) ( upperbound v/s lowerbound Combinatorial approximation algorithms for other problems ? (similar to one for SPARSEST CUT from [A., Hazan, Kale] ) Other applications of expander flows? (Useful in some geometric results [Naor, Rabani, Sinclair04]) Ways to use spectral ideas a la [ABS10] for SPARSEST CUT?

Example of expander flow n-cycle Take any 3-regular expander on n nodes Put a weight of 1/3n on each edge Embed this into the n-cycle Routing of edges does not exceed any capacity ) expansion = (1/n)

Other extensions of flow-based techniques Generalization to problems other than sparsest cut [A., Kale07] Primal-dual approach to SDP. Very fast algorithms for O(log n) approximation: O(n m) time (Faster than [LR88] type algorithms!) Very simple algorithms; use only maxflow and eigenvalue computations [KRV06]

Looking forward to more progress… Thanks !

New Result (A., Hazan, Kale;FOCS04) O(n 2 ) time algorithm that given any graph G finds for some D >0 a D-regular expander flow a cut of expansion O( D ) log n Ingredients: Approximate eigenvalue computations; Approximate flow computations (Garg-Konemann; Fleischer) Random sampling (Benczur-Karger + some more) Idea: Define a zero-sum game whose optimum solution is an expander flow; solve approximately using Freund-Schapire approximate solver. ) D) · (G) ·O(D ) log n

Expander flows: LP view LP feasible ) ¸ (D) G G · D · 1 Thm [ARV]: 9 0 s.t. the LP is feasible with D = /log n Thm [ARV]: 9 0 s.t. the LP is feasible with D = /log n

Open problems (circa April04) Better running time/combinatorial algorithm? Improve approximation ratio to O(1); better rounding?? (our conjectures may be useful…) Extend result to other expansion-like problems (multicut, general sparsest cut; MIN-2CNF deletion) Resolve conjecture about embeddability of l 2 2 into l 1 ; of l 1 into l 2 Any applications of expander flows? O(n 2 ) time; [A., Hazan, Kale] log 3/4 n distortion; [Chawla,Gupta, Racke] Integrality gap is (log n) [Charikar] Yes [Naor,Sinclair,Rabani] Better embeddings of l p into l q [Lee]

Various new results O(n 2 ) time combinatorial algorithm for sparsest cut (does not use semidefinite programs) [A., Hazan, Kale04] New results about embeddings: (i) l p into l q [J. Lee04] (ii) l 2 2 and l 1 into l 2 [CGR04] (approx for general sparsest cut) Clearer explanation of expander flows and their connection to embeddings [NRS04]

A concrete conjecture (prove or refute) G = (V, E); = (G) For every distribution on n/3 –balanced cuts {z S } (i.e., S z S =1) there exist (n) disjoint pairs ( i 1, j 1 ), ( i 2, j 2 ), ….. such that for each k, distance between i k, j k in G is O(1/ ) i k, j k are across (1) fraction of cuts in {z S } ( i.e., S: i 2 S, j 2 S c z S = (1) ) Conjecture ) existence of d-regular expander flows for d =

log n

Example of l 2 2 space: hypercube {-1, 1} k |u – v| 2 = i |u i – v i | 2 = 2 i |u i – v i | = 2 |u – v| 1 In fact, every l 1 space is also l 2 2 Conjecture (Goemans, Linial): Every l 2 2 space is l 1 up to distortion O(1)

LP Relaxations for c-balanced separator Motivation: Every cut (S, S c ) defines a (semi) metric X ij 2 {0,1} i< j X ij ¸ c(1-c)n 2 X ij + X j k ¸ X ik 0 · X ij · 1 Semidefinite There exist unit vectors v 1, v 2, …, v n 2 < n such that X ij = |v i - v j | 2 /4 Min (i, j) 2 E X ij

Semidefinite relaxation (contd) Min (i, j) 2 E |v i –v j | 2 /4 |v i | 2 = 1 |v i –v j | 2 + |v j –v k | 2 ¸ |v i –v k | 2 8 i, j, k i < j |v i –v j | 2 ¸ 4c(1-c)n 2 Unit l 2 2 space Many other NP-hard problems have similar relaxations.

Algorithm to produce two –separated sets <d<d u SuSu TuTu 0.01 d Check if S u and T u have size (n) If any v i 2 S u and v j 2 T u satisfy |v i –v j | 2 · repeat until no such v i, v j remain delete them and If S u, T u still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted d Stretched pair: v i, v j such that |v i –v j | 2 · and | h v i –v j, u i | ¸ 0.01 Obs: Deleted pairs are stretched and they form a matching.

Next min: Proof-sketch of Structure Thm ( algorithm to produce -separated S, T of size (n); = 1/ ) S T

Matching is of size o(n) whp : naive argument fails d Stretched pair: v i, v j such that |v i –v j | 2 · and | h v i –v j, u i | ¸ 0.01 O( 1 ) £ standard deviation ) Pr U [ v i, v j get stretched] = exp( - 1 ) = exp( - ) log n E[# of stretched pairs] = O( n 2 ) £ exp(- )log n

Generating a contradiction: the walk on stretched pairs u ViVi VjVj 0.01 d d r steps 0.01 d r |v final - v i | < r | | ¸ 0.01r d = O( r ) x standard dev. v fina l Contradiction if r is large enough!

Measure concentration (P. Levy, Gromov etc.) <d<d A A : measurable set with (A) ¸ 1/4 A : points with distance · to A A A ) ¸ 1 – exp(- 2 d) Reason: Isoperimetric inequality for spheres

Expander flows (approximate certificates of expansion)