Expander Flows, Graph Spectra and Graph Separators Umesh Vazirani U.C. Berkeley Based on joint work with Khandekar and Rao and with Orrechia, Schulman and Vishnoi

Graph Separators S T Sparsest Cut/Edge Expansion: c-Balanced Separator:

Applications Clustering Image segmentation VLSI layout Underlie many divide-and-conquer graph algorithms

Interesting Techniques Spectral methods. Connection to differential geometry, discrete isoperimetric inequalities. Linear/semidefinite programming Measure concentration Metric embeddings

Geometrical view Map vertices to points in some abstract space: - points well-spread - edges short

Geometrical view Map vertices to points in some abstract space: - points well-spread - edges short “Good bisection” of the space yields sparse cut in graph

Spectral Method Cut at random Minimize sum of “edge lengths”: Spread out vertices: [Cheeger’70] [Alon, Milman ’85][Jerrum, Sinclair’89]

Leighton-Rao ‘ Cut along ball of random radius Distances form a metric: satisfy triangle inequality. w ij + w jk >= w ik Minimize sum of “edge lengths”: Spread out vertices: O(log n) approximation: Approximate max-flow min-cut thm for multi-commodity flows.

ARV ‘04 Triangle inequality: Unit sphere in R d Unit L 2 2 embedding: No angles obtuse Minimize sum of “edge lengths” Spread out vertices Procedure to recover cut of size

ARV Procedure to recover cut Slice a randomly oriented “fat”-hyperplane of width Unit sphere in R d

ARV Procedure to recover cut Slice a randomly oriented “fat”-hyperplane of width Discard pairs of points (u,v): Arrange points according to distance from S Cut along ball of random radius r: Unit sphere in R d S

Metric Embeddings Finite Metric Space (X, d) x y R k with L 2 norm f(x) f(y) Distortion of f is min c: [Bourgain ’85] Every finite metric space can be embedded in L 2 with distortion O(log n). Longstanding open question: Better bound for L 1 ? [Enflo ’69] [Arora, Lee, Naor ’05] Any finite L 1 metric can be embedded in L 2 with distortion f()

Today’s Talk Leighton-Rao: multi-commodity flow O(n 2 ). Arora, Hazan, Kale: O * (n 2 ) ARV implementation based on expander-flow formalism Much faster in practice. [Khandekar-Rao-V] : O*(min{n 1.5, n/α(G)}) single commodity flow based algorithm. O(log 2 n) approx. ratio. [Arora, Kale]: matrix multiplicative weights algorithm based O(log n) approx [Orrechia, Schulman, V, Vishnoi] O(log n) approx using KRV style algorithm Multi-commodity flow: Single commodity flow:

Expander Flows Any algorithm for approximating sparse cuts must find a good cut, of expansion say β Must also certify no cut is much smaller. To give a k-approximation must certify that no cut has expansion less than β/k. Problem: there are exponentially many cuts. S T

Expander Flows G = H = For each edge of H, route one unit of flow through G

Expander Flows G = H = For each edge of H, route one unit of flow through G Must route Ώ(|S|) units of flow from S to T. Therefore |E S,T | = Ώ(|S|/c) expansion = Ώ(1/c) Ideally c = O(1/α(G)) S T max congestion = c expansion = Ώ(1/c)

Expander Flows max congestion = c. expansion = Ώ(1/c). ARV: max congestion = Leighton-Rao: H = complete graph. max cong = O(logn/α(G)) tight example: G = expander graph. Motivating idea for ARV: write LP to find best embedding of H in G + exponentially many constraints saying H expander eigenvalue bound gives efficient test for expansion! Therefore poly time using Ellipsoid algorithm. [Arora, Hazan, Kalle] O * (n 2 ) implementation of ARV

Know large number of vertices on each side of cut. A max-flow, min-cut computation should reveal sparse cut. But this is circular… KRV s t

H Φ Embed candidate expander H in G with small congestion. Test whether H is expander (if so done!) Else non-expanding cut in H gives a bipartition of G; route a flow in G across this bipartition. Decompose flow into flow paths and add the resulting matching to H. Outline of Algorithm

Cut-Matching Game H Φ Cut Player Find bad cut in H Goal: min # iterations until H is an expander Matching Player Pick a perfect matching across cut Goal: max # iterations until H is an expander Claim: There is a cut player strategy that succeeds in O(log 2 n) rounds.

Finding a cut: Spectral-like-method = +1 charge = –1 charge Mix the charges along the matchings { M 1, M 2, …, M t } Random assignment of charge V: Vertex set x y (x+y)/2 After t iterations, H = { M 1, M 2, …, M t }.

Finding a cut: Spectral-like-method Order the vertices according to the final charge present and cut in half. n/2 S S But how to formalize intuition?

Lift to R n Cannot directly formalize previous intuition Therefore lift random walk to R n – walk embedding of H. n-dimensional vector associated with each vertex In each step, replace vectors at endpoints of matched edge by their average vector. Potential function to measure progress of this process. Potential function small implies H expander. Relate lifted process to original random walk: each successive matching decreases potential function.

Walk Embedding H R n, Vertex i mapped to P i = (p i1, …, p in ) p ij = P[walk started at j ends at i] H t = { M 1, M 2, …, M t }. Small cut in graph shows up as clusters in walk embedding. (1/n, …, 1/n) P1P1 P3P3 P2P2 PnPn Potential: Claim: ψ(t) ≤1/4n 2 implies α(H t )≥ ½ Will show potential reduces by (1 – 1/log n) in each iteration. Ψ(0) = n-1

(1/n, …, 1/n) P1P1 P3P3 P2P2 PnPn Main Question: How to augment H t = { M 1, M 2, …, M t } by M t+1 so H closer to expander? Potential: If M t+1 matches vertex u to vertex v, then potential reduction in t+1-st step Since each of P u and P v replaced by So potential reduction = The Lifted Walk

Potential Reduction PvPv  =  v |P v  1/n| 2 Reduction in  =  |green| 2 1-d: reduction =  (  ) n-d  1-d: log n stretch Actual potential reduction =  /log n  Original random walk = projection of lifted walk on random vector

Running time Number of iterations = O(log 2 n) Each iteration = 1 max-flow + O*(n) work = O*(m 3/2 ) [Benczur-Karger’96] In O*(m) time, we can transform any graph G on n vertices into G’ on same vertices: –G’ has O(n log (n)/ε 2 ) edges –All cuts in G’ have size within (1 ± ε) of those in G Overall running time = O*(m + n 3/2 )

Improving to O(log n) approximation [Arora, Kale]: matrix multiplicative weights algorithm based combinatorial primal-dual schema for semidefinite progs [Orrechia, Schulman, V, Vishnoi]: simple KRV style algorithm Idea: To find M t+1 perform t steps of natural random walk (instead of round-robin walk) on H t = { M 1, M 2, …, M t }

Brief Sketch Instead of showing that H has constant edge expansion after O(log 2 n) steps, will show that the spectral gap of H is at least 1/log n, and therefore the conductance of H is at least 1/log n. Since degree of H is log 2 n, this means its edge edge expansion is at least log n.

Why natural walk? Suppose round robin walk on M 1, …, M k mixes perfectly on each of S, T. Now a single averaging step on M k+1 ensures perfect mixing on entire graph! S T M k+1

Matrix inequality: Question: Replace ½ self-loop with a ¾ self-loop in round-robin random walk! x y (3x/4+y/4) (3y/4 + x/4 Gives a way of relating round robin walk to time independent walk.

Conclusions and Open Questions Our algorithm is very similar to some heuristics. [Lang’04] similar to one iteration of our algorithm. METIS [Karypis-Kumar’99] –collapses random edges –finds a good partition in collapsed graph –induces it up to original graph, using local search Connections with these heuristics? Rigorous analysis?

When the Expansion is large … Could have used [Spielman-Teng’04] “nibble” algorithm instead of walk-embedding. But: AlgorithmOutput sparsityRunning Time Spectral  1/2 n2/2n2/2 Spielman-Teng  1/3 log 3 nn/  3 KRV  log 2 nmin {n 3/2,n/  } Conjecture: A single iteration of round-robin walk + max-flow should give a sparse cut.

[Khot, Vishnoi] Ώ(loglog n) integrality gap [Orrechia, Schulman, V, Vishnoi] Ώ(√logn) bound on cut-matching game. Is it possible to obtain a O(√log n) approximation algorithm using single commodity flows via the cut-matching game? Limits to these methods