An algorithmic proof of the Lovasz Local Lemma via resampling oracles Jan Vondrak IBM Almaden TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A Nick Harvey University of British Columbia

Unlikely, but possible Alex Tsipras replaces Mario Draghi as head of ECB Random assignment to a k -SAT formula is satisfiable? O(congestion + dilation) packet routing O( 1 )-approx for Santa Claus (max-min allocation) problem Other uses in rounding linear program relaxations

Lovasz Local Lemma: [Erdos-Lovasz ‘75] Let E 1, E 2,…, E n be events in a probability space. Suppose E i is jointly independent of all but d events P [ E i ] · 1 / ed for all i Then P [ Å i E i ] > 0. Simultaneously avoiding many events “Needle in a haystack” It is possible to avoid all events E 1, E 2,…, E n. But the probability of this may be exponentially small.

Lovasz Local Lemma: [Erdos-Lovasz 1975] Let E 1, E 2,…, E n be events in a probability space. Suppose E i is jointly independent of all but d events P [ E i ] · 1 / ed for all i Then P [ Å i E i ] > 0. Simultaneously avoiding many events Beyond what nature intended: Finding a very rare object Methods are simple, but consequences are dramatic.

Example: 2-coloring hypergraphs Let E i be event i th set is all-red or all-blue. Note P [ E i ]= 2 1-k. E i is independent of all but 2 k-1 / e = d sets. Since P [ E i ] · 1 / ed, LLL implies P [ Å i E i ]> 0. So, there is a coloring where no set is all-red or all-blue. Given: System of sets of size k, each intersecting · 2 k-1 / e sets. Goal: Color vertices red/blue so that each set has both colors. Random approach: color each vertex independently red/blue.

Under the original distribution P, it is unlikely but possible to avoid all events E 1, E 2,…, E n. Can we find another distribution (i.e. a randomized algorithm) in which it is likely to avoid E 1, E 2,…, E n ? Limited scenarios, weaker parameters: [Beck ’91], [Alon ’91], [Molloy-Reed ’98], [Czumaj-Scheideler’00], [Srinivasan ’08] Algorithmic LLL

Under the original distribution P, it is unlikely but possible to avoid all events E 1, E 2,…, E n. Can we find another distribution (i.e. a randomized algorithm) in which it is likely to avoid E 1, E 2,…, E n ? Algorithmic LLL Moser’s Algorithm: [Moser ‘08] While some set is all-red or all-blue Pick an arbitrary one and re-color it randomly. Theorem: [Moser-Tardos ‘08] If each set intersects · 2 k-1 / e sets, the algorithm terminates in expected polynomial time.

The Moser-Tardos Framework (probability spaces with a product measure) Independent random variables Y 1,…, Y m “Bad” events E 1, E 2,…, E n E i depends on variables var( E i ) A dependency graph G : i » j if var( E i ) Å var( E j )  ;. 9 x 1,..., x n 2 ( 0, 1 ) such that P [ E i ] · x i ¢  j 2 ¡ ( i ) ( 1 - x j ) 8 i. Moser-Tardos Algorithm: While some event E i occurs, resample all variables in var( E i ). Theorem: [Moser-Tardos ‘08] The algorithm finds ! 2 Å i E i after resampling operations, in expectation. Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8 Y9Y9 Y 10 Y 11 Y 12 Y 13 Y 10 Y 11 Y 12

Algorithmic Improvements Many extensions of Moser-Tardos: – deterministic LLL algorithm [Chandrasekaran et al. ’10] – exponentially many events [Haupler-Saha-Srinivasan ’10] – better conditions on probabilities [Kolipaka-Szegedy’11], [Pegden ’13] Require independent RVs Y 1,…, Y m with product measure We consider events determined by subsets of underlying mutually independent random variables... this appears to be necessary in order to get any algorithmic access to the problem [Moser-Tardos ’08] Beyond independent RVs – Permutations [Harris-Srinivasan ‘14] – Abstract flaw-correction framework [Achlioptas-Iliopoulos ‘14, next talk]

Algorithmic Local Lemma for general probability spaces? For any application of LLL on a probability space , can we design a randomized algorithm to quickly find ! 2 Å i E i ? – [Achlioptas-Iliopoulos ’14] partially answers this; goes beyond LLL too.  How can algorithm “move about” in general probability space? Our key definition: resampling oracles

Consider probability space , measure ¹, events E 1, E 2,…, E n and dependency relation denoted ». A resampling oracle for E i is a random function r i :   !    Measure Regeneration: If !  has distribution ¹ conditioned on E i, then r i ( ! ) has distribution ¹. (Removes conditioning on E i )  Local Causation: If E k ¿ E i and !  E k, then r i ( ! )  E k. (Cannot cause non-neighbor events) Resampling Oracles  ! EiEi ri(!)ri(!)

Consider probability space , measure ¹, events E 1, E 2,…, E n and dependency relation denoted ». A resampling oracle for E i is a random function r i :   !    Measure Regeneration: If !  has distribution ¹ conditioned on E i, then r i ( ! ) has distribution ¹. (Removes conditioning on E i )  Local Causation: If E k ¿ E i and !  E k, then r i ( ! )  E k. (Cannot cause non-neighbor events) Resampling Oracles

Our Main Result: An Algorithmic LLL in a General Setting Arbitrary probability space  Events E 1, E 2,…, E n A dependency graph G for which E i independent of non-neighbors (more generally, positive association with non-nbrs) A resampling oracle for each E i 9 x 1,..., x n 2 ( 0, 1 ) such that P [ E i ] · x i ¢  j 2 ¡ ( i ) ( 1 - x j ) 8 i. Theorem: [H-Vondrak ‘15] There is an algorithm that finds a point ! 2 Å i E i after resampling operations, with high probability.

Rainbow trees Application: finding many disjoint rainbow-colored trees in edge-colored complete graphs.

Rainbow-tree packing conjecture Conjecture: [Brualdi-Hollingsworth ’96] For any proper edge-coloring of K 2n with 2n-1 colors, there is a decomposition of its edges into n rainbow spanning trees. Theorem: [Carraher et al ’13] If each color appears · n times (and n ¸ 10 6 ), then there are n / ( 1000 log n ) disjoint rainbow spanning trees. Our Result: If each color appears · n/50 times, then we can efficiently construct n / 50 disjoint rainbow spanning trees.

Disjoint rainbow trees via the LLL  Sample n / 50 random spanning trees, independently.  Hope that they are (a) edge-disjoint, and (b) rainbow. The magic of the Lovász Local Lemma... this actually works! Two types of bad events  E eik : the same edge e appears in two spanning trees T i and T k  F efi : two edges with same color appear in spanning tree T i Dependencies only happen if edges are incident ) P [ E eik ], P [ F efi ] · n - 2, d · n 2 / 3, so LLL applies

Resampling Spanning Trees in K n Let T be a uniformly random spanning tree in K n For edge set A, let E A = { A µ T }. Dependency Graph: Make E A a neighbor of E B, unless A and B are vertex-disjoint. Resampling oracle r A :  If T uniform conditioned on E A, want r A ( T ) uniform.  But, should not disturb edges that are vtx-disjoint from A.

E A = { A µ T }. Resampling oracle r A ( T ): – If T uniform conditioned on E A, want r A ( T ) uniform. – But, should not disturb edges that are vtx-disjoint from A. A Lemma: r A ( T ) is uniformly random.

E A = { A µ T }. Resampling oracle r A ( T ): – If T uniform conditioned on E A, want r A ( T ) uniform. – But, should not disturb edges that are vtx-disjoint from A. T – Contract edges of T vtx-disjoint from A Lemma: r A ( T ) is uniformly random.

E A = { A µ T }. Resampling oracle r A ( T ): – If T uniform conditioned on E A, want r A ( T ) uniform. – But, should not disturb edges that are vtx-disjoint from A. – Contract edges of T vtx-disjoint from A – Delete edges adjacent to A Lemma: r A ( T ) is uniformly random.

E A = { A µ T }. Resampling oracle r A ( T ): – If T uniform conditioned on E A, want r A ( T ) uniform. – But, should not disturb edges that are vtx-disjoint from A. – Contract edges of T vtx-disjoint from A – Delete edges adjacent to A – Let r A ( T ) be a uniformly random spanning tree in resulting graph. rA(T )rA(T ) Lemma: r A ( T ) is uniformly random.

LLL via Resampling Oracles  Given resampling oracles, how to get algorithm for LLL?

MIS Resample Draw ! from ¹ While any bad events occur in ! Pick a maximal independent set J of events that occur For each j 2 J, let ! Ã r i ( ! ) Output ! LLL via Resampling Oracles Note: In iteration i, we resample a maximal independent set J i of events. In iteration i + 1, all violated events are in ¡ + ( J i ) = ¡ ( J i ) [ J i. Thus, J i + 1 µ ¡ + ( J i ) for all i. Similar to Moser & Tardos’ parallel algorithm

Analysis Def: Seq = { I 1,…, I t : I s indep, I s  ;, I i + 1 µ ¡ + ( I i ) } Two Ingredients 1.Coupling Lemma: For any sequence I 1,…, I t of independent sets, P [ algorithm resamples I 1,…, I t ] · 2.Summation Bound: is small if t 0 is sufficiently large.

1. Coupling Lemma Resampling oracle r i for E i satisfies  If !  has distr ¹ conditioned on E i, then r i ( ! ) has distr ¹.  Does algorithm’s ! always have distribution ¹ ?  No: ! is also conditioned on some events not occurring.  Couple Alg with Alg( I 1,…, I t ) which checks in sequence if events I 1,…, I t occur and resamples them.  P [ Alg( I 1,…, I t ) succeeds ] · Draw ! from ¹ While any bad events occur in ! Pick a maximal independent set J of events that occur For each j 2 J, let ! Ã r i ( ! )

2. Summation Bound Let p i = P [ E i ]. Suppose p i · ( 1 - ² ) ¢ x i ¢  j 2 ¡ ( i ) ( 1 - x j ) 8 i Key Idea: (LLL) ) p i · ( 1 - ² ) x i “Binomial Formula” by (LLL) (LLL) This is ¼ 90% of the proof!

Our algorithm works under the cluster expansion condition and under Shearer’s condition Works for directed graphs if the LLL condition has ² slack We don’t know how to parallelize or derandomize our algorithm – Witness Tree Lemma is demonstrably false in our setting! Technical Details

Summary Algorithmic proof of LLL that works for any probability space and any events, under usual LLL conditions, so long as you can design resampling oracles. Efficiency is similar to Moser-Tardos, but slightly worse. Proof is similar to Moser-Tardos but actually simpler: no witness trees or branching processes