Presentation is loading. Please wait.

Presentation is loading. Please wait.

1/26 Constructive Algorithms for Discrepancy Minimization Nikhil Bansal (IBM)

Similar presentations


Presentation on theme: "1/26 Constructive Algorithms for Discrepancy Minimization Nikhil Bansal (IBM)"— Presentation transcript:

1 1/26 Constructive Algorithms for Discrepancy Minimization Nikhil Bansal (IBM)

2 2/26 Combinatorial Discrepancy Universe: U= [1,…,n] Subsets: S 1,S 2,…,S m Color elements red/blue so each set is colored as evenly as possible. S1S1 S2S2 S3S3 S4S4

3 3/26 Applications CS: Computational Geometry, Comb. Optimization, Monte-Carlo simulation, Machine learning, Complexity, Pseudo-Randomness, … Math: Dynamical Systems, Combinatorics, Mathematical Finance, Number Theory, Ramsey Theory, Algebra, Measure Theory, …

4 4/26 General Set System Universe: U= [1,…,n] Subsets: S 1,S 2,…,S m Find  : [n] ! {-1,+1} to Minimize |  (S)| 1 = max S |  i 2 S  (i) | For simplicity consider m=n henceforth.

5 5/26 Best Known Algorithm Random: Color each element i independently as x(i) = +1 or -1 with probability ½ each. Thm: Discrepancy = O (n log n) 1/2 Pf: For each set, expect O(n 1/2 ) discrepancy Standard tail bounds: Pr[ |  i 2 S x(i) | ¸ c n 1/2 ] ¼ e -c 2 Union bound + Choose c ¼ (log n) 1/2 Analysis tight: Random actually incurs  (n log n) 1/2 ).

6 6/26 Better Colorings Exist! [Spencer 85]: (Six standard deviations suffice) Always exists coloring with discrepancy · 6n 1/2 (In general for arbitrary m, discrepancy = O(n 1/2 log(m/n) 1/2 ) Tight: For m=n, cannot beat 0.5 n 1/2 (Hadamard Matrix, “orthogonal” sets) Inherently non-constructive proof (pigeonhole principle on exponentially large universe) Challenge: Can we find it algorithmically ? Certain algorithms do not work [Spencer] Conjecture [Alon-Spencer]: May not be possible.

7 7/26 Approximating Discrepancy Question: If a set system has low discrepancy (say << n 1/2 ) Can we find a good discrepancy coloring ? [Charikar, Newman, Nikolov 11]: Even 0 vs. O (n 1/2 ) is NP-Hard (Matousek): What if system has low Hereditary discrepancy? herdisc (U,S) = max U’ ½ U disc (U’, S |U’ ) Robust measure of discrepancy Widely used: TU set systems, Geometry, … S1S2…S1S2… S’ 1 S’ 2 … 1 2 … n1’ 2’ … n’

8 8/26 Our Results Thm 1: Can get Spencer’s bound constructively. That is, O(n 1/2 ) discrepancy for m=n sets. Thm 2: For any set system, can find Discrepancy · O(log (mn)) Hereditary discrepancy. General Technique: Constructive bounds for geometric problems, Beck Fiala setting, k-permutation problem, …

9 9/26 Relaxations: LPs and SDPs Not clear how to use. Linear Program is useless. Can color each element ½ red and ½ blue. Discrepancy of each set = 0! SDPs (vector coloring) |  i 2 S v i | 2 · n 8 S |v i | 2 = 1 Intended solution v i = (+1,0,…,0) or (-1,0,…,0). Trivially feasible: v i = e i (all v i ’s orthogonal) Yet, SDPs will be the major tool.

10 10/26 Key Point The SDP gap example does not work if discrepancy << n 1/2 As we will see, SDPs are very useful in that regime. But seems useless for Spencer’s problem Idea: “Tighter” bounds for some sets. |  i 2 S v i | 2 · 2 n |  i 2 S’ v i | 2 · n/log n |v i | 2 = 1 Why can one do this: Entropy Method. Tighter bound for S’

11 11/26 Talk Outline Introduction The Method Low Hereditary discrepancy -> Good coloring Additional Ideas Spencer’s O(n 1/2 ) bound

12 12/26 Algorithm (at high level) Cube: {-1,+1} n Analysis: Few steps to reach a vertex (walk has high variance) Disc( S i ) does a random walk (with low variance) start finish Algorithm: “Sticky” random walk Each step generated by rounding a suitable SDP Move in various dimensions correlated, e.g.  t 1 +  t 2 ¼ 0 Each dimension: An Element Each vertex: A Coloring

13 13/26 An SDP Hereditary disc. ) the following SDP is feasible SDP: Low discrepancy: |  i 2 S j v i | 2 · 2 |v i | 2 = 1 Rounding: Pick random Gaussian g = (g 1,g 2,…,g n ) each coordinate g i is iid N(0,1) For each i, consider  i = g ¢ v i Obtain v i 2 R n

14 14/26 Properties of Rounding Lemma: If g 2 R n is random Gaussian. For any v 2 R n, g ¢ v is distributed as N(0, |v| 2 ) Pf: N(0,a 2 ) + N(0,b 2 ) = N(0,a 2 +b 2 ) g ¢ v =  i v(i) g i » N(0,  i v(i) 2 ) 1.Each  i » N(0,  ) 2.For each set S,  i 2 S  i = g ¢ (  i 2 S v i ) » N(0, · 2 ) (std deviation · ) SDP: |v i | 2 = 1 |  i 2 S v i | 2 · 2 Recall:  i = g ¢ v i  ’s mimics a low discrepancy coloring (but is not {-1,+1})

15 15/26 Algorithm Overview Construct coloring iteratively. Initially: Start with coloring x 0 = (0,0,0, …,0) at t = 0. At Time t: Update coloring as x t = x t-1 +  (  t 1,…,  t n ) (  tiny: 1/n suffices) x(i) x t (i) =  (  1 i +  2 i + … +  t i ) Color of element i: Does random walk over time with step size ¼  Fixed if reaches -1 or +1. time +1 Set S: x t (S) =  i 2 S x t (i) does a random walk w/ step  N(0, · 2 )

16 16/26 Analysis Consider time T = O(1/  2 ) Claim 1: With prob. ½, at least n/2 elements reach -1 or +1. Pf: Each element doing random walk with size ¼  Recall: Random walk with step 1, is ¼ O(t 1/2 ) away in t steps. A Trouble: Walks for various elements are correlated Consider basic walk x(t+1) = x(t) 1 with prob ½ Define Energy  (t) = x(t) 2 if never reached n 1/2 =  (t-1) + 1 otherwise E[  (t)] = ½ (x(t-1)+1) 2 + ½ (x(t-1)-1) 2 = x(t-1) 2 + 1 =  (t-1)+1 So, E[  (10 n) ] = 10 n. Moreover,  (10 n) · 11 n So only a small fraction of walks can have  < n.

17 17/26 Analysis Consider time T = O(1/  2 ) Claim 2: Each set has O( ) discrepancy in expectation. Pf: For each S, x t (S) doing random walk with step size ¼  Define a round as T = O(1/  2 ) time steps. Claim 1: ) Everything colored in O(log n) rounds. Claim 2: ) Expected discrepancy of a set at end = O( log n) + By Chernoff Bounds, discrepancy = O( log mn) whp over all sets.

18 18/26 Recap At each step of walk, formulate SDP on unfixed variables. Use some (existential) property to argue SDP is feasible Rounding SDP solution -> Step of walk Properties of walk: High Variance -> Quick convergence Low variance for discrepancy on sets -> Low discrepancy

19 19/26 Refinements Spencer’s six std deviations result: Goal: Obtain O(n 1/2 ) discrepancy for any set system on m = O(n) sets. Random coloring has n 1/2 (log n) 1/2 discrepancy Previous approach seems useless: Expected discrepancy for a set O(n 1/2 ), but some random walks will deviate by up to (log n) 1/2 factor Need an additional idea to prevent this.

20 20/26 Spencer’s O(n 1/2 ) result Partial Coloring Lemma: For any system with m sets, there exists a coloring on ¸ n/2 elements with discrepancy O(n 1/2 log 1/2 (2m/n)) [For m=n, disc = O(n 1/2 )] Algorithm for total coloring: Repeatedly apply partial coloring lemma Total discrepancy O( n 1/2 log 1/2 2 ) [Phase 1] + O( (n/2) 1/2 log 1/2 4 ) [Phase 2] + O((n/4) 1/2 log 1/2 8 ) [Phase 3] + … = O(n 1/2 )

21 21/26 Proving Partial Coloring Lemma Beautiful Counting argument (entropy method + pigeonhole) Idea: Too many colorings (2 n ), but few “discrepancy profiles” Key Lemma: There exist k=2 4n/5 colorings X 1,…,X k such that every two X i, X j are “similar” for every set S 1,…,S n. Some X 1,X 2 differ on ¸ n/2 positions Consider X = (X 1 – X 2 )/2 Pf: X(S) = (X 1 (S) – X 2 (S))/2 2 [-10 n 1/2, 10 n 1/2 ] X 1 = ( 1,-1, 1, …,1,-1,-1) X 2 = (-1,-1,-1, …,1, 1, 1) X = ( 1, 0, 1, …,0,-1,-1)

22 22/26 A useful generalization There exists a partial coloring with non-uniform discrepancy bounds  S for set S Provided  S =  ( n 1/2 ) in some average sense

23 23/26 An SDP Suppose there exists partial coloring X: 1. On ¸ n/2 elements 2. Each set S has |X(S)| ·  S SDP: Low discrepancy: |  i 2 S j v i | 2 ·  S 2 Many colors:  i |v i | 2 ¸ n/2 |v i | 2 · 1 Pick random Gaussian g = (g 1,g 2,…,g n ) each coordinate g i is iid N(0,1) For each i, consider  i = g ¢ v i Obtain v i 2 R n

24 24/26 Algorithm Initially write SDP with  S = c n 1/2 Each set S does random walk and expects to reach discrepancy of O(  S ) = O(n 1/2 ) Some sets will become problematic. Reduce their  S on the fly. Not many problematic sets, and entropy penalty low. 0 20n 1/2 30n 1/2 35n 1/2 … Danger 1 Danger 2 Danger 3 …

25 25/26 Concluding Remarks Probable right answer: O( (log m) 1/2 ) for her. Disc. Can be derandomized [Bansal-Spencer] (add new constraints to SDP) Gives derandomization for prob. inequalities stronger than Chernoff bounds. (Exponential moment technique for derandomizing Chernoff loses (log n) 1/2 ) Other non-constructive problems: Lattices (Minkowski Thm) Fixed-Point Based (Nash, Sperner’s Lemma) Topological (Hypergraph matching), … Discrepancy problems: Beck Fiala Conjecture (more generally Komlos conjecture) Erdos Discrepancy problem 3-permutation conjecture, …

26 26/26 Thank You!


Download ppt "1/26 Constructive Algorithms for Discrepancy Minimization Nikhil Bansal (IBM)"

Similar presentations


Ads by Google