Download presentation

Presentation is loading. Please wait.

Published byHoward Sanburn Modified about 1 year ago

1
Sparse Approximations Nick Harvey University of British Columbia TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A

2
Approximating Dense Objects by Sparse Objects Floor joists Wood JoistsEngineered Joists

3
Approximating Dense Objects by Sparse Objects Bridges Masonry ArchTruss Arch

4
Approximating Dense Objects by Sparse Objects Bones Human Femur Robin Bone

5
Mathematically Can an object with many pieces be approximately represented by fewer pieces? Independent random sampling usually does well Theme of this talk: When can we beat random sampling? Dense Graph Sparse Graph Dense Matrix Sparse Matrix

6
Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

7
Discrepancy Given vectors v 1, …, v n 2 R d with k v i k p bounded. Want y 2 {- 1, 1 } n with k i y i v i k q small. Eg1: If k v i k 1 · 1 then E k i y i v i k 1 · Eg2: If k v i k 1 · 1 then 9 y s.t. k i y i v i k 1 · Spencer ‘85: Partial Coloring + Entropy Method Gluskin ‘89: Sidak’s Lemma Giannopoulos ‘97: Partial Coloring + Sidak Bansal ‘10: Brownian Motion + Semidefinite Program Bansal-Spencer ‘11: Brownian Motion + Potential function Lovett-Meka ‘12: Brownian Motion Non-algorithmic Algorithmic

8
Discrepancy Given vectors v 1, …, v n 2 R d with k v i k p bounded. Want y 2 {- 1, 1 } n with k i y i v i k q small. Eg1: If k v i k 1 · 1 then E k i y i v i k 1 · Eg2: If k v i k 1 · 1 then 9 y s.t. k i y i v i k 1 · Eg3: If k v i k 1 · ¯, k v i k 1 · ±, and k i v i k 1 · 1, then 9 y with k i y i v i k 1 · Harvey ’13: Using Lovasz Local Lemma. Question: Can log ( ± / ¯ 2 ) factor be improved?

9
Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

10
Partitioning sums of rank- 1 matrices Let v 1, …, v n 2 R d satisfy i v i v i T =I and k v i k 2 · ±. Want y 2 { - 1, 1 } n with k i y i v i v i T k 2 small. Random sampling: E k i y i v i v i T k 2 ·. Rudelson ’96: Proofs using majorizing measures, then nc-Khintchine Marcus-Spielman-Srivastava ’13: 9 y 2 { - 1, 1 } n with k i y i v i v i T k 2 ·. 2

11
Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k i y i M i k 2 small. Random sampling: E k i y i M i k 2 · Also follows from nc-Khintchine. Ahlswede-Winter ’02: Using matrix moment generating function. Tropp ‘12: Using matrix cumulant generating function.

12
Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k i y i M i k 2 small. Random sampling: E k i y i M i k 2 · Question: 9 y 2 {- 1, 1 } n with k i y i M i k 2 · ? Conjecture: Suppose i M i =I and k M i k Sch-1 · ±. 9 y 2 {- 1, 1 } n with k i y i M i k 2 · ? – MSS ’13: Rank-one case is true – Harvey ’13: Diagonal case is true (ignoring log ( ¢ ) factor) False!

13
Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k i y i M i k 2 small. Random sampling: E k i y i M i k 2 · Question: Suppose only that k M i k 2 · 1. 9 y 2 {- 1, 1 } n with k i y i M i k 2 · ? – Spencer/Gluskin: Diagonal case is true

14
Column-subset selection Given vectors v 1, …, v n 2 R d with k v i k 2 = 1. Let st.rank = n / k i v i v i T k 2. Let. 9 y 2 { 0, 1 } n s.t. i y i = k and ( 1 - ² ) 2 · ¸ k ( i y i v i v i T ). Spielman-Srivastava ’09: Potential function argument Youssef ’12: Let. 9 y 2 { 0, 1 } n s.t. i y i = k, ( 1 - ² ) 2 · ¸ k ( i y i v i v i T ) and ¸ 1 ( i y i v i v i T ) · ( 1 + ² ) 2.

15
Column-subset selection up to the stable rank Given vectors v 1, …, v n 2 R d with k v i k 2 = 1. Let st.rank = n / k i v i v i T k 2. Let. For y 2 { 0, 1 } n s.t. i y i = k, can we control ¸ k ( i y i v i v i T ) and ¸ 1 ( i y i v i v i T ) ? –¸ k can be very small, say O ( 1 / d ). – Rudelson’s theorem: can get ¸ 1 · O ( log d ) and ¸ k > 0. – Harvey-Olver ’13: ¸ 1 · O ( log d / log log d ) and ¸ k > 0. – MSS ‘13: If i v i v i T = I, can get ¸ 1 · O ( 1 ) and ¸ k > 0.

16
Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

17
Graph Laplacian L u = D - A = a b c d a b cd weighted degree of node c negative of u ( ac ) Graph with weights u : Laplacian Matrix: a b d c Effective Resistance from s to t : voltage difference when each edge e is a ( 1 / u e )-ohm resistor and a 1 -amp current source placed between s and t = ( e s - e t ) T L u y ( e s - e t ) Effective Conductance: c st = 1 / (effective resistance from s to t )

18
Spectral approximation of graphs ® -spectral sparsifier: L u ¹ L w ¹ ® ¢ L u Edge weights u Edge weights w Lu =Lu = Lw =Lw =

19
Ramanujan Graphs Suppose L u is complete graph on n vertices ( u e = 1 8 e ). Lubotzky-Phillips-Sarnak ’86: For infinitely many d and n, 9 w 2 { 0, 1 } E such that e w e = dn / 2 (actually L w is d -regular) and MSS ‘13: Holds for all d ¸ 3, and all n =c ¢ 2 k. Friedman ‘04: If L w is a random d -regular graph, then 8 ² >0 with high probability.

20
Arbitrary graphs Spielman-Srivastava ’08: For any graph L u with n =| V |, 9 w 2 R E such that | support ( w )| = O ( n log ( n )/ ² 2 ) and Proof: Follows from Rudelson’s theorem MSS ’13: For any graph L u with n =| V |, 9 w 2 R E such that w e 2 £ ( ² 2 ) ¢ N ¢ (effective conductance of e ) | support ( w )| = O ( n / ² 2 ) and

21
Spectrally-thin trees Question: Let G be an unweighted graph with n vertices. Let C = min e (effective conductance of edge e ). Want a subtree T of G with. Equivalent to Goddyn’s Conjecture ‘85: There is a subtree T with – Relates to conjectures of Tutte (‘54) on nowhere-zero flows, and to approximations of the traveling salesman problem.

22
Spectrally-thin trees Question: Let G be an unweighted graph with n vertices. Let C = min e (effective conductance of edge e ). Want a subtree T of G with. Rudelson’s theorem: Easily gives ® = O ( log n ). Harvey-Olver ‘13: ® = O ( log n / log log n ). Moreover, there is an efficient algorithm to find such a tree. MSS ’13: ® = O ( 1 ), but not algorithmic.

23
Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

24
Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works.

25
Matrix Concentration Theorem: [Tropp ‘12] Let Y 1,…, Y m be independent, PSD matrices of size n x n. Let Y = i Y i and Z = E [ Y ]. Suppose Y i ¹ R ¢ Z a.s. Then

26
Define sampling probabilities x e = 1 / c e. It is known that e x e = n – 1. Claim: Independent sampling gives T µ E with E [| T |]= n – 1 and Theorem [Tropp ‘12]: Let M 1,…, M m be n x n PSD matrices. Let D ( x ) be a product distribution on { 0, 1 } m with marginals x. Let Suppose M i ¹ Z. Then Define M e = c e ¢ L e. Then Z = L G and M e ¹ Z holds. Setting ® = 6 log n / log log n, we get whp. But T is not a tree! Independent sampling Laplacian of the single edge e Properties of conductances used

27
Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works. ► Run pipage rounding to get tree T with Pr[ e 2 T ] = x e = 1 / c e

28
Pipage rounding [Ageev-Svirideno ‘04, Srinivasan ‘01, Calinescu et al. ‘07, Chekuri et al. ‘09] Let P be any matroid polytope. E.g., convex hull of characteristic vectors of spanning trees. Given fractional x Find coordinates a and b s.t. line z x + z ( e a – e b ) stays in current face Find two points where line leaves P Randomly choose one of those points s.t. expectation is x Repeat until x = Â T is integral x is a martingale: expectation of final Â T is original fractional x. Â T1 Â T2 Â T3 Â T4 Â T5 ÂT6ÂT6 x

29
Say f : R m ! R is concave under swaps if z ! f ( x + z ( e a - e b ) ) is concave 8 x 2 P, 8 a, b 2 [ m ]. Let X 0 be initial point and Â T be final point visited by pipage rounding. Claim: If f concave under swaps then E [ f ( Â T )] · f ( X 0 ). [Jensen] Let E µ { 0, 1 } m be an event. Let g : [ 0, 1 ] m ! R be a pessimistic estimator for E, i.e., Claim: Suppose g is concave under swaps. Then Pr[ Â T 2 E ] · g ( X 0 ). Pipage rounding and concavity

30
Chernoff Bound Chernoff Bound: Fix any w, x 2 [ 0, 1 ] m and let ¹ = w T x. Define. Then, Claim: g t, µ is concave under swaps. [Elementary calculus] Let X 0 be initial point and Â T be final point visited by pipage rounding. Let ¹ = w T X 0. Then Bound achieved by independent sampling also achieved by pipage rounding

31
Matrix Pessimistic Estimators Main Theorem: g t, µ is concave under swaps. Theorem [Tropp ‘12]: Let M 1,…, M m be n x n PSD matrices. Let D ( x ) be a product distribution on { 0, 1 } m with marginals x. Let Suppose M i ¹ Z. Let Then and. Bound achieved by independent sampling also achieved by pipage rounding Pessimistic estimator

32
Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works. ► Run pipage rounding to get tree T with Pr[ e 2 T ] = x e = 1 / c e

33
Matrix Analysis Matrix concentration inequalities are usually proven via sophisticated inequalities in matrix analysis Rudelson: non-commutative Khinchine inequality Ahlswede-Winter: Golden-Thompson inequality if A, B symmetric, then tr ( e A + B ) · tr ( e A e B ). Tropp: Lieb’s concavity inequality [1973] if A, B Hermitian and C is PD, then z ! tr exp ( A + log ( C + zB ) ) is concave. Key technical result: new variant of Lieb’s theorem if A Hermitian, B 1, B 2 are PSD, and C 1, C 2 are PD, then z ! tr exp ( A + log ( C 1 + zB 1 ) + log ( C 2 – zB 2 ) ) is concave.

34
Questions Can Spencer/Gluskin theorem be extended to matrices? Can MSS’13 be made algorithmic? Can MSS’13 be extended to large-rank matrices? O ( 1 )-spectrally thin trees exist. Can one be found algorithmically? Are O ( 1 )-spectrally thin trees helpful for Goddyn’s conjecture?

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google