Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sparse Approximations Nick Harvey University of British Columbia TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A.

Similar presentations


Presentation on theme: "Sparse Approximations Nick Harvey University of British Columbia TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A."— Presentation transcript:

1 Sparse Approximations Nick Harvey University of British Columbia TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A

2 Approximating Dense Objects by Sparse Objects Floor joists Wood JoistsEngineered Joists

3 Approximating Dense Objects by Sparse Objects Bridges Masonry ArchTruss Arch

4 Approximating Dense Objects by Sparse Objects Bones Human Femur Robin Bone

5 Mathematically Can an object with many pieces be approximately represented by fewer pieces? Independent random sampling usually does well Theme of this talk: When can we beat random sampling? Dense Graph Sparse Graph Dense Matrix Sparse Matrix

6 Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

7 Discrepancy Given vectors v 1, …, v n 2 R d with k v i k p bounded. Want y 2 {- 1, 1 } n with k  i y i v i k q small. Eg1: If k v i k 1 · 1 then E k  i y i v i k 1 · Eg2: If k v i k 1 · 1 then 9 y s.t. k  i y i v i k 1 · Spencer ‘85: Partial Coloring + Entropy Method Gluskin ‘89: Sidak’s Lemma Giannopoulos ‘97: Partial Coloring + Sidak Bansal ‘10: Brownian Motion + Semidefinite Program Bansal-Spencer ‘11: Brownian Motion + Potential function Lovett-Meka ‘12: Brownian Motion Non-algorithmic Algorithmic

8 Discrepancy Given vectors v 1, …, v n 2 R d with k v i k p bounded. Want y 2 {- 1, 1 } n with k  i y i v i k q small. Eg1: If k v i k 1 · 1 then E k  i y i v i k 1 · Eg2: If k v i k 1 · 1 then 9 y s.t. k  i y i v i k 1 · Eg3: If k v i k 1 · ¯, k v i k 1 · ±, and k  i v i k 1 · 1, then 9 y with k  i y i v i k 1 · Harvey ’13: Using Lovasz Local Lemma. Question: Can log ( ± / ¯ 2 ) factor be improved?

9 Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

10 Partitioning sums of rank- 1 matrices Let v 1, …, v n 2 R d satisfy  i v i v i T =I and k v i k 2 · ±. Want y 2 { - 1, 1 } n with k  i y i v i v i T k 2 small. Random sampling: E k  i y i v i v i T k 2 ·. Rudelson ’96: Proofs using majorizing measures, then nc-Khintchine Marcus-Spielman-Srivastava ’13: 9 y 2 { - 1, 1 } n with k  i y i v i v i T k 2 ·. 2

11 Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with  i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k  i y i M i k 2 small. Random sampling: E k  i y i M i k 2 · Also follows from nc-Khintchine. Ahlswede-Winter ’02: Using matrix moment generating function. Tropp ‘12: Using matrix cumulant generating function.

12 Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with  i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k  i y i M i k 2 small. Random sampling: E k  i y i M i k 2 · Question: 9 y 2 {- 1, 1 } n with k  i y i M i k 2 · ? Conjecture: Suppose  i M i =I and k M i k Sch-1 · ±. 9 y 2 {- 1, 1 } n with k  i y i M i k 2 · ? – MSS ’13: Rank-one case is true – Harvey ’13: Diagonal case is true (ignoring log ( ¢ ) factor) False!

13 Partitioning sums of matrices Given d x d symmetric matrices M 1, …, M n 2 R d with  i M i =I and k M i k 2 · ±. Want y 2 {- 1, 1 } n with k  i y i M i k 2 small. Random sampling: E k  i y i M i k 2 · Question: Suppose only that k M i k 2 · 1. 9 y 2 {- 1, 1 } n with k  i y i M i k 2 · ? – Spencer/Gluskin: Diagonal case is true

14 Column-subset selection Given vectors v 1, …, v n 2 R d with k v i k 2 = 1. Let st.rank = n / k  i v i v i T k 2. Let. 9 y 2 { 0, 1 } n s.t.  i y i = k and ( 1 - ² ) 2 · ¸ k (  i y i v i v i T ). Spielman-Srivastava ’09: Potential function argument Youssef ’12: Let. 9 y 2 { 0, 1 } n s.t.  i y i = k, ( 1 - ² ) 2 · ¸ k (  i y i v i v i T ) and ¸ 1 (  i y i v i v i T ) · ( 1 + ² ) 2.

15 Column-subset selection up to the stable rank Given vectors v 1, …, v n 2 R d with k v i k 2 = 1. Let st.rank = n / k  i v i v i T k 2. Let. For y 2 { 0, 1 } n s.t.  i y i = k, can we control ¸ k (  i y i v i v i T ) and ¸ 1 (  i y i v i v i T ) ? –¸ k can be very small, say O ( 1 / d ). – Rudelson’s theorem: can get ¸ 1 · O ( log d ) and ¸ k > 0. – Harvey-Olver ’13: ¸ 1 · O ( log d / log log d ) and ¸ k > 0. – MSS ‘13: If  i v i v i T = I, can get ¸ 1 · O ( 1 ) and ¸ k > 0.

16 Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

17 Graph Laplacian L u = D - A = a b c d a b cd weighted degree of node c negative of u ( ac ) Graph with weights u : Laplacian Matrix: a b d c Effective Resistance from s to t : voltage difference when each edge e is a ( 1 / u e )-ohm resistor and a 1 -amp current source placed between s and t = ( e s - e t ) T L u y ( e s - e t ) Effective Conductance: c st = 1 / (effective resistance from s to t )

18 Spectral approximation of graphs ® -spectral sparsifier: L u ¹ L w ¹ ® ¢ L u Edge weights u Edge weights w Lu =Lu = Lw =Lw =

19 Ramanujan Graphs Suppose L u is complete graph on n vertices ( u e = 1 8 e ). Lubotzky-Phillips-Sarnak ’86: For infinitely many d and n, 9 w 2 { 0, 1 } E such that  e w e = dn / 2 (actually L w is d -regular) and MSS ‘13: Holds for all d ¸ 3, and all n =c ¢ 2 k. Friedman ‘04: If L w is a random d -regular graph, then 8 ² >0 with high probability.

20 Arbitrary graphs Spielman-Srivastava ’08: For any graph L u with n =| V |, 9 w 2 R E such that | support ( w )| = O ( n log ( n )/ ² 2 ) and Proof: Follows from Rudelson’s theorem MSS ’13: For any graph L u with n =| V |, 9 w 2 R E such that w e 2 £ ( ² 2 ) ¢ N ¢ (effective conductance of e ) | support ( w )| = O ( n / ² 2 ) and

21 Spectrally-thin trees Question: Let G be an unweighted graph with n vertices. Let C = min e (effective conductance of edge e ). Want a subtree T of G with. Equivalent to Goddyn’s Conjecture ‘85: There is a subtree T with – Relates to conjectures of Tutte (‘54) on nowhere-zero flows, and to approximations of the traveling salesman problem.

22 Spectrally-thin trees Question: Let G be an unweighted graph with n vertices. Let C = min e (effective conductance of edge e ). Want a subtree T of G with. Rudelson’s theorem: Easily gives ® = O ( log n ). Harvey-Olver ‘13: ® = O ( log n / log log n ). Moreover, there is an efficient algorithm to find such a tree. MSS ’13: ® = O ( 1 ), but not algorithmic.

23 Talk Outline Vignette #1: Discrepancy theory Vignette #2: Singular values and eigenvalues Vignette #3: Graphs Theorem on “Spectrally Thin Trees”

24 Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works.

25 Matrix Concentration Theorem: [Tropp ‘12] Let Y 1,…, Y m be independent, PSD matrices of size n x n. Let Y =  i Y i and Z = E [ Y ]. Suppose Y i ¹ R ¢ Z a.s. Then

26 Define sampling probabilities x e = 1 / c e. It is known that  e x e = n – 1. Claim: Independent sampling gives T µ E with E [| T |]= n – 1 and Theorem [Tropp ‘12]: Let M 1,…, M m be n x n PSD matrices. Let D ( x ) be a product distribution on { 0, 1 } m with marginals x. Let Suppose M i ¹ Z. Then Define M e = c e ¢ L e. Then Z = L G and M e ¹ Z holds. Setting ® = 6 log n / log log n, we get whp. But T is not a tree! Independent sampling Laplacian of the single edge e Properties of conductances used

27 Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works. ► Run pipage rounding to get tree T with Pr[ e 2 T ] = x e = 1 / c e

28 Pipage rounding [Ageev-Svirideno ‘04, Srinivasan ‘01, Calinescu et al. ‘07, Chekuri et al. ‘09] Let P be any matroid polytope. E.g., convex hull of characteristic vectors of spanning trees. Given fractional x Find coordinates a and b s.t. line z  x + z ( e a – e b ) stays in current face Find two points where line leaves P Randomly choose one of those points s.t. expectation is x Repeat until x =  T is integral x is a martingale: expectation of final  T is original fractional x.  T1  T2  T3  T4  T5 ÂT6ÂT6 x

29 Say f : R m ! R is concave under swaps if z ! f ( x + z ( e a - e b ) ) is concave 8 x 2 P, 8 a, b 2 [ m ]. Let X 0 be initial point and  T be final point visited by pipage rounding. Claim: If f concave under swaps then E [ f (  T )] · f ( X 0 ). [Jensen] Let E µ { 0, 1 } m be an event. Let g : [ 0, 1 ] m ! R be a pessimistic estimator for E, i.e., Claim: Suppose g is concave under swaps. Then Pr[  T 2 E ] · g ( X 0 ). Pipage rounding and concavity

30 Chernoff Bound Chernoff Bound: Fix any w, x 2 [ 0, 1 ] m and let ¹ = w T x. Define. Then, Claim: g t, µ is concave under swaps. [Elementary calculus] Let X 0 be initial point and  T be final point visited by pipage rounding. Let ¹ = w T X 0. Then Bound achieved by independent sampling also achieved by pipage rounding

31 Matrix Pessimistic Estimators Main Theorem: g t, µ is concave under swaps. Theorem [Tropp ‘12]: Let M 1,…, M m be n x n PSD matrices. Let D ( x ) be a product distribution on { 0, 1 } m with marginals x. Let Suppose M i ¹ Z. Let Then and. Bound achieved by independent sampling also achieved by pipage rounding Pessimistic estimator

32 Given an (unweighted) graph G with eff. conductances ¸ C. Can find an unweighted tree T with Spectrally Thin Trees Proof overview: 1.Show independent sampling gives spectral thinness, but not a tree. ► Sample every edge e independently with prob. x e = 1 / c e 2.Show dependent sampling gives a tree, and spectral thinness still works. ► Run pipage rounding to get tree T with Pr[ e 2 T ] = x e = 1 / c e

33 Matrix Analysis Matrix concentration inequalities are usually proven via sophisticated inequalities in matrix analysis Rudelson: non-commutative Khinchine inequality Ahlswede-Winter: Golden-Thompson inequality if A, B symmetric, then tr ( e A + B ) · tr ( e A e B ). Tropp: Lieb’s concavity inequality [1973] if A, B Hermitian and C is PD, then z ! tr exp ( A + log ( C + zB ) ) is concave. Key technical result: new variant of Lieb’s theorem if A Hermitian, B 1, B 2 are PSD, and C 1, C 2 are PD, then z ! tr exp ( A + log ( C 1 + zB 1 ) + log ( C 2 – zB 2 ) ) is concave.

34 Questions Can Spencer/Gluskin theorem be extended to matrices? Can MSS’13 be made algorithmic? Can MSS’13 be extended to large-rank matrices? O ( 1 )-spectrally thin trees exist. Can one be found algorithmically? Are O ( 1 )-spectrally thin trees helpful for Goddyn’s conjecture?


Download ppt "Sparse Approximations Nick Harvey University of British Columbia TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A."

Similar presentations


Ads by Google