Download presentation

Presentation is loading. Please wait.

Published byKevin Poole Modified over 5 years ago

1
Optimal Bounds for Johnson- Lindenstrauss Transforms and Streaming Problems with Sub- Constant Error T.S. Jayram David Woodruff IBM Almaden

2
Data Stream Model Have a stream of m updates to an n-dimensional vector v –add x to coordinate i –Insertion model -> all updates x are positive –Turnstile model -> x can be positive or negative –stream length and updates < poly(n) Estimate statistics of v –# of distinct elements F 0 –L p -norm |v| p = (Σ i |v i | p ) 1/p –entropy –and so on Goal: output a (1+ ε)-approximation with limited memory

3
Lots of Optimal Papers Lots of optimal results –An optimal algorithm for the distinct elements problem [KNW] –Fast moment estimation in optimal space [KNPW] –A near-optimal algorithm for estimating entropy of a stream [CCM] –Optimal approximations of the frequency moments of data streams [IW] –A near-optimal algorithm for L1-difference [NW] –Optimal space lower bounds for all frequency moments [W] This paper –Optimal Bounds for Johnson-Lindenstrauss Transforms and Streaming Problems with Sub-Constant Error

4
What Is Optimal? F 0 = # of non-zero entries in v For a stream of indices in {1, …, n}, our algorithm computes a (1+ε)-approximation using an optimal O(ε -2 + log n) bits of space with 2/3 success probability… This probability can be amplified by independent repetition. If we want high probability, say, 1-1/n, this increases the space by a multiplicative log n So optimal algorithms are only optimal for algorithms with constant success probability

5
Can We Improve the Lower Bounds? x 2 {0,1} ε -2 y 2 {0,1} ε -2 Gap-Hamming: either Δ(x,y) > ½ + ε or Δ(x,y) < ½-ε Lower bound of Ω(ε -2 ) with 1/3 error probability But upper bound of ε -2 with 0 error probability

6
Our Results

7
Streaming Results Independent repetition is optimal! Estimating L p -norm in turnstile model up to 1+ε w.p. 1-δ –Ω(ε -2 log n log 1/δ) bits for any p –[KNW] get O(ε -2 log n log 1/δ) for 0 · p · 2 Estimating F 0 in insertion model up to 1+ε w.p. 1-δ –Ω(ε -2 log 1/δ + log n) bits –[KNW] get O(ε -2 log 1/δ) for ε -2 > log n Estimating entropy in turnstile model up to 1+ε w.p. 1-δ –Ω(ε -2 log n log 1/δ) bits –Improves Ω(ε -2 log n) bound [KNW]

8
Johnson-Lindenstrauss Transforms Let A be a random matrix so that with probability 1- δ, for any fixed q 2 R d |Aq| 2 = (1 ± ε) |q| 2 [JL] A can be a 1/ε 2 log 1/δ x d matrix –Gaussians or sign variables work [Alon] A needs to have (1/ε 2 log 1/δ) / log 1/ε rows Our result: A needs to have 1/ε 2 log 1/δ rows

9
Communication Complexity Separation x y f(x,y) 2 {0,1} D 1/3, ρ (f) = communication of best 1-way deterministic protocol that errs w.p. 1/3 on distribution ρ [KNR]: R || 1/3 (f) = max product distributions ¹ £ λ D ¹ £ λ,1/3 (f) 0 1

10
Communication Complexity Separation f(x,y) 2 {0,1} VC-dimension: maximum number r of columns for which all 2 r rows occur in communication matrix on these columns [KNR]: R || 1/3 (f) = Θ(VC-dimension(f)) Our result: there exist f and g with VC-dimension k, but: R || δ (f) = Θ(k log 1/δ) while R || δ (g) = Θ(k)

11
Our Techniques

12
Lopsided Set Intersection (LSI) S ½ {1, 2, …, U} |S| = 1/ ε 2 U = 1/ ε 2 ¢ 1/δ Is S Å T = ; ? - Alice cannot describe S with o( ε -2 log U) bits - If x, y are uniform then with constant probability, S Å T = ; - R || 1/3 (LSI) > D uniform, 1/3 (LSI) = Ω(ε -2 log 1/δ) T ½ {1, 2, …, U} |T| = 1/δ

13
Lopsided Set Intersection (LSI2) S ½ {1, 2, …, U} |S| = 1/ ε 2 U = 1/ ε 2 ¢ 1/δ Is S Å T = ; ? T ½ {1, 2, …, U} |T| = 1 -R || δ/3 (LSI2) ¸ R | | 1/3 (LSI) = Ω(ε -2 log 1/δ) - Union bound over set elements in LSI instance

14
Low Error Inner Product x 2 {0, ε} U |x| 2 = 1 y 2 {0, 1} U |y| 2 = 1 Does = 0? Estimate up to ε w.p. 1-δ -> solve LSI2 w.p. 1-δ R || δ (inner product ε ) = Ω(ε -2 log 1/δ) U = 1/ε 2 ¢ 1/δ

15
L 2 -estimation ε x 2 {0, ε} U |x| 2 = 1 What is |x-y| 2 ? U = 1/ ε 2 ¢ 1/δ y 2 {0, 1 } U |y| 2 = 1 - |x-y| 2 2 = |x| 2 2 + |y| 2 2 - 2 = 2 – 2 - Estimate |x-y| 2 up to (1+ Θ( ε))-factor solves inner-product ε - So R || δ (L 2 -estimation ε ) = Ω(ε -2 log 1/δ) - log 1/δ factor is new, but want an (ε -2 log n log 1/δ) lower bound - Can use a known trick to get an extra log n factor - log 1/δ factor is new, but want an (ε -2 log n log 1/δ) lower bound - Can use a known trick to get an extra log n factor

16
Augmented Lopsided Set Intersection (ALSI2) Universe [U] = [1/ ε 2 ¢ 1/δ] S 1, …, S r ½ [U] All i: |S i | = 1/ ε 2 j 2 [U] i * 2 {1, 2, …, r} S i*+1 …, S r Is j 2 S i * ? R || 1/3 (ALSI2) = (r ε -2 log 1/δ)

17
Reduction of ALSI2 to L 2 -estimation ε S1S2…SrS1S2…Sr x1x2…xrx1x2…xr j S i*+1 … S r y i* x i*+1 … x r } x } y y - x = 10 i* y i* - i i* 10 i ¢ x i |y-x| 2 is dominated by 10 i* |y i* – x i* | 2 - Set r = Θ(log n) - R || δ (L 2 -estmation ε ) = ( ε -2 log n log 1/δ) - Streaming Space > R || δ (L 2 -estimation ε ) - Set r = Θ(log n) - R || δ (L 2 -estmation ε ) = ( ε -2 log n log 1/δ) - Streaming Space > R || δ (L 2 -estimation ε )

18
Lower Bounds for Johnson- Lindenstrauss Use public randomness to agree on a JL matrix A x 2 {-n O(1), …, n O(1) } t AxAy- |A(x-y)| 2 - Can estimate |x-y| 2 up to 1+ ε w.p. 1-δ - #rows(A) = (r ε -2 log 1/δ / log n) - Set r = Θ(log n) - Can estimate |x-y| 2 up to 1+ ε w.p. 1-δ - #rows(A) = (r ε -2 log 1/δ / log n) - Set r = Θ(log n) y 2 {-n O(1), …, n O(1) } t

19
Low-Error Hamming Distance Universe = [n ] Δ(x,y) = Hamming Distance between x and y x 2 {0,1} n y 2 {0,1} n R || δ (Δ(x,y) ε ) = (ε -2 log 1/ δ log n) Reduction to ALSI2 Gap-Hamming to LSI2 reductions with Low Error Implies our lower bounds for estimating Any L p -norm Distinct Elements Entropy

20
Conclusions Prove first streaming space lower bounds that depend on probability of error δ –Optimal for L p -norms, distinct elements –Improves lower bound for entropy –Optimal dimensionality bound for JL transforms Adds several twists to augmented indexing proofs –Augmented indexing with a small set in a large domain –Proof builds upon lopsided set disjointness lower bounds –Uses multiple Gap-Hamming to Indexing reductions that handle low error

21
ALSI2 to Hamming Distance - Let t = 1/ ε 2 log 1/δ - Use public coin to generate t random strings b 1, …, b t 2 {0,1} t - Alice sets x i = majority k in S i b i, k - Bob sets y i = b i,j S 1, …, S r ½ [1/ ε 2 ¢ 1/δ ] All i: |S i | = 1/ ε 2 j 2 [U] i * 2 {1, 2, …, r} S i*+1 …, S r Embed multiple copies by duplicating coordinates at different scales

Similar presentations

Presentation is loading. Please wait....

OK

Lecturer: Moni Naor Weizmann Institute of Science

Lecturer: Moni Naor Weizmann Institute of Science

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google