1. Turnstile Streaming Algorithms Might as Well Be Linear Sketches
Yi Li Huy L. Nguyen David Woodruff
Max-Planck Princeton IBM Almaden

2. Turnstile Streaming Model
Underlying n-dimensional vector x initialized to 0n
Long stream of updates x à x + ei or x à x - ei for standard unit vector ei
At end of the stream, x 2 {-m, -m+1, …, m-1, m}n for some bound m · poly(n)
Output an approximation to f(x) whp
Goal: use as little space (in bits) as possible

3. Example: Euclidean Norm
Want to output Z with (1-Ɛ) |x|2 · Z · (1+Ɛ) |x|2
Let r = 1/Ɛ2
Choose an r x n matrix A of i.i.d. sign random variables (+1 w.pr. ½, -1 w.pr. ½)
Maintain Ax in the stream
Output |Ax|2
Proof: Johnson-Lindenstrauss Lemma

4. Generic Features Algorithm for 2-norm has the following form:
Choose a random matrix A independent of x
Maintain Ax in the stream
Output a function of Ax
Question (?!): does the optimal algorithm for approximating any function in the turnstile model have this form?
All known algorithms have this form
Some functions f(x) may be weird:
What is xx1?

5. Our Result Yes, up to a factor of log n
Theorem: for computing a relation f for x in {-m, -m+1, …, m}n in the turnstile model, there is a correct (whp) algorithm which:
samples an integer matrix A uniformly from O(n log m) hardwired matrices with poly(n) bounded integer entries, independent of x,
outputs a function of Ax
Logarithm of the number of states of Ax, for x in {-m, -m+1, …, m}n, plus amount of randomness, is optimal up to a log n factor

6. Consequences b 2 {0,1}n a 2 {0,1}n Create stream s(b)
Create stream s(a)
Lower Bound Technique
1. Run Alg on s(a), transmit state of Alg(s(a)) to Bob
2. Bob computes Alg(s(a), s(b))
3. If Bob solves g(a,b), space complexity of Alg at least the 1-way communication complexity of g

7. Consequences a 2 {0,1}n Create stream s(a) b 2 {0,1}n
Create stream s(b)
Our main theorem implies:
If players can solve g(a,b), then space of Alg at least the simultaneous communication complexity of g
Weaker public-coin model in which Alice and Bob simultaneously send a message to a referee

8. Non-Uniformity Restriction
Careful wording: “samples an integer matrix A uniformly from O(n log m) hardwired matrices, with poly(n) bounded entries, independent of x”
Algorithm is non-uniform
Output of each state for each A also hardwired
Alternatively, allow algorithm to use more space to process a stream update, provided it only retains Ax and its randomness
Regenerate A during each stream update

9. Comment on the Model
For each random seed, algorithm is a deterministic automaton with a finite number of states
Main theorem only requires correctness for
x 2 {-m, -m+1, …, m}n
It counts the number of states as x varies in this range
While processing the stream, may have |x|1 > m
The algorithm can’t abort if this happens. It must still be correct at the end of the stream for x in {-m, -m+1, …, m}n

10. Related Work Ganguly Specific to heavy hitters problem
Holds only for deterministic algorithms

11. Talk Outline Proof Overview Applications and Open Questions
Reduction to path-independent automata
From path-independent automata to linear sketches
Applications and Open Questions

12. Stream Automaton for Fixed Randomness…
Streaming algorithm only depends on x, not how it got there
+en …
-en …
-e1, +e2
Start …
+e1
+e1
0n in two different states
+e5
-e1 … …

13. Path-Independent Automaton
Each x 2 Zn in a unique state
Undirected connected graph
Goal: for each randomness, can we modify the automaton to make it path-independent?
Rule out algorithms that e.g., an algorithm that stores the last 5 stream updates

14. Intuitively makes things path-independent
Strategy
Intuitively makes things path-independent
For stream σ, freq(σ) 2 Zn is “net update” to each
coordinate
Idea: 1. if in a state s, and update by a stream σ,
with freq(σ) = 0, answers ought to be similar
2. collapse all states s, s’ for which s+σ = s’ and
freq(σ) = 0 for some stream σ
Issue: how to formally define states, transition and
output function of new automaton?

15. Zero-Frequency Graph Directed multi-graph G = (V,E)
V = states of old automaton Aold (for fixed randomness)
(s,t) 2 E for each stream σ of finite length with s+σ=t and freq(σ) = 0
Terminal equivalence class: strongly connected component with no outgoing edge
Walk in G eventually reaches a terminal equivalence class
(Walk in G is a long sequence of zero-streams)
States of new automaton Anew = terminal equivalence classes

16. New Transition Function
Suppose in terminal equivalence class C
Given an update ei
Let v 2 C be an arbitrary node
Compute v+ei using transition function of Aold
Walk from v+ei until reach terminal equivalence class C’
C’ is unique
Does not depend on choice of v
Only one terminal equivalence class reachable in any walk

17. Contradiction: zero frequency path w-x-v-y
Terminal equivalence class
-ei x y u v
Contradiction: zero frequency
path w-x-v-y
+ei
+ei
freq(σ) = 0
freq(σ’) = 0
Terminal equivalence class
Terminal equivalence class w

18. Output Function of Anew
In each terminal equivalence class C, sample node u from stationary distribution from random walk in C (add self-loops)
Output of Anew on C = Output of Aold on u
If v is starting vertex of Aold,
take a random walk in G from v
let starting vertex of Anew be terminal equivalence class C reached
Why is it correct?

19. Correctness Let ¦ be an arbitrary distribution on streams ¾
Choose fixed randomness so Aold correct on ¦’:
Long sequence of zero frequency streams,
Followed by ¾ sampled from ¦,
Followed by long sequence of zero frequency streams
Output of Anew on ¦ statistically close to output of Aold on ¦’
=> for every ¦ there is an Anew correct on ¦
Can show Anew is path-independent (lying a little..)

20. Path Independence to Linear Sketches
M = {x 2 Zn such that x in same state as 0n}
States of automaton are cosets of Zn/M
Use lattice tools…

21. Applications and Open Questions
Simpler proof of existing lower bounds
No communication complexity
Many dimension lower bounds known for sketching norms over the reals
Matrix norms, etc.
Do these give turnstile streaming lower bounds with finite precision?