New Characterizations in Turnstile Streams with Applications Yuqing Ai Tsinghua University Wei Hu Tsinghua University Yi Li Facebook David Woodruff IBM Almaden

Turnstile Streaming Model Underlying 𝑛-dimensional vector 𝑥 initialized to 0 Stream of updates 𝑥←𝑥+ 𝑒 𝑖 or 𝑥←𝑥− 𝑒 𝑖 for standard unit vector 𝑒𝑖 At end of the stream, 𝑥∈{−𝑚, …, −1, 0, 1,…, 𝑚}𝑛 Output an approximation to 𝑓(𝑥) w.h.p. Goal: use as small space in bits as possible

Example: Estimating the ℓ 2 -norm Output 𝑍 with 1−𝜖 𝑥 2 ≤𝑍≤ 1+𝜖 𝑥 2 Algorithm: Let 𝑟=1/ 𝜖 2 Choose an 𝑟×𝑛 matrix 𝐴 of i.i.d. sign random variables (+1 w.p. 1/2, −1 w.p. 1/2) Maintain 𝐴𝑥 in the stream Output 𝐴𝑥 2 𝑟

Generic Form All known algorithms have the following generic form (linear sketch): Sample a random matrix 𝐴 Maintain 𝐴𝑥 in the stream Output a function of 𝐴𝑥 Question (?!): does the optimal algorithm for approximating any function in the turnstile model have this form?

The LNW Reduction Yes! [Li, Nguyễn, Woodruff’14] Theorem: for computing a function 𝑓 of 𝑥 in −𝑚, …, 𝑚 𝑛 in the turnstile model, there is a randomized algorithm which samples a matrix 𝐴 and a vector 𝑞 uniformly from 𝑂(𝑛 log 𝑚 ) instances maintains (𝐴𝑥 mod 𝑞) in the stream outputs a function of (𝐴𝑥 mod 𝑞) Space complexity is optimal up to a constant factor (not including the 𝑂( log 𝑛 + log log 𝑚 ) bits for randomness)

Consequence Input 𝑥 Input 𝑦 Create stream 𝑠(𝑥) Create stream 𝑠(𝑦) Lower Bound Technique Streaming algorithm 𝒜 Run 𝒜 on 𝑠(𝑥), send state of 𝒜(𝑠(𝑥)) to Bob Bob computes 𝒜(𝑠(𝑥), 𝑠(𝑦)) If Bob solves 𝑔(𝑥,𝑦), space complexity of 𝒜 at least the 1-way communication complexity of 𝑔

Consequence Input 𝑥 Input 𝑦 Create stream 𝑠(𝑥) Create stream 𝑠(𝑦) The LNW reduction implies If players can solve 𝑔(𝑥,𝑦), then space of 𝒜 at least the simultaneous communication complexity of 𝑔 Weaker model in which Alice and Bob simultaneously send a message to a referee who outputs the answer

Our Result Strengthen the LNW reduction from several aspects: Remove the “box constraint” Generalize to the strict turnstile model Extend to multi-pass algorithms Obtain new tight lower bounds

Strengthen the LNW Reduction Remove the “box constraint” Generalize to the strict turnstile model Extend to multi-pass algorithms

The “Box Constraint” The LNW reduction requires the algorithm to be correct as long as 𝑥∈ −𝑚, …, 𝑚 𝑛 at the end of the stream. While processing the stream, may have 𝑥 ∞ ≫𝑚 The algorithm is not allowed to abort if this happens. It must still be correct at the end of the stream as long as 𝑥∈ −𝑚, …, 𝑚 𝑛 . More natural requirement: the algorithm only needs to be correct when 𝑥 belongs to −𝑚, …, 𝑚 𝑛 at all time in the stream.

Stream Automaton … + 𝑒 𝑛 … − 𝑒 𝑛 … − 𝑒 1 , + 𝑒 2 … + 𝑒 1 + 𝑒 1 + 𝑒 5 Start … + 𝑒 1 + 𝑒 1 + 𝑒 5 − 𝑒 1 … …

Path-Independent Automaton Every 𝑥∈ ℤ 𝑛 in a unique state

Path-Independent Automaton + 𝑒 𝑛 … − 𝑒 𝑛 … − 𝑒 1 , + 𝑒 2 Start … + 𝑒 1 + 𝑒 1 0 in two different states + 𝑒 5 − 𝑒 1 … …

Path-Independent Automaton Every 𝑥∈ ℤ 𝑛 in a unique state Equivalent to 𝐴𝑥 mod 𝑞

Zero-Frequency Graph For stream 𝜎, let freq 𝜎 ∈ ℤ 𝑛 be the “net update” to all coordinates. Zero-freq graph: directed graph 𝐺=(𝑉, 𝐸) 𝑉 = states of the automaton 𝑢, 𝑣 ∈𝐸 if there exists stream 𝜎 such that 𝑢⊕𝜎 =𝑣 and freq 𝜎 = 0 Terminal equivalence class: strongly connected component in 𝐺 with no outgoing edge Walk in G is a sequence of zero-frequency streams

The LNW Reduction 𝐺: zero-frequency graph of 𝒜 old States of new automaton 𝒜 new = terminal equivalence classes in 𝐺 For a terminal equivalence class 𝐶 and an update 𝑒 𝑖 , define transition as: Let 𝑣∈𝐶 be an arbitrary node Compute 𝑣⊕ 𝑒 𝑖 using transition function of 𝒜 old Walk from 𝑣⊕ 𝑒 𝑖 in 𝐺 until reach a terminal equivalence class 𝐶′ 𝐶′ is unique Does not depend on 𝑣 or the walk

Terminal equivalence class 𝐶 𝑣 𝑒𝑖 freq(𝜎) = 0 Terminal equivalence class 𝐶′

The Box Constraint For a stream 𝜎, define | 𝜎| max = max prefix 𝜔 of 𝜎 freq 𝜔 ∞ 𝜏 1 , 𝜏 2 , … are zero-frequency streams (walks in 𝐺) Length of 𝜏 𝑖 could be very large When | 𝜎| max ≤𝑚, | 𝜎′| max could be very large 𝜎=( 𝜎 1 , 𝜎 2 , …, 𝜎 𝑘 ) on 𝒜 new 𝜎′=(… ,𝜎 1 ,…, 𝜎 2 , …, 𝜎 𝑘 , …) on 𝒜 old 𝜏 1 𝜏 2 𝜏 3 𝜏 4 𝜏 5 𝜏 6 …

Zero-Freq Stream Length 𝐿: upper bound on the lengths of 𝜏 𝑖 ’s | 𝜎| max ≤𝑚 ⟹| 𝜎′| max ≤𝑚+𝐿/2 Want 𝐿≤𝑚 Let s = # states in 𝒜 old Lemma: if there is a zero-freq stream from 𝑢 to 𝑣, then there exists such a stream with length at most poly 𝑛𝑠 ⋅ 𝑠 𝑛 +1 𝑛 𝐿≤poly 𝑛𝑠 ⋅ 𝑠 𝑛 +1 𝑛

Tightness of Our Bound 𝐿≤poly 𝑛𝑠 ⋅ 𝑠 𝑛 +1 𝑛 Lower bound: 𝐿≥ 𝑠 𝑛 Ω(𝑛)

Removing the Box Constraint Want 𝐿≤𝑚 𝐿≤poly 𝑛𝑠 ⋅ 𝑠 𝑛 +1 𝑛 ≤ 𝑠 𝑐𝑛 𝐿≤𝑚 ⟸ 𝑠 𝑐𝑛 ≤𝑚 ⟸ log 𝑠 ≤ log 𝑚 𝑐𝑛 Space of 𝒜 old

Application: Counting 𝑛=1 Problem: output |𝑥| up to additive error 𝑚/4, while 𝑥 varies in {−𝑚, …, 𝑚} 𝑂( log 𝑚 ) space algorithm Is there an Ω( log 𝑚 ) lower bound? For insertion streams, no: approximate counting For relative error, yes: but proof doesn’t apply For additive error… yes!

Application: Counting Condition for removing box constraint: space ≤ log 𝑚 𝑐𝑛 = log 𝑚 𝑐 Assume space ≤ log 𝑚 𝑐 , otherwise done 𝐴𝑥 mod 𝑞=( 𝑎 1 𝑥 mod 𝑞 1 , 𝑎 2 𝑥 mod 𝑞 2 , …, 𝑎 𝑟 𝑥 mod 𝑞 𝑟 ) Show lcm 𝑞 1 , …, 𝑞 𝑟 =Ω(𝑚) Cannot distinguish 𝑥, 𝑥+lcm, 𝑥+2⋅lcm, … Ω(𝑚) different states, Ω( log 𝑚 ) space

Application: Norm Estimation Problem: for 𝑥∈ −𝑚, …, 𝑚 𝑛 , output 𝑥 𝑝 up to additive error 1 4 𝑛 1/𝑝 𝑚 Ω( log 𝑚 ) space lower bound 𝑂( log 𝑚 + log log 𝑛 ) space algorithm (1≤𝑝≤2) [KNW’10] Lower bound tight when log log 𝑛 =𝑂 log 𝑚 ⟺ 𝑛 ≤ exp poly(𝑚)

Strengthen the LNW Reduction Remove the “box constraint” Generalize to the strict turnstile model Extend to multi-pass algorithms

The Strict Turnstile Model The strict turnstile model: no negative coordinates, i.e., 𝑥 𝑖 ≥0 at all times in the stream Dynamic graph streams: insertions and deletions of edges Allow multi-graphs, but no negative edges Generalize the LNW reduction to the strict turnstile model 𝐿: upper bound on the length of zero-freq streams Initialize all coordinates of 𝑥 to be 𝐿 Now the reduction guarantees 𝑥 is always nonnegative Subtract 𝐿 from all coordinates at the end of the stream

Application: Maximum Matching [AKLY’16]: For outputting an 𝑛 𝜖 -approximate maximum matching, space is Θ ( 𝑛 2−3𝜖 ) Lower bound only in simultaneous communication model Can apply our reduction

Strengthen the LNW Reduction Remove the “box constraint” Generalize to the strict turnstile model Extend to multi-pass algorithms

Multi-Pass Algorithms 𝑝-pass automaton After 𝑖-th pass (𝑖<𝑝), output an automaton 𝒜 𝑖+1 Run 𝒜 𝑖+1 on input stream in (𝑖+1)-st pass After 𝑝-th pass, output answer Theorem: There is a 𝑝-pass automaton for which each automaton in each pass is path-independent Space is optimal up to a constant factor

Conclusions New progress on characterizing turnstile streaming algorithms as linear sketches Applications Optimal lower bounds for counting with additive error, maximum matching in dynamic graph Open questions Box constraint After removing box constraint, still have very long streams Better reduction? Thank you!