Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees Hazem Elmeleegy Purdue University Ahmed Elmagarmid (Purdue) Emmanuel Cecchet (UMass) Walid Aref (Purdue) Willy Zwaenepoel (EPFL)

Outline Introduction Swing & Slide Filters Experiments Conclusion

Application Scenario TransmitterReceiver Some Common Applications:  Cluster Monitoring  Sensor Networks  Stock Market

The Problem Goal  Minimize amount of transmitted data Saves bandwidth Saves storage (at the receiver side) Saves battery life (esp. for sensor networks)  Using piece-wise linear approximation Assumptions  Receiver can tolerate: A bounded error for each data point received (max error =  ) A maximum lag behind the transmitter Terminology  We refer to any algorithm to solve this problem as a filtering technique, or simply a filter

Existing Techniques t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4 x5x5 Cache Filter  The transmitter caches the last transmitted value.  A new value is transmitted only if it is more than  away from the cached value.  Piece-wise constant approximation

Existing Techniques t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4 x5x5 Cache Filter  The transmitter caches the last transmitted value.  A new value is transmitted only if it is more than  away from the cached value.  Piece-wise constant approximation

Existing Techniques t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4 x5x5 Linear Filter  The transmitter maintains a line segment that can approximate the last observed data points.  The line segment is updated only when a new data point falls more than  away from the maintained line.

Outline

Swing and Slide Filters Key Idea  Maintain a set of candidate line segments at any given time  Postpone the selection decision as late as possible to accommodate more points

Swing Filter t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4 x5x5 Connected line segments Complexity  Maintains upper and lower segments only  O(1) space and time complexity Lag  If max lag is exceeded, switch to linear filter Correctness  Proof of correctness in the paper

Slide Filter t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3

Slide Filter t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4

Slide Filter t1t1 t2t2 t3t3 t4t4 t5t5 Time          Value x1x1 x2x2 x3x3 x4x4 x5x5 Optimization #1  Connect line segments whenever possible Optimization #2  Do not maintain all the data points currently being approximated  Maintain their convex hull only Complexity  O(h) space and time complexity  h is the number of data points on the convex hull --- very small in practice Lag  If max lag is exceeded, switch to linear filter Correctness  Proof of correctness in the paper

Outline

Compression Ratios for the Sea Temperature Signal Sea Surface Temperature 

Effect of Signal Behavior (Degree of Monotonicity) Synthetic Signal: Random walk with probability p to increase and (1-p) to decrease.

Overhead for the Sea Temperature Signal 

Outline

Conclusion We introduced two new filtering techniques: the swing and slide filters They have significantly higher compression ratios compared to earlier techniques, especially the slide filter They have a small overhead, and hence are suitable for overhead-sensitive applications

Thank you Questions?