1. Fast Subsequence Matching in Time-Series Databases.
C. Faloustos, M. Ranganathan,
Y. Manolopoulos
Presented by
George Liu / Luis L. Perez

2. Time series? Definition Applications Financial markets
Weather forecasting
Healthcare

3. What kind of problem are we trying to solve?
Whole sequence matching
Given a database S with n sequences, all of them equally long, and a query sequence Q of the same length.
Find all sequences in S that match with Q.
Subsequence matching
Given a database S with n sequences, with potentially different lengths, and a query sequence Q.
Find all sequences in S that contain Q.

4. Useful notation Given a sequence S Given two sequences, S and Q
Len(S) denotes the length of the sequence
S[i] denotes the ith element
S[i:j] denotes the subsequence between S[i] and S[j]
Given two sequences, S and Q
D(S,Q) denotes the distance between S and Q.
Euclidean
Distance bound: e
Max. distance for two sequences to be considered “equal”

5. Naïve approaches Sequential scanning R-tree
Clearly unfeasible
R-tree
Might work, but dimensionality is extremely high (proportional to sequence length)‏
Poor performance
What can we do to improve performance?

6. Dimensionality reduction
Redundant data, lots of patterns
Feature extraction
Data transformation
Cosine
Wavelet
Fourier <-- we'll focus on this.

7. Discrete Fourier Transformation
Map a sequence x in time-domain to a sequence X in frequency-domain
Reversible!
Fast and easy-to-implement algorithms
Energy preservation property
Key concept in dimensionality reduction.
Just keep the first 2 or 3 coefficients.

8. Parseval's theorem Let S and Q be the original sequences.
S' and Q' after applying DFT D(S,Q) = D(S',Q')
Why is this important?
Distance underestimation, remember the bound e.
D(S,Q) < e ---> D(S', Q') < e
We will get no false dismissals.

9. Subsequence Matching The problem: Solutions:
You are given a collection of N sequences of real numbers. (S1, S2, .., Sn). Potentially different length.
User specifies query subsequence of length Q and the tolerance e, the max. acceptable dis-similarity.
You want all to return all the sequences along with the correct offsets k that matches the query and acceptable e.
Solutions:
many!

10. Possible Solutions
1) Brute Force method - Sequential scan every possible subsequence of the data sequences for a match.
2) I-Naive - Transform all subsequences to points in feature space and store those points into an R-tree.
3) ST-Index - Transform all subsequences to points in feature space. Store MBRs of sub-trails into an R*-tree.
Note: I-Naive and ST-Index are similar in the initial steps.

11. Possible Solutions I-naive
*Assume that the min. query length is w. w changes according to the application. (ie, stock markets have a larger w that are interested in weekly/monthly patterns)‏
Procedure:
1) Use the "sliding window" to find every subsequence in a sequence.
2) DFT those subsequences of size w to a point in featured space.
3) A trail is produced of Len(S)-w+1 points.

12. Possible Solutions I-naive
Procedure cont:
4) Store all the points of the trails in feature space in a spatial access method. (R*-tree)‏
5) When presented with a query of length w and tolerance e, extract the features of the query and perform the spatial access range query with radius e.
6) Discard false alarms by retrieving all those subsequences and calculating their actual distance from the query.
Note: Very, very slow approach. Worst that Sequential Scan. You have a large R*-tree (tall and slow).

13. Possible Solutions ST-Index
*Assume that the min. query length is w. w changes according to the application. (ie, stock markets have a larger w that are interested in weekly/monthly patterns)‏
Procedure:
1) Use the "sliding window" to find every subsequence in a sequence.
2) DFT those subsequences of size w to a point in featured space.
3) A trail is produced of Len(S)-w+1 points.

14. Possible Solutions ST-Index
Procedure cont.
4) Divide the trail of points in feature space into sub-trails. (algorithm mentioned later)‏
5) Represent each of them in a MBR.
6) Store the MBR into a spatial access method. (ie. R*-Tree)‏

15. MBRs in F-Dimension

16. MBRs in F-Dimension

17. MBRs in F-Dimension

18. MBRs in F-Dimension

19. MBRs in F-Dimension

20. Insertions Problem: How do we divide these trails into sub-trails?
Two heuristics:
1) Every sub-trail has a predetermined, fixed number. (I-fixed)‏
2) Every sub-trail has a predetermined, fixed length. (I-fixed)‏
Solution: Use an "adaptive heuristic." (I-adaptive)‏

21. I-adaptive Algorithm
- Based on the idea of the marginal cost of a point in terms of disk accesses.
Marginal cost (mc) = Disk Accesses of a given MBR / k points in a given MBR
Algorithm
Assign the first point of the trail in a sub-trail.
FOR each successive point
IF it increase the marginal cost of the current sub-trail
THEN start another sub-trail
ELSE include it in the current sub-trail

22. I-adaptive Algorithm

23. Searching Consider the sub-trail length w and distance bound e.
Let Q be the query sequence
If Len(Q) = w, it's all good.
Algorithm Search_Short:
Use DFT to map Q to a point q in feature space. Make it a sphere with radius e.
Retrieve all the sub-trails whose MBRs intersect the query region using our index.
Throw away false alarms.

24. Searching Now, what if Len(Q) > w?
Requires more analysis, but basically we have that Len(Q) = p*w
So we can split Q in several subsequences of length p.
What about the radius? r = e/sqrt(p)‏

25. Searching So we have... Algorithm Search_Long:
Break sequence Q in p sub-queries with radius e/sqrt(p)‏
Retrieve from the index all the sub-trails whose MBRs insersect at least one of the other sub-query regions.
Examine the sub-sequences, discard false alarms.

26. Experimental results

27. Experimental results Stock price database with ~300,000 points
1 number = 4 bytes
DFT keeping first 3 coefficients (actually 6)
w = 512 bytes
R*-tree

28. Experimental results Space Time - “short” queries (Len(Q) = w)‏
Naïve methods: 24mb
This method: 5kb
Time - “short” queries (Len(Q) = w)‏
3 to 100 times better response times
Time - “long” queries (Len(Q) > w)‏
10 to 100 times better response times