1. Making Time-series Classification More Accurate Using Learned Constraints © Chotirat “Ann” Ratanamahatana Eamonn Keogh 2004 SIAM International Conference on DATA MINING April 22, 2004

2. Roadmap Time series and their similarity measures Euclidean distance and its limitation Dynamic time warping (DTW) Global constraints R-K band Experimental Evaluation Conclusions and future work

3. Important Note! You are free to use any slides in this talk for teaching purposes, provide that the authorship of the slides is clearly attributed to Ratanamahatana and Keogh. You may not use any text or images contained here in a paper (including tech reports or unpublished works) or tutorial, without the express permission of Dr.Keogh. Chotirat Ann Ratanamahatana and Eamonn Keogh. Making Time-series Classification More Accurate Using Learned Constraints. In proceedings of SIAM International Conference on Data Mining (SDM '04), Lake Buena Vista, Florida, April 22-24, 2004. pp. 11-22Making Time-series Classification More Accurate Using Learned Constraints.

4. Classification in Time Series Classification, in general, maps data into predefined groups (supervised learning) Pattern Recognition is a type of supervised classification where an input pattern is classified into one of the classes based on its similarity to these predefined classes. Class B Class A Which class does belong to? AgeIncomeStudentCreditRatingClass: buy comp. 28HighNoFairNo 25HighNoExcellentNo 35HighNoFairYes 45MediumNoExcellentNo 18LowYesFairYes 49HighNoFair?? Will this person buy a computer?

5. Euclidean Distance Metric Given 2 time series Q = q 1, …, q n and C = c 1, …, c n their Euclidean distance is defined as Q C

6. Limitations of Euclidean Metric Very sensitive to some distortion in the data Training data consists of 10 instances from each of the 3 classes Training data consists of 10 instances from each of the 3 classes Perform a 1-nearest neighbor algorithm, with “leaving-one-out” evaluation, averaged over 100 runs. Euclidean distance Error rate: 29.77% DTW Error rate: 3.33 %

7. Dynamic Time Warping (DTW) Euclidean Distance One-to-one alignments Time Warping Distance Non-linear alignments are allowed

8. How Is DTW Calculated? (I) Q C Warping path w

9. How Is DTW Calculated? (II) Each warping path w can be found using dynamic programming to evaluate the following recurrence: where γ(i, j) is the cumulative distance of the distance d(i, j) and its minimum cumulative distance among the adjacent cells. (i-1, j) (i, j-1) (i, j) (i-1, j-1)

10. Global Constraints (I) Sakoe-Chiba BandItakura Parallelogram Prevent any unreasonable warping Prevent any unreasonable warping

11. Global Constraints (II) RiRi Sakoe-Chiba Band Itakura Parallelogram A Global Constraint for a sequence of size m is defined by R, where R i = d 0  d  m, 1  i  m. R i defines a freedom of warping above and to the right of the diagonal at any given point i in the sequence.

12. Is Wider the Band, the Better? DTW dist = 1.6389 R = 1 DTW dist = 1.0204 R = 25 DTW dist = 1.0204 R = 10 Euclidean distance = 2.4836 identical

13. Wider Isn’t Always Better Larger warping window is not always a good thing. Recall this example Most accuracies peak at smaller window size

14. Ratanamahatana-Keogh Band (R-K Band) Solution: we create an arbitrary shape and size of the band that is appropriate for the data we want to classify.

15. How Many Bands Do We Need? Of course, we could use ONE same band to classify all the classes, as almost all of the researchers do. But…the width of the band does depend on the characteristic of the data within each class. Having one single band for classification is unlikely to generalize. Our proposed solution: We create an arbitrary band (R-K band) for each class and use it accordingly for classification.

16. How Do We Create an R-K Band? First Attempt: We could look at the data and manually create the shape of the bands. (then we need to adjust the width of each band as well until we get a good result) 100 % Accuracy!

17. Learning an R-K Band Automatically Our heuristic search algorithm automatically learns the bands from the data. (sometimes, we can even get an unintuitive shape that give a good result.) 100 % Accuracy as well!

18. Solution Revisited An ideal solution is being able to find a way to perform classification accurately and in a timely fashion. Our approach utilizes the global constraint concept in creating an R-K band, as well as the Keogh’s lower bounding technique to speed up the calculation in search without degrading classification accuracy. In most cases, it further improves the accuracies of the classification after learning.

19. R-K Band Learning With Heuristic Search

20. R-K Band Learning in Action! Click on figure to animate

21. 1.Gun Problem 2.Trace (transient classification benchmark) 3.Handwritten Word Spotting data Experiment: Datasets

22. Experimental Design We measure the accuracy and CPU time of each dataset, using the following methods: 1.Euclidean distance 2.Uniformed warping window (size 1 to 100) 3.Learning different R-K bands for all classes, and perform classification based on them. The leaving-one-out in 1-nearest-neighbor classification is used to Measure the accuracy. The lower bounding method is also used to prune off unnecessary Calculation of DTW.

23. Experimental Results (I) Gun DrawPoint EuclideanBest Unif. Warping10% Unif. WarpingDTW with R-K Band Error Rate (%) 5.51.0 (width = 4)4.5 (width = 15)0.5 (max width = 4) CPU Time (msec) N/A2,4405,430 1,440 CPU Time (no LB) 60 11,82017,2909,440

24. Experimental Results (II) EuclideanBest Unif. Warping10% Unif. WarpingDTW with R-K Band Error Rate (%) 110 (width = 8)0 (width = 27)0 (max width = 7) CPU Time (msec) N/A16,02034,980 7,420 CPU Time (no LB) 210 144,470185,46088,630

25. Conclusions Different shapes and widths of the band contributes to the classification accuracy. Each class can be better recognized using its own individual R-K Band. Heuristic search algorithm is a good approach to R-K Band learning. R-K Band combining with the Lower Bounding technique yields higher accuracy and makes a classification task much faster.

26. Future Work Investigate other choices that may make envelope learning more accurate. –Heuristic functions –Search algorithm (refining the search) Is there a way to always guarantee an optimal solution? Examine the best way to deal with multi-variate time series. Consider a more generalized form of our framework, i.e. a single R-K Band is learned for a particular domain. Explore the utility of R-K Band specifically on real-world problems: music, bioinformatics, biomedical data, etc.

27. UCR Time Series Data Mining Archive: http://www.cs.ucr.edu/~eamonn/TSDMA Contact: ratana@cs.ucr.edu eamonn@cs.ucr.edu Homepage: http://www.cs.ucr.edu/~ratana All datasets are publicly available at: