1. Pattern Finding and Pattern Discovery in Time Series
Long Q Tran
College of Computing, Georgia Tech
Trần Quốc Long
College of Computing, Georgia Tech

2. Contents Pattern Finding & Pattern Discovery
Pattern Finding & Pattern Discovery in Time Series
Hidden Markov Models (HMMs)
Summary

3. Pattern Finding
Problems: given observed patterns O1, O2, … OK, specify which pattern the new data X possess?
Other names: pattern recognition, pattern classification
Examples
Recognition: matching fingerprints of the claimant with those of authorized personnel.

4. Pattern Finding
Patterns are known beforehand and are observed/described by
Explicit samples
Similar samples (usually)
Modeling approaches:
Build a model for each pattern
Find the best fit model for new data
Usually require training using observed samples

5. Pattern Discovery Patterns are not known
But data which are believed to possess patterns are given
Examples:
Clustering: grouping similar samples into clusters
Associative rule mining: discover certain features that often appear together in data

6. Contents Pattern Finding & Pattern Discovery
Pattern Finding & Pattern Discovery in Time Series
Hidden Markov Models (HMMs)
Summary

7. Time Series Data are sampled over time X = X1 X2 … Xt … XL
Xt : data sampled at time t
L : sequence length
Xt are NOT independently and identically distributed (NOT i.d.d)
In other words, Xt may come from different processes that are dependent of each other

8. Pattern Finding in Time Series
Examples
In control, certain pattern of sensor signals indicate critical point of the production process
In stock, certain pattern (up/down) of price indicate the trend of the market
People often have to look at the graph by their own eyes and act accordingly when spotting known pattern
X. Ge & P. Smyth (2000): detecting end-point
in plasma etch (semiconductor manufacturing)

9. Pattern Finding in Time Series
Problems:
Data may contain one or more patterns inside
Data can be multi-dimensional (i.e. look at multiple graphs at the same time)
Automated pattern finding is crucial when time series are lengthy and multi-dimensional

10. Pattern Discovery in Time Series
Goals: From collected data, discover
Replicated, interesting patterns
Associative rule on patterns (can use to predict trends of time series)

11. Pattern Modeling in Time Series
Both pattern finding and pattern discovery need modeling
Desired properties of the model
The model can be built or trained using observed data
The similarity of new data and the model can be easily computed

12. Contents Pattern Finding & Pattern Discovery
Pattern Finding & Pattern Discovery in Time Series
Hidden Markov Models (HMMs)
Summary

13. Hidden Markov Models (HMMs)
One way to model time series pattern
Assumptions:
Xt is generated from certain probability distribution Yt (called state)
Number of states is finite (i.e. finite sources of data)
State transition follows Markov property X1 Y1 X2 Y2 XL YL … 1 2
0.6
0.4 1 2
0.4
0.6

14. Hidden Markov Models (HMMs)
Parameters to estimate:
Transition probabilities
Distribution parameters in each state
Estimation procedure:
Initialization: k-means, viterbi training
Iterative training: forward-backward procedure (EM algorithm)
Variants of HMM:
Mixture of HMMs: allow many HMMs computed simultaneously
State durational HMM: allow a state remains for a duration

15. Mixture of HMMs Assumption: Mixture of HMMs allows
There are different processes (pattern) that generate the time series
Each process can be represented by a HMM
Mixture of HMMs allows
Packing all pattern models in one place
Identifying the processes that generate the time series
Training be efficiently implemented

16. Experiment Experiment settings Generate 200 sequences for each HMM1 2
0.6
0.4 1 2
0.4
0.6
Experiment settings
Generate 200 sequences for each HMM
After 200 iterations 2
Gaussian:  = 0,  = 1 1
Gaussian:  = 2,  = 1 2
11 = -0.07, 11 = 0.97
21 = 2.01, 21 = 0.99
12 = 1.90, 12 = 1.10
22 = -0.01, 22 = 0.98

17. Summary
Automated pattern finding and pattern discovery in time series are needed
HMMs and its variants can model time series patterns
Parameters can be efficiently initialized and estimated using observed data

18.  = (transition prob., distribution params.)
Appendix: HMMs
Parameters:
 = (transition prob., distribution params.)
Recognition
Calculate P(X1X2…XL |)
Forward procedure
Estimation:
Maximize L() = P(X1X2…XL |)
EM algorithm: forward – backward procedure
Clustering
Find
Viterbi algorithm