Presentation is loading. Please wait.

Presentation is loading. Please wait.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Sequential Pattern Mining COMP 790-90 Seminar BCB 713 Module Spring 2011.

Published byChad Foster Modified over 8 years ago

Similar presentations

Presentation on theme: "The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Sequential Pattern Mining COMP 790-90 Seminar BCB 713 Module Spring 2011."— Presentation transcript:

1 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Sequential Pattern Mining COMP 790-90 Seminar BCB 713 Module Spring 2011

2 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 2 Sequential Pattern Mining Why sequential pattern mining? GSP algorithm FreeSpan and PrefixSpan Boarder Collapsing Constraints and extensions

3 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 3 Sequence Databases and Sequential Pattern Analysis (Temporal) order is important in many situations Time-series databases and sequence databases Frequent patterns  (frequent) sequential patterns Applications of sequential pattern mining Customer shopping sequences: First buy computer, then CD-ROM, and then digital camera, within 3 months. Medical treatment, natural disasters (e.g., earthquakes), science & engineering processes, stocks and markets, telephone calling patterns, Weblog click streams, DNA sequences and gene structures

4 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 4 What Is Sequential Pattern Mining? Given a set of sequences, find the complete set of frequent subsequences A sequence database A sequence : An element may contain a set of items. Items within an element are unordered and we list them alphabetically. is a subsequence of Given support threshold min_sup =2, is a sequential pattern SIDsequence 10 20 30 40

5 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 5 Challenges on Sequential Pattern Mining A huge number of possible sequential patterns are hidden in databases A mining algorithm should Find the complete set of patterns satisfying the minimum support (frequency) threshold Be highly efficient, scalable, involving only a small number of database scans Be able to incorporate various kinds of user- specific constraints

6 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 6 A Basic Property of Sequential Patterns: Apriori A basic property: Apriori (Agrawal & Sirkant’94) If a sequence S is not frequent Then none of the super-sequences of S is frequent E.g, is infrequent  so do and 50 40 30 20 10 SequenceSeq. ID Given support threshold min_sup =2

7 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 7 Basic Algorithm : Breadth First Search (GSP) L=1 While (Result L != NULL) Candidate Generate Prune Test L=L+1

8 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 8 Finding Length-1 Sequential Patterns Initial candidates: all singleton sequences,,,,,,, Scan database once, count support for candidates 50 40 30 20 10 SequenceSeq. ID min_sup =2 CandSup 3 5 4 3 3 2 1 1

9 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 9 The Mining Process … … … … 1 st scan: 8 cand. 6 length-1 seq. pat. 2 nd scan: 51 cand. 19 length-2 seq. pat. 10 cand. not in DB at all 3 rd scan: 46 cand. 19 length-3 seq. pat. 20 cand. not in DB at all 4 th scan: 8 cand. 6 length-4 seq. pat. 5 th scan: 1 cand. 1 length-5 seq. pat. Cand. cannot pass sup. threshold Cand. not in DB at all 50 40 30 20 10 SequenceSeq. ID min_sup =2

10 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 10 Generating Length-2 Candidates 51 length-2 Candidates Without Apriori property, 8*8+8*7/2=92 candidates Apriori prunes 44.57% candidates

11 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 11 Generating Length-4 Candidates Frequent 3-Sequences Candidates (after join) Candidates (After pruning)

12 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 12 Pattern Growth (prefixSpan) Prefix and Suffix (Projection),, and are prefixes of sequence Given sequence PrefixSuffix (Prefix-Based Projection)

13 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 13 Example Sequence_id Sequence 10 20 30 40 An Example ( min_sup=2): PrefixSequential Patterns,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

14 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 14 PrefixSpan (the example to be continued) Step1: Find length-1 sequential patterns; :4, :4, :4, :3, :3, :3 pattern support Step2: Divide search space; six subsets according to the six prefixes; Step3: Find subsets of sequential patterns; By constructing corresponding projected databases and mine each recursively.

15 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 15 Example to be continued PrefixProjected(suffix) databasesSequential Patterns,,,,,,,,,,,,,,,,,, Sequence_id SequenceProjected(suffix) databases 10 20 30 40

16 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 16 Example Find sequential patterns having prefix : 1.Scan sequence database S once. Sequences in S containing are projected w.r.t to form the - projected database. 2.Scan -projected database once, get six length-2 sequential patterns having prefix : :2, :4, :2, :4, :2, :2 3.Recursively, all sequential patterns having prefix can be further partitioned into 6 subsets. Construct respective projected databases and mine each. e.g. -projected database has two sequences : and.

17 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 17 PrefixSpan Algorithm PrefixSpan( , i, S|  ) 1.Scan S|  once, find the set of frequent items b such that b can be assembled to the last element of  to form a sequential pattern; or can be appended to  to form a sequential pattern. 2.For each frequent item b, appended it to  to form a sequential pattern  ’, and output  ’; 3.For each  ’, construct  ’-projected database S|  ’, and call PrefixSpan(  ’, i+1,S|  ’). Main Idea: Use frequent prefixes to divide the search space and to project sequence databases. only search the relevant sequences.

18 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 18 Approximate match When you observe d1 Spread count as d1: 90%, d2: 5%, d3: 5% Compatibility Matrix

19 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 19 Match The degree to which pattern P is retained/reflected in S M(P,S) = P(P|S)=  C(p,s) when when l S =l P M(P,S) = max over all possible when l S >l P Example PSM d1d1d1d30.9*0 d1d2 0.9*0.8 d1d2d1d30.9*0.05 d1d2d2d30.1*0.05 d1d2d1d2d30.9*0.8

20 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 20 Calculate Max over all Dynamic Programming M(p 1 p 2..p i, s 1 s 2 …s j )= Max of M(p 1 p 2..p i-1, s 1 s 2 …s j-1 ) * C(p i,s j ) M(p 1 p 2..p i, s 1 s 2 …s j-1 ) O(l P *l S ) When compatibility Matrix is sparse O(l S )

21 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 21 Match in D Average over all sequences in D

22 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 22 Spread of match If compatibility matrix is identity matrix Match = support

23 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 23 Anti-Monotone The match of a pattern P in a symbol sequence S is less than or equal to the match of any subpattern of P in S The match of a pattern P in a sequence database D is less than or equal to the match of any subpattern of P in D Can use any support based algorithm More patterns match so require efficient solution Sample based algorithms Border collapsing of ambiguous patterns

24 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 24 Chernoff Bound Given sample size=n, range R, with probability 1-  true value:    = sqrt([R 2 ln(1/  )]/2n) Distribution free More conservative Sample size : fit in memory Restricted spread : For pattern P= p 1 p 2..p L R=min (match[p i ]) for all 1  i  L Frequent Patterns min_match +  min_match -  Infrequent patterns

25 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 25 Algorithm Scan DB: O(N*min (L s *m, L s +m 2 )) Find the match of each individual symbol Take a random sample of sequences Identify borders that embrace the set of ambiguous patterns O(m Lp * |S| * Lp * n) Min_match   existing methods for association rule mining Locate the border of frequent patterns in the entire DB via border collapsing

26 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 26 Border Collapsing If memory can not hold the counters of all ambiguous patterns Probe-and-collapse : binary search Probe patterns with highest collapsing power until memory is filled If memory can hold all patterns up to the 1/x layer the space of ambiguous patterns can be narrowed to at least 1/x of the original one where x is a power of 2 If it takes a level-wise search y scans of the DB, only O(log x y) scans are necessary when the border collapsing technique is employed

27

28 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 28 Studies on Sequential Pattern Mining Concept introduction and an initial Apriori- like algorithm [AgSr95] GSP—An Apriori-based, influential mining method [SrAg96] Mining sequential patterns with constraints [GRS99] Mining long sequential pattern [JWY02]

29 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 29 Periodic Pattern Full periodic pattern ABC ABC ABC Partial periodic pattern ABC ADC ACC ABC Pattern hierarchy ABC ABC ABC DE DE DE DE ABC ABC ABC DE DE DE DE ABC ABC ABC DE DE DE DE

30 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 30 Periodic Pattern Recent Achievements Partial Periodic Pattern Asynchronous Periodic Pattern Meta Pattern InfoMiner/InfoMiner+/STAMP

31 COMP 790-090 Data Mining: Concepts, Algorithms, and Applications 31 Clustering Sequential Data CLUSEQ ApproxMAP

Download ppt "The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Sequential Pattern Mining COMP 790-90 Seminar BCB 713 Module Spring 2011."

Similar presentations

Frequent Itemset Mining Methods. The Apriori algorithm Finding frequent itemsets using candidate generation Seminal algorithm proposed by R. Agrawal and.

Frequent Itemset Mining Methods. The Apriori algorithm Finding frequent itemsets using candidate generation Seminal algorithm proposed by R. Agrawal and.
Sequential Patterns & Process Mining Current State of Research Edgar de Graaf LIACS.

Sequential Patterns & Process Mining Current State of Research Edgar de Graaf LIACS.
ICDM'06 Panel 1 Apriori Algorithm Rakesh Agrawal Ramakrishnan Srikant (description by C. Faloutsos)

ICDM'06 Panel 1 Apriori Algorithm Rakesh Agrawal Ramakrishnan Srikant (description by C. Faloutsos)
1 of 25 1 of 45 Association Rule Mining CIT366: Data Mining & Data Warehousing Instructor: Bajuna Salehe The Institute of Finance Management: Computing.

1 of 25 1 of 45 Association Rule Mining CIT366: Data Mining & Data Warehousing Instructor: Bajuna Salehe The Institute of Finance Management: Computing.
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Rakesh Agrawal Ramakrishnan Srikant

Rakesh Agrawal Ramakrishnan Srikant
IncSpan: Incremental Mining of Sequential Patterns in Large Databases Hong Cheng,Xifeng Yan,Jiawei Han University of Illinois at Urbana-Champaign.

IncSpan: Incremental Mining of Sequential Patterns in Large Databases Hong Cheng,Xifeng Yan,Jiawei Han University of Illinois at Urbana-Champaign.
Data Mining Techniques So Far: Cluster analysis K-means Classification Decision Trees J48 (C4.5) Rule-based classification JRIP (RIPPER) Logistic Regression.

Data Mining Techniques So Far: Cluster analysis K-means Classification Decision Trees J48 (C4.5) Rule-based classification JRIP (RIPPER) Logistic Regression.
Data Mining Association Analysis: Basic Concepts and Algorithms Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction.

Data Mining Association Analysis: Basic Concepts and Algorithms Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction.
Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,

Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
Data Mining: Concepts and Techniques (2nd ed.) — Chapter 5 —

Data Mining: Concepts and Techniques (2nd ed.) — Chapter 5 —
Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,

Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
Multi-dimensional Sequential Pattern Mining

Multi-dimensional Sequential Pattern Mining
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Sequential Pattern Mining

Sequential Pattern Mining
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Sequence Databases & Sequential Patterns

Sequence Databases & Sequential Patterns
1 Mining Frequent Patterns Without Candidate Generation Apriori-like algorithm suffers from long patterns or quite low minimum support thresholds. Two.

1 Mining Frequent Patterns Without Candidate Generation Apriori-like algorithm suffers from long patterns or quite low minimum support thresholds. Two.
Business Systems Intelligence: 4. Mining Association Rules Dr. Brian Mac Namee (www.comp.dit.ie/bmacnamee)www.comp.dit.ie/bmacnamee.

Business Systems Intelligence: 4. Mining Association Rules Dr. Brian Mac Namee (
1 Mining Association Rules in Large Databases Association rule mining Algorithms for scalable mining of (single-dimensional Boolean) association rules.

1 Mining Association Rules in Large Databases Association rule mining Algorithms for scalable mining of (single-dimensional Boolean) association rules.

Similar presentations

Ads by Google