Reducing Number of Candidates Apriori principle: – If an itemset is frequent, then all of its subsets must also be frequent Apriori principle holds due to the following property of the support measure: – Support of an itemset never exceeds the support of its subsets – This is known as the anti-monotone property of support

Apriori Algorithm Method: – Let k=1 – Generate frequent itemsets of length 1 – Repeat until no new frequent itemsets are identified Generate length (k+1) candidate itemsets from length k frequent itemsets Prune candidate itemsets containing subsets of length k that are infrequent Count the support of each candidate by scanning the DB Eliminate candidates that are infrequent, leaving only those that are frequent

3 Apriori L1L1 C2C2 L2L2 Candidate Generation “ Large ” Itemset Generation Large 2-itemset Generation C3C3 L3L3 Candidate Generation “ Large ” Itemset Generation Large 3-itemset Generation … Disadvantage 1: It is costly to handle a large number of candidate sets 1.Join Step 2.Prune Step Counting Step Disadvantage 2: It is tedious to repeatedly scan the database and check the candidate patterns

Frequent-pattern mining methods Max-patterns Max-pattern: frequent patterns without proper frequent super pattern – BCDE, ACD are max-patterns – BCD is not a max-pattern TidItems 10A,B,C,D,E 20B,C,D,E, 30A,C,D,F Min_sup=2

Frequent-pattern mining methods Maximal Frequent Itemset Border Infrequent Itemsets Maximal Itemsets An itemset is maximal frequent if none of its immediate supersets is frequent

A Simple Example Database TDB TidItems 10A, C, D 20B, C, E 30A, B, C, E 40B, E Here is a database that has 5 transactions: A, B, C, D, E. Let the min support = 2. Find all frequent max itemsets using Apriori algorithm. C1C1 L1L1 Itemsetsup {A}2 {B}3 {C}3 {E}3 Eliminate candidates that are infrequent 1 st scan: count support Generate frequent itemsets of length 1

A Simple Example L1L1 Itemsetsup {A}2 {B}3 {C}3 {E}3 Generate length 2 candidate itemsets from length 1 frequent itemsets C2C2 Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} generate frequent itemsets of length 2 C2C2 Itemsetsup {A, B}1 {A, C}2 {A, E}1 {B, C}2 {B, E}3 {C, E}2 2 nd scan: Count support L2L2 Itemsetsup {A, C}2 {B, C}2 {B, E}3 {C, E}2

A Simple Example Generate length 3 candidate itemsets from length 2 frequent itemsets generate frequent itemsets of length 3 3 rd scan: Count support L2L2 Itemsetsup {A, C}2 {B, C}2 {B, E}3 {C, E}2 C3C3 Itemset {B, C, E} C3C3 Itemsetsup {B, C, E}2 L3L3 Itemsetsup {B, C, E}2 The algorithm stops here

9 A Simple Example Database TDB 1 st scan C1C1 L1L1 L2L2 C2C2 C2C2 2 nd scan C3C3 L3L3 3 rd scan TidItems 10A, C, D 20B, C, E 30A, B, C, E 40B, E Itemsetsup {A}2 {B}3 {C}3 {D}1 {E}3 Itemsetsup {A}2 {B}3 {C}3 {E}3 Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} Itemsetsup {A, B}1 {A, C}2 {A, E}1 {B, C}2 {B, E}3 {C, E}2 Itemsetsup {A, C}2 {B, C}2 {B, E}3 {C, E}2 Itemset {B, C, E} Itemsetsup {B, C, E}2 Sup min = 2

Draw a graph to illustrate the result Null ABCDE CEDEABACADAEBCBDCDBE ABDABCABEACDACEADEBCDBCE BDECDE ABCDABCE ABDEACDE BCDE ABCDE Border Maximum Frequent Itemsets Green and Yellow: Frequent Itemsets

FP-growth Algorithm Scan the database once to store all essential information in a data structure called FP- tree(Frequent Pattern Tree) The FP-tree is concise and is used in directly generating large itemsets Once an FP-tree has been constructed, it uses a recursive divide-and-conquer approach to mine the frequent itemsets

FP-growth Algorithm Step 1: Deduce the ordered frequent items. For items with the same frequency, the order is given by the alphabetical order. Step 2: Construct the FP-tree from the above data Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Step 4: Determine the frequent patterns.

A Simple Example of FP-tree Problem: Find all frequent itemsets with support >= 2) TIDItems 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C

FP-growth Algorithm Step 1: Deduce the ordered frequent items. For items with the same frequency, the order is given by the alphabetical order. Step 2: Construct the FP-tree from the above data Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Step 4: Determine the frequent patterns.

A Simple Example: Step 1 ItemFrequency A4 B3 C3 D3 E2 Item Ordered Frequency A4 B3 C3 D3 E2 Threshold = 2 TIDItems 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C

A Simple Example: Step 1 TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C TIDItems 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C Item Ordered Frequency A4 B3 C3 D3 E2

FP-growth Algorithm Step 1: Deduce the ordered frequent items. For items with the same frequency, the order is given by the alphabetical order. Step 2: Construct the FP-tree from the above data Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Step 4: Determine the frequent patterns.

Step 2: Construct the FP-tree from the above data TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C ItemHead of node-link A B C D E Root A: 1 B: 1

Step 2: Construct the FP-tree from the above data TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C ItemHead of node-link A B C D E Root A: 1 B: 1 C: 1 D: 1 B: 1

Step 2: Construct the FP-tree from the above data TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C ItemHead of node-link A B C D E Root A: 2 B: 1 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1

Step 2: Construct the FP-tree from the above data TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C ItemHead of node-link A B C D E Root A: 3 B: 1 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1

Step 2: Construct the FP-tree from the above data TIDItems(Ordered) Frequent Items 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1

FP-growth Algorithm Step 1: Deduce the ordered frequent items. For items with the same frequency, the order is given by the alphabetical order. Step 2: Construct the FP-tree from the above data Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Step 4: Determine the frequent patterns.

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ E ” { } (A:1, C:1, D:1, E:1),

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ E ” { } (A:1, C:1, D:1, E:1), (A:1, D:1, E:1), Threshold = 2 ItemFrequency A2 C1 D2 E2

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ E ” { } (A:1, C:1, D:1, E:1), (A:1, D:1, E:1), Threshold = 2 ItemFrequency A2 C1 D2 E2 2

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Cond. FP-tree on “ E ” { } (A:1, C:1, D:1, E:1), (A:1, D:1, E:1), Threshold = 2 ItemFrequency A2 C1 D2 E2 ItemFrequency A2 D2 E2 ItemHead of node-link A D { } (A:1, D:1, E:1), A:2 Root D:2 conditional pattern base of “E”

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ D ” { } (A:1, C:1, D:1), Threshold = 2

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ D ” { } (A:1, C:1, D:1), Threshold = 2 (A:1, D:1),

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ D ” { } (A:1, C:1, D:1), Threshold = 2 (A:1, D:1), (B:1, C:1, D:1), ItemFrequency A2 B1 C2 D3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ D ” { } (A:1, C:1, D:1), Threshold = 2 (A:1, D:1), (B:1, C:1, D:1), ItemFrequency A2 B1 C2 D3 3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Cond. FP-tree on “ D ” Threshold = 2 ItemFrequency A2 C2 D2 ItemHead of node-link A C { } (A:1, D:1), (C:1, D:1), A:2 Root C:2 { } (A:1, C:1, D:1), (A:1, D:1), (B:1, C:1, D:1), ItemFrequency A2 B1 C2 D3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ C ” { } (A:1, B:1, C:1), Threshold = 2 (A:1, C:1), (B:1, C:1), 3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Cond. FP-tree on “ C ” Threshold = 2 ItemHead of node-link A B { } (A:1, C:1), (B:1, C:1), A:2 Root B:2 { } (A:1, B:1, C:1), (A:1, C:1), (B:1, C:1), ItemFrequency A2 B2 C3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ B ” { } (A:2, B:2), Threshold = 2 (B:1), 3

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Cond. FP-tree on “ B ” Threshold = 2 ItemHead of node-link A { } (A:2, B:2), (B:1), A:2 Root { } ItemFrequency A2 B3 (A:2, B:2), (B:1),

Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). ItemHead of node-link A B C D E Root A: 4 B: 2 C: 1 D: 1 B: 1 C: 1 D: 1 E: 1 D: 1 E: 1 C: 1 Cond. FP-tree on “ A ” { } (A:4), Threshold = 2 4 { } Root ItemFrequency A4 (A:4),

FP-growth Algorithm Step 1: Deduce the ordered frequent items. For items with the same frequency, the order is given by the alphabetical order. Step 2: Construct the FP-tree from the above data Step 3: From the FP-tree above, construct the FP-conditional tree for each item (or itemset). Step 4: Determine the frequent patterns.

Cond. FP-tree on “ A ” Cond. FP-tree on “ E ” Cond. FP-tree on “ D ” Cond. FP-tree on “ C ” Cond. FP-tree on “ B ” ItemHead of node-link A D A:2 Root D:2 ItemHead of node-link A C A:2 Root C:2 ItemHead of node-link A B A:2 Root B:2 ItemHead of node-link A A:2 Root 3

Cond. FP-tree on “ A ” Cond. FP-tree on “ E ” Cond. FP-tree on “ D ” Cond. FP-tree on “ C ” Cond. FP-tree on “ B ” ItemHead of node-link A D A:2 Root D:2 ItemHead of node-link A C A:2 Root C:2 ItemHead of node-link A B A:2 Root B:2 ItemHead of node-link A A:2 Root 3 1. Before generating this cond. tree, we generate {E} (support = 2) 1. Before generating this cond. tree, we generate {C} (support = 3) 1. Before generating this cond. tree, we generate {D} (support = 3) 1. Before generating this cond. tree, we generate {B} (support = 3) 1. Before generating this cond. tree, we generate {A} (support = 4) 2. After generating this cond. tree, we generate {A, D, E}, {A, E}, {D, E} (support = 2) 2. After generating this cond. tree, we generate {A, C}, {B, C}(support = 2) 2. After generating this cond. tree, we generate {A, D}, {C, D} (support = 2) 2. After generating this cond. tree, we generate {A, B} (support = 2) 2. After generating this cond. tree, we do not generate any itemset.

Step 4: Determine the frequent patterns Answer: According to the above step 4, we can find all frequent itemsets with support >= 2): {E}, {A,D,E}, {A,E}, {D,E}, {D}, {A,D}, {C,D}, {C}, {A,C}, {B,C}, {B}, {A,B} {A} TIDItems 1A, B 2B, C, D 3A, C, D, E 4A, D, E 5A, B, C