1. Learning Fuzzy Association Rules and Associative Classification Rules Jianchao Han Computer Science Department California State University Dominguez Hills

2. July 19, 2006 WCCI 2006 2 Agenda Introduction Traditional Association Rules Positive and Negative Fuzzy Association Rules An Illustrative Example Positive and Negative Fuzzy Associative Classification Rules Implementation Algorithms Conclusion

3. July 19, 2006 WCCI 2006 3 Introduction Association –a relationship between data items Sales data association –If a set of items A occurs in a sale transaction, then another set of items B will likely also occurs in the same transaction Limitations –Data are described in binary attribute values –Only positive associations are pursued Solutions –Fuzzy attribute values –Negative associations

4. July 19, 2006 WCCI 2006 4 Traditional Association Rules Basket data –I={I 1, I 2, …, I m }, a set of possible items –D={t 1, t 2, …, t n }, a database of transactions –t ∈ D is represented as a binary vector, with t[I k ]=1 if t contains I k t[I k ]=0 if t does not contain I k Support of itemset – ∀ X ⊂ I, t satisfies X, if ∀ I k ∈ I, t[I k ]=1 –The support of X in D is defined as Supp(X) = |{t ∈ D| t satisfies X}| That is the number of transactions that satisfy X

5. July 19, 2006 WCCI 2006 5 Traditional Association Rules Itemset (binary) association rules –For any X, Y ⊂ I, X ⋂ Y=Ф, X  Y is an association rule if –The support of the rule Supp(X  Y) is the probability of occurrence of X ⋃ Y in D –The confidence of the rule Conf(X  Y) is the conditional probability of Y given X Mining association rules –Look for all possible associations X  Y such that Supp(X  Y) ≥ α – a given threshold and Conf(X  Y) ≥ β– another given threshold

6. July 19, 2006 WCCI 2006 6 Association Rules Mining Algorithm Two steps –Discovering all frequent itemsets that have the support ≥ α –Generating association rules Partition each frequent itemset into two parts, X and Y Test the Conf(X  Y) Level-wise algorithm –Observation: if X is a frequent itemset, its all subsets are –Test all 1-item itemsets –Test all 2-item itemsets that are the superset of frequent 1-item itemsets –Repeat until no new frequent itemsets are found

7. July 19, 2006 WCCI 2006 7 Fuzzy Association Rules Binary value is extended to the interval [0,1] Example -- Item Tomato belongs to Vegetable in some degree, say 0.7 Itemset A={A 1, A 2, …, A l } ⊂ I, where A i is a fuzzy subset of I Support of an itemset A is defined as Support of a rule A  B is Confidence of a rule A  B is

8. July 19, 2006 WCCI 2006 8 Positive vs. Negative Association Rules Positive association rules –Like A  B Negative association rules –Like ¬A  B, ¬A  ¬B, A  ¬B Different rule-interest measures exist for negative association rules, e.g. –Negative example of A  B is positive example of B  A –A  ¬B, if A ⋃ B is infrequent A ⋃ ¬B is frequent Supp(A ⋃ ¬B) – Supp(A)Supp(¬B)≥α Supp(A ⋃ ¬B)/Supp(A) ≥β

9. July 19, 2006 WCCI 2006 9 Fuzzy Positive Association Rules Simple fuzzy extension to traditional association rules A  B is a fuzzy positive association rule, if 1)A ⋂ B = Ф 2) 2) 3) 3)

10. July 19, 2006 WCCI 2006 10 Fuzzy Negative Association Rules A  ¬B is a negative association rule if 1)A ⋂ B = Ф 2)Supp(A) ≥α 3)Supp(B) ≥α 4)Supp(AB) <  5) 5) 6) 6)

11. July 19, 2006 WCCI 2006 11 Fuzzy Negative Association Rules ¬A  B is a negative association rule if 1)A ⋂ B = Ф 2)Supp(A) ≥α 3)Supp(B) ≥α 4)Supp(AB) <  5) 5) 6) 6)

12. July 19, 2006 WCCI 2006 12 Fuzzy Negative Association Rules ¬A  ¬B is a negative association rule if 1)A ⋂ B = Ф 2)Supp(A) ≥α 3)Supp(B) ≥α 4)Supp(AB) <  5) 5) 6) 6)

13. July 19, 2006 WCCI 2006 13 Algorithm for Mining both Positive and Negative Fuzzy Rules Two steps –Generating all frequent and infrequent itemsets –Extracting fuzzy association rules Positive rules are extracted from the frequent itemsets Negative rules are extracted from the infrequent itemsets

14. July 19, 2006 WCCI 2006 14 An Example Trans.i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 t1t1 1.00.70.20.01.0 t2t2 0.80.00.60.80.40.2 t3t3 0.50.80.00.8 0.0 t4t4 0.70.21.00.91.00.8 t5t5 0.4 0.00.60.80.9 t6t6 0.80.00.11.00.10.8 t7t7 0.9 0.80.21.0 t8t8 0.60.1 0.80.70.8 1-itemset2-itemsets3-itemsets itemsetsupportitemsetSupportitemsetsupport i1i1 5.7/8i 1, i 4 3.37/8i 1, i 4, i 5 1.99/8 i2i2 3.1/8i 1, i 5 4.14/8i 1, i 5, i 6 3.21/8 i3i3 2.8/8i 1, i 6 4.10/8 i4i4 5.1/8i 4, i 5 3.20/8 i5i5 5.8/8i 4, i 6 3.06/8 i6i6 5.5/8i 5, i 6 4.24/8 Transaction Database Frequent vs. Infrequent Itemsets With support threshold 40%

15. July 19, 2006 WCCI 2006 15 An Example: Positive Fuzzy Association Rules itemsetassociationsupportconfidence i 1, i 4 i1i4i4i1i1i4i4i1 3.37/859.1% 66.1% i 1, i 5 i1i5i5i1i1i5i5i1 4.14/872.6% 71.4% i 1, i 6 i1i6i6i1i1i6i6i1 4.10/871.9% 74.5% i 4, i 5 i4i5i5i4i4i5i5i4 3.20/862.7% 55.2% i 5, i 6 i5i6i6i5i5i6i6i5 4.24/873.1% 77.1% i 1, i 5, i 6 i 1, i 5  i 6 i 1, i 6  i 5 i 5, i 6  i 1 i 1  i 5, i 6 i 5  i 1, i 6 i 6  i 1, i 5 3.21/877.6% 78.3% 75.8% 56.4% 55.4% 58.4% Support threshold: 40% Confidence threshold: 75% Support threshold: 50% Confidence threshold: 70%

16. July 19, 2006 WCCI 2006 16 An Example: Negative Fuzzy Association Rules Support threshold: 25% Confidence threshold: 70% itemsetassociationsupportconfidence i 4, i 6 i4i6i6i4i4i6i6i4 2.04/8 35.8% 73.0% i6i4i4i6i6i4i4i6 2.44/8 44.4% 84.1% i4i6i6i6i4i6i6i6 0.46/8 15.9% 18.4% i 1, i 4, i 5 i 1, i 4   i 5  i 5  i 1, i 4 1.376/8 40.8% 62.5% i 1, i 5  i 4  i 4  i 1, i 5 2.146/8 51.8% 74.0% i 4, i 5   i 1  i 1  i 4, i 5 1.206/8 37.6% 52.4% i 1   i 4,  i 5  i 4,  i 5  i 1 0.184/8 3.20% 61.3% i 4   i 1,  i 5  i 1,  i 5  i 4 0.524/8 10.3% 81.9% i 5   i 1,  i 4  i 1,  i 4  i 5 0.454/8 7.80% 79.6%  i 1   i 4,  i 5  i 4,  i 5   i 1  i 4   i 1,  i 5  i 1,  i 5   i 4  i 5   i 1,  i 4  i 1,  i 4   i 5 0.116/8 5.00% 38.7% 4.00% 18.1% 5.30% 20.4%

17. July 19, 2006 WCCI 2006 17 Associative Classification Rules Associative classification rules are a special subset of association rules whose right- hand-side is restricted to the class labels. In classification, data attributes are partitioned into two categories: condition attributes and decision attributes. For simplicity, decision attributes are converted into decision attribute-value pairs that are indicated as class labels. Thus, class labels are also items in the database, but separate from condition items.

18. July 19, 2006 WCCI 2006 18 Two Constraints the left-hand-side of classification rules must be frequent itemsets of condition attributes, or the negation of infrequent conditional itemsets the class labels that appear in the right-hand-side of classification rules must also be frequent 1-itemsets

19. July 19, 2006 WCCI 2006 19 Positive Fuzzy Associative Classification Rules Let AI be an itemset, and c C be a class label. The relationship A  c is a positive fuzzy associative classification rule, if the following conditions hold: 1)A {c} is a frequent itemsets in D, Supp(A{c})/|D|  minsupp 2) A  c is confident, Conf(A  c}=Supp(A{c})/Supp(A) minconf

20. July 19, 2006 WCCI 2006 20 Negative Fuzzy Associative Classification Rules We only consider the format A  c –where A is a frequent itemset, –{c} is a frequent class label, –A{c} is infrequent A  c is a negative fuzzy associative classification rule if 1 Supp(A) ≥ minsupp; 2 Supp({c}) ≥ minsupp; 3 Supp(A{c})/|D| < minsupp; 4 Supp(¬A{c})/|D| ≥ minsupp; 5 Conf(A  c)=Supp(¬A{c})/Supp(¬A)≥minconf.

21. July 19, 2006 WCCI 2006 21 Learning Algorithm Step 1: Finding the set of frequent conditional itemsets for associative classification rules Step 2: Inducing both positive and negative fuzzy associative classification rules –add each frequent class label c to each frequent itemset X If X {c} is still frequent, then test if X  c is a positive fuzzy association rule; If X {c} is infrequent, then test if X  c is a negative fuzzy association rule. –a frequent itemset Y is partitioned into two subsets A and B, and the associations AB  c and AB  c are tested against the support threshold and confidence threshold.

22. July 19, 2006 WCCI 2006 22 Conclusion Traditional association rules Fuzzy extensions and negative rules Fuzzy associative classification rules An example Algorithms