Download presentation

Presentation is loading. Please wait.

Published byEmmett Face Modified over 2 years ago

1
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Association Rule Mining l Given a set of transactions, find rules that will predict the occurrence of an item based on the occurrences of other items in the transaction Market-Basket transactions Example of Association Rules {Diaper} {Beer}, {Milk, Bread} {Eggs,Coke}, {Beer, Bread} {Milk}, Implication means co-occurrence, not causality!

2
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 2 Definition: Frequent Itemset l Itemset –A collection of one or more items Example: {Milk, Bread, Diaper} –k-itemset An itemset that contains k items l Support count ( ) –Frequency of occurrence of an itemset –E.g. ({Milk, Bread,Diaper}) = 2 l Support –Fraction of transactions that contain an itemset –E.g. s({Milk, Bread, Diaper}) = 2/5 l Frequent Itemset –An itemset whose support is greater than or equal to a minsup threshold

3
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 3 Definition: Association Rule Example: l Association Rule –An implication expression of the form X Y, where X and Y are itemsets –Example: {Milk, Diaper} {Beer} l Rule Evaluation Metrics –Support (s) Fraction of transactions that contain both X and Y –Confidence (c) Measures how often items in Y appear in transactions that contain X

4
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 4 Association Rule Mining Task l Given a set of transactions T, the goal of association rule mining is to find all rules having –support ≥ minsup threshold –confidence ≥ minconf threshold l Brute-force approach: –List all possible association rules –Compute the support and confidence for each rule –Prune rules that fail the minsup and minconf thresholds Computationally prohibitive!

5
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 5 Mining Association Rules Example of Rules: {Milk,Diaper} {Beer} (s=0.4, c=0.67) {Milk,Beer} {Diaper} (s=0.4, c=1.0) {Diaper,Beer} {Milk} (s=0.4, c=0.67) {Beer} {Milk,Diaper} (s=0.4, c=0.67) {Diaper} {Milk,Beer} (s=0.4, c=0.5) {Milk} {Diaper,Beer} (s=0.4, c=0.5) Observations: All the above rules are binary partitions of the same itemset: {Milk, Diaper, Beer} Rules originating from the same itemset have identical support but can have different confidence Thus, we may decouple the support and confidence requirements

6
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 6 Mining Association Rules l Two-step approach: 1.Frequent Itemset Generation – Generate all itemsets whose support minsup 2.Rule Generation – Generate high confidence rules from each frequent itemset, where each rule is a binary partitioning of a frequent itemset l Frequent itemset generation is still computationally expensive

7
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 7 Frequent Itemset Generation Given d items, there are 2 d possible candidate itemsets

8
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 8 Frequent Itemset Generation l Brute-force approach: –Each itemset in the lattice is a candidate frequent itemset –Count the support of each candidate by scanning the database –Match each transaction against every candidate –Complexity ~ O(NMw) => Expensive since M = 2 d !!!

9
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 9 Frequent Itemset Generation Strategies l Reduce the number of candidates (M) –Complete search: M=2 d –Use pruning techniques to reduce M l Reduce the number of transactions (N) –Reduce size of N as the size of itemset increases –Used by DHP and vertical-based mining algorithms l Reduce the number of comparisons (NM) –Use efficient data structures to store the candidates or transactions –No need to match every candidate against every transaction

10
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 10 Reducing Number of Candidates l Apriori principle: –If an itemset is frequent, then all of its subsets must also be frequent l Apriori principle holds due to the following property of the support measure: –Support of an itemset never exceeds the support of its subsets –This is known as the anti-monotone property of support

11
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 11 Found to be Infrequent Illustrating Apriori Principle Pruned supersets

12
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 12 Illustrating Apriori Principle Items (1-itemsets) Pairs (2-itemsets) (No need to generate candidates involving Coke or Eggs) Triplets (3-itemsets) Minimum Support = 3 If every subset is considered, 6 C 1 + 6 C 2 + 6 C 3 = 41 With support-based pruning, 6 + 6 + 1 = 13

13
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 13 Frequent Itemset Mining l 2 strategies: –Breadth-first: Apriori Exploit monotonicity to the maximum –Depth-first strategy: Eclat Prune the database Do not fully exploit monotonicity

14
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 14 Apriori ACBD {} minsup=2 0 0 00 Candidates 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D

15
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 15 Apriori ACBD {} 0 1 10 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D Candidates

16
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 16 Apriori ACBD {} 0 2 20 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D Candidates

17
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 17 Apriori ACBD {} 1 2 31 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D Candidates

18
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 18 Apriori ACBD {} 2 3 42 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D Candidates

19
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 19 Apriori ACBD {} 2 4 43 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D Candidates

20
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 20 Apriori ABBCACADCDBD ACBD {} 2 4 43 Candidates minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D

21
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 21 Apriori ABBCACADCDBD ACBD {} 2 4 43 1 22322 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D

22
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 22 Apriori ACDBCD ABBCACADCDBD ACBD {} 1 22322 Candidates 2 4 43 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D

23
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 23 Apriori ACDBCD ABBCACADCDBD ACBD {} 1 22322 2 1 2 4 43 minsup=2 1B, C 2B, C 3A, C, D 4A, B, C, D 5B, D

24
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 24 Apriori Algorithm l Method: –Let k=1 –Generate frequent itemsets of length 1 –Repeat until no new frequent itemsets are identified Generate length (k+1) candidate itemsets from length k frequent itemsets Prune candidate itemsets containing subsets of length k that are infrequent Count the support of each candidate by scanning the DB Eliminate candidates that are infrequent, leaving only those that are frequent

25
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 25 Frequent Itemset Mining l 2 strategies: –Breadth-first: Apriori Exploit monotonicity to the maximum –Depth-first strategy: Eclat Prune the database Do not fully exploit monotonicity

26
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 26 Depth-First Algorithms Find all frequent itemsets minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D

27
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 27 Depth-First Algorithms Find all frequent itemsets with D without D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D Find all frequent itemsets

28
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 28 Depth-First Algorithms Find all frequent itemsets with D without D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D Find all frequent itemsets A, B, C, AC A, B, C, AC, BC

29
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 29 Depth-First Algorithms Find all frequent itemsets with D without D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D Find all frequent itemsets A, B, C, AC A, B, C, AC, BC AD, BD, CD, ACD A, B, C, AC, BC add D again

30
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 30 Depth-First Algorithms Find all frequent itemsets with D without D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D minsup=2 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D Find all frequent itemsets A, B, C, AC A, B, C, AC, BC AD, BD, CD, ACD + A, B, C, AC, BC A, B, C, AC, BC, AD, BD, CD, ACD

31
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 31 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB

32
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 32 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 DB DB[D] AC: 2

33
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 33 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 3A, 4A, B A: 2 DB DB[D] DB[CD]

34
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 34 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 3A, 4A, B A: 2 DB DB[D] DB[CD] AC: 2

35
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 35 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 DB DB[D] AC: 2

36
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 36 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 DB DB[D] AC: 2 4A4A DB[BD] A:1

37
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 37 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 DB DB[D] AC: 2

38
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 38 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 3A, C 4A, B, C 5 B, A: 2 B: 4 C: 2 DB DB[D] AC: 2 AD: 2 BD: 4 CD: 2 ACD: 2

39
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 39 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2

40
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 40 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 1 B 2 B 3A 4A, B DB[C] A: 2 B: 3

41
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 41 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 1 B 2 B 3A 4A, B DB[C] A: 2 B: 3 1 24A1 24A A: 1 DB[BC]

42
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 42 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 1 B 2 B 3A 4A, B DB[C] A: 2 B: 3

43
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 43 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 AC: 2 BC: 3 1 B 2 B 3A 4A, B DB[C] A: 2 B: 3

44
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 44 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 AC: 2 BC: 3

45
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 45 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 AC: 2 BC: 3 124A5124A5 DB[B] A:1

46
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 46 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 AC: 2 BC: 3

47
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 47 Depth-First Algorithm 1 B, C 2 B, C 3A, C, D 4A, B, C, D 5 B, D A: 2 B: 4 C: 4 D: 3 DB AD: 2 BD: 4 CD: 2 ACD: 2 AC: 2 BC: 3 Final set of frequent itemsets

48
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 48 ECLAT l For each item, store a list of transaction ids (tids) TID-list

49
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 49 ECLAT l Determine support of any k-itemset by intersecting tid-lists of two of its (k-1) subsets. l Depth-first traversal of the search lattice l Advantage: very fast support counting l Disadvantage: intermediate tid-lists may become too large for memory

50
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 50 Rule Generation l Given a frequent itemset L, find all non-empty subsets f L such that f L – f satisfies the minimum confidence requirement –If {A,B,C,D} is a frequent itemset, candidate rules: ABC D, ABD C, ACD B, BCD A, A BCD,B ACD,C ABD, D ABC AB CD,AC BD, AD BC, BC AD, BD AC, CD AB, l If |L| = k, then there are 2 k – 2 candidate association rules (ignoring L and L)

51
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 51 Rule Generation l How to efficiently generate rules from frequent itemsets? –In general, confidence does not have an anti- monotone property c(ABC D) can be larger or smaller than c(AB D) –But confidence of rules generated from the same itemset has an anti-monotone property –e.g., L = {A,B,C,D}: c(ABC D) c(AB CD) c(A BCD) Confidence is anti-monotone w.r.t. number of items on the RHS of the rule

52
© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 52 Rule Generation for Apriori Algorithm Lattice of rules Pruned Rules Low Confidence Rule

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google