Presentation on theme: "Mining Frequent Item Sets by Opportunistic Projection"— Presentation transcript:
1 Mining Frequent Item Sets by Opportunistic Projection Junqiang Liu1,4, Yunhe Pan1, Ke Wang2, Jiawei Han31 Institute of Artificial Intelligence, Zhejiang University, China2 School of Computing Science, Simon Fraser University, Canada3 Department of Computer Science, UIUC, USA4 Dept. of CS, Hangzhou University of Commerce, China
2 Outline How to discover frequent item sets Previous works Our approach: Mining Frequent Item Sets by Opportunistic ProjectionPerformance evaluationsConclusions
3 What Are Frequent Items Sets What is a frequent item set?set of items, X, that occurs together frequently in a database, i.e., support(X) ≥ a given thresholdExampleGiven support threshold 3, frequent item sets are as follows:a:3, b:3, c:4, f :4, m:3, p:3,ac:3, af :3, am:3, cf :3, cm:3, cp:3, fm:3,acf :3, acm:3, afm:3, cfm:3,acfm:3tiditems01a c d f g i m p02a b c f l m o03b f h j o04b c k p s05a c e f l m n p
4 How To Discover Frequent Item Sets Frequent item sets can be represented by a tree, which is not necessarily materailized.Mining process:a process of tree construction, accompanied bya process of projecting transaction subsets
5 Frequent Item Set Tree - FIST FIST is an ordered treeeach node: (item,weight)the following are imposeditems ordered on a path (top-down)items ordered at children (left to right)Frequent item seta path starting from the FIST rootits support is the ending node’s weightPTS - projected transaction subsetEach FIST node has its own PTS, filtered or unfilteredAll transactions that support the frequent item set represented by the node
7 Factors relate to Mining Efficiency and Scalability The FIST construction strategybreadth first v.s. depth firstThe PTS representationMemory-based representation: array-based, tree-based, vertical bitmap, horizontal bitstring, etc.Disk-based representationPTS projecting method and item counting method
8 Previous Works Research Strategy PTS Representation Projecting Method RemarksAprioribreadth firstoriginal DBon the flyRepetitive DB ScansHuge FIST for denseExp. pattern matchingTreeProjectionFPGrowthdepth firstFP-treerecursively materialize conditional DB/Fptree#of conditional FPtree in same order of mag. as # of fre. item setsH-MineH-structpartially materialize sub H-structNot most eff. for sparseCall FP-Growth for densePartition for largeDepthProjecthorizontal bitstringselective projectionMaximal fre. item setsLess efficient than array-based for sparse & largeLess efficient than tree-based for denseMAFIAvertical bitmaprecursively materialize compressions
9 Our Approach: Mining Frequent Item Sets by Opportunistic Projection Philosophy:The algorithm must adapt the construction strategy of FIST, the representation of PTS, and the methods of item counting in and projection of PTSs to the features of PTSs.Main points:Mining sparse data by projecting array-based PTSIntelligent projecting tree-based PTS for dense dataHeuristics for opportunistic projection
10 Mining sparse data by projecting array-based PTS TVLA – threaded varied length array for sparse PTSFIL– local frequent items listLQ – linked queuesarraysEach local frequent item has a FIL entry that consists of an item, a count, & a pointer.Each transaction is stored in an array that is threaded to FIL by LQ according to the heading item in the imposing order.
11 How to project TVLA for PTS Arrays (transactions) that support a node’s first child are threaded by the LQ attached to the first entry of FIL. (see previous figure)TVLA for a child node’s PTS has its own FIL and LQ.A child TVLA is unfiltered if it shares arrays with its parent, filtered otherwise.
12 How to project TVLA for PTS (cont.) Get next child’s PTS by shifting transactions threaded in the LQ currently explored (current child’s PTS)
13 Intelligent projecting tree-based PTS for dense data Tree-based Representation of dense PTS, inspired by FP-GrowthNovel projecting methods, totally differ from FP-GrowthBottom up pseudo projectionTop down pseudo projection
14 Tree-based Representation of dense PTS TTF - threaded transaction forestIL - item list: each entry consists of an item, a count, and a pointer.Forest: each node labeled by an item, associated with a weight.Each local item in PTS has an entry in the IL.Each transaction in the PTS is one path starting from a root in the forest.count is the number of transactions represented by the path.All nodes of the same item threaded by an IL entry.TTF is filtered if only local frequent items appear in TTF, otherwise unfiltered.
17 Opportunistic Projection: Observations and Heuristics Upper portion of a FIST can fit in memory.Transactions’ Number that support length k item sets decreases sharply when k is greater than 2.Heuristic 1:Grow the upper portion of a FIST breadth first.Grow the lower portion under level k depth first, whenever the reduced transaction set can be represented by a memory based structure, either TVLA or TTF.
18 Opportunistic Projection: Observations and Heuristics(2) TTF compresses well at lower levels or denser branches, where there are fewer local frequent items in PTSs and the relative support is larger.TTF is space expensive relative to TVLA if its compression ratio is less than 6-t/n ( t: number of transactions, n: number of items in a PTS).Heuristic 2:Represent PTSs by TVLA at high levels on FIST, unless the estimated compression ratio of TTF is sufficiently high.
19 Opportunistic Projection: Observations and Heuristics(3) PTSs shrink very quickly at high levels or sparse branches on FIST where filtered PTSs are usually in form of TVLA.PTSs at lower levels or dense branches shrink slowly where PTSs are represented by TTF. The creation of filtered TTF involves expensive pattern matching.Heuristic 3:Make a filtered copy for the child TVLA as long as there is free memory when projecting a parent TVLA.Delimitate the pseudo child TTF first and then make a filtered copy if it shrinks substantially sharp when projecting a parent TTF.
20 Algorithm OpportuneProject OpportuneProject(Database: D)begincreate a null root for frequent item set tree T;D’= BreadthFirst(T, D);GuidedDepthFirst(root_of_T, D’);end
21 Performance Evaluation: Efficiency on BMS-POS (sparse)
22 Performance Evaluation: Efficiency on BMS-WebView1 (sparse)
23 Performance Evaluation: Efficiency on BMS-WebView2 (sparse)
24 Performance Evaluation: Efficiency on Connect4 (dense)
25 Performance Evaluation: Efficiency on T25I20D100kN20kL5k
26 Performance Evaluation: Scalability on T25I20D1mN20kL5k
27 Performance Evaluation: Scalability on T25I20D10mN20kL5k
28 Performance Evaluation: Scalability on T25I20D100k~15mN20kL5k
29 Conclusions OpportuneProject maximize efficiency and scalability for all data features by combiningdepth first with breadth first search strategiesarray-based and tree-based representation for projected transaction subsetsunfiltered, and filetered projections
30 Acknowledgement We would like to thank Blue Martini Software, Inc. for providing us the BMS datasets!
31 References R. Agarwal, C. Aggarwal, and V. V. V. Prasad. A tree projection algorithm for generation of frequent itemsets. In Journal of Parallel and Distributed Computing (Special Issue on High Performance Data Mining), 2000. R. Agarwal, C. Aggarwal, and V. V. V. Prasad. Depth first generation of long patterns, in Proceedings of SIGKDD Conference, 2000. R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. In SIGMOD’93, Washington, D.C., May 1993. R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In VLDB'94, pp , Santiago, Chile, Sept R.J.Bayardo. Efficiently mining long patterns from databases. In SIGMOD’98, pp , Seattle, Washington, June 1998. D.Burdick, M.Calimlim, J.Gehrke. MAFIA: A maximal frequent itemset algorithm for transactional databases. In proceedings of the 17th Internation Conference on Data Engineering, Heidelberg, Germany, April 2001. Sergey Brin, Rajeev Motwani, Jeffrey D. Ullman, Shalom Tsur. Dynamic Itemset Counting and Implication Rules for Market Basket Analysis. In SIGMOD’97, Tucson, AZ, May 1997.
32 References (2) J. Han and Y. Fu. Discovery of multiple-level association rules from large databases. In VLDB'95, Zuich, Switzerland, Sept J. Han, J. Pei, and Y. Yin. Mining frequent patterns without candidate generation. In SIGMOD’2000, Dallas, TX, May 2000. D-I. Lin and Z. M. Kedem. Pincer-search: A new algorithm for discovering the maximum frequent set. In 6th Intl. Conf. Extending Database Technology, March 1998. J.S.Park, M.S.Chen, and P.S.Yu. An effective hash based algorithm for mining association rules. In Proc ACM-SIGMOD, , San Jose, CA, Feb J. Pei, J. Han, H. Lu, S. Nishio, S. Tang, and D. Yang, H-Mine: Hyper-Structure Mining of Frequent Patterns in Large Databases, Proc Int. Conf. on Data Mining (ICDM'01)}, San Jose, CA, Nov Ashok Sarasere, Edward Omiecinsky, and Shamkant Navathe. An efficient algorithm for mining association rules in large databases. In 21st Int'l Conf. on Very Large Databases (VLDB), Zurich, Switzerland, Sept
33 References (3) H.Toivonen. Sampling large databases for association rules. In Proc Int. Conf. Very Large Data Bases (VLDB’96), , Bombay, India, Sept Zijian Zheng, Ron Kohavi and Llew Mason. Real World Performance of Association Rule Algorithms. In Proc Int. Conf. on Knowledge Discovery in Databases (KDD'01), San Francisco, California, Aug