EFFICIENT ITEMSET EXTRACTION USING IMINE INDEX By By U.P.Pushpavalli U.P.Pushpavalli II Year ME(CSE) II Year ME(CSE)

OBJECTIVE The main objective is to provide an index support for frequent itemset mining. To provide a compact and complete structure for item set extraction. Implemented by FP based and LCM based algorithms.

  A frequent itemset is an itemset whose support is ≥ minsup   Support: For rule of form A=>B, Support refers to percentage of transaction in D that contain AUB.   Confidence: For rule of form A=>B, confidence is the conditional probability that B is true when A is known to be true.   support(LHS U RHS) / support(LHS)

Existing-Apriori Algorithm Uses database scan and pattern matching to collect counts for the candidate itemsets Any subset of a frequent itemset must be frequent. Any subset of a frequent itemset must be frequent.

Apriori –Example TIDItems 10a, c, d 20b, c, e 30a, b, c, e 40b, e Min_sup=2 ItemsetSup a2 b3 c3 d1 e3 Database D 1-candidates Scan D ItemsetSup a2 b3 c3 e3 Freq 1-itemsets Itemset ab ac ae bc be ce 2-candidates ItemsetSup ab1 ac2 ae1 bc2 be3 ce2 Counting Scan D ItemsetSup ac2 bc2 be3 ce2 Freq 2-itemsets Itemset bce 3-candidates ItemsetSup bce2 Freq 3-itemsets Scan D

Bottleneck of Apriori: Huge candidate sets Multiple scans of database

Mining Frequent Patterns- Without Candidate Generation Large database is compressed into a compact, Frequent-Pattern tree (FP-tree) structure Highly condensed, but complete for frequent pattern mining Avoids costly database scans Divide-and-conquer methodology Avoids candidate generation

FP-tree {} f:4c:1 b:1 p:1 b:1c:3 a:3 b:1m:2 p:2m:1 Header Table Item frequency head f4 c4 a3 b3 m3 p3 min_support = 3 TIDItems bought (ordered) frequent items 100{f, a, c, d, g, i, m, p}{f, c, a, m, p} 200{a, b, c, f, l, m, o}{f, c, a, b, m} 300 {b, f, h, j, o}{f, b} 400 {b, c, k, s, p}{c, b, p} 500 {a, f, c, e, l, p, m, n}{f, c, a, m, p}

Drawbacks:   Requires two database scans   Rebuilding tree for every support count   Memory utilization high

IMINE-PROPOSED SYSTEM IMINE-PROPOSED SYSTEM Covering index. Covering index. No constraints are enforced during the index creation phase. Efficiently exploited by various item set extraction algorithms. Efficiently exploited by various item set extraction algorithms. Physical organization supports efficient data access during item set extraction. Physical organization supports efficient data access during item set extraction. Support item set extraction in large data sets. Support item set extraction in large data sets.

Creating I-Tree based on the FP-tree data structure Creating I-Btree based on the B+Tree structure Extraction task – Reading selected I-Tree portions. Data access methods frequent-item,Support and Item- based projection Designing IMine Physical organization to reduce I/O Item set mining- Implementing FP-based and LCM algorithms Performance evaluation System Flow Diagram

MODULES: Implementation of I-tree I-Btree IMine Data Access Methods IMine Physical Organization Item set mining using FP-based and LCM algorithms

Index Structure Index Structure Characterized by 2 components and provide 2 levels of indexing I-Tree (Itemset-Tree) I-Tree (Itemset-Tree) Prefix-tree based on FP-tree data structure. Scans the database once. I-Btree (Item-Btree) I-Btree (Item-Btree) Reading selected I-Tree portions during extraction.

IMine Parent pointer First child pointer Right brother pointer I-Tree

IMine I-Btree

I-TREE I-TREE I-Tree layers: Top layer Top layer Very frequently accessed during the mining process. Nodes with high support are stored. Middle layer Middle layer Quite frequently accessed during the mining process. Bottom layer Bottom layer Rarely accessed during the mining process Nodes with unitary support are stored.

Physical organization : Minimize the cost of reading the data needed for the current extraction process Minimize the cost of reading the data needed for the current extraction process Correlation types: Correlation types: Intratransaction correlation Intratransaction correlation I-Tree layers I-Tree layers Intertransaction correlation I-Tree path correlation I-Tree path correlation

I/O analysis for index data access: Through I-Btree, block 3 is loaded in the buffer cache. Through I-Btree, block 3 is loaded in the buffer cache. Following the node parent, block 1 is loaded [p:3]→[d:5] →[h:7] →[e:7] →[b:10] is in memory Following the node parent, block 1 is loaded [p:3]→[d:5] →[h:7] →[e:7] →[b:10] is in memory If the 2 blocks are still in the buffer cache, reading other prefix path does not require additional disk reads If the 2 blocks are still in the buffer cache, reading other prefix path does not require additional disk reads

IMine data access method: Frequent-item based projection Frequent-item based projection Support projection-based algorithm FP-growth FP-growth Support-based projection Support-based projection Support level-based and array-based algorithm Apriori and LCM v.2 Apriori and LCM v.2 Item-based projection Item-based projection Load all transactions

Loading frequent-item based projected DB: Ex: item p appears in 2 nodes [p:3], [p:2] Ex: item p appears in 2 nodes [p:3], [p:2] Starting from I-Btree and reading 2 Starting from I-Btree and reading 2 prefix path for p prefix path for p[p:3→d:5→h:7→e:7→b:10][p:2→i:2→h:3→e:3]

Loading Support-based projected DB: Given the I-Tree,subpaths between the I-Tree roots and the first node with an infrequent item. Given the I-Tree,subpaths between the I-Tree roots and the first node with an infrequent item. Reads a node subtree by means of a top-down depth-first I-Tree visit exploiting both the node child and brother pointers. Reads a node subtree by means of a top-down depth-first I-Tree visit exploiting both the node child and brother pointers.

Item Set Mining Step1: The needed index data is loaded The needed index data is loadedStep2: Item set extraction takes place on loaded data Item set extraction takes place on loaded data

I-MINE

I_BTree

LCM

IMINE -Execution Time

IMINE-Memory Usage

Software Specification Operating system : Windows XP/Vista Language : JDK and above Back End : SQLServer2000

Conclusion Provide a complete and compact representation of transactional data Supports different algorithmic approaches to item set extraction Performance better than the existing FP-growth, LCM v.2 algorithms.

Future Enhancements Compact structure suitable for different data distributions Incremental update of the index

Thank You