Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN 978-1-84480-891-5 © 2010 Cengage Learning Chapter Two Principles of data mining.

Similar presentations


Presentation on theme: "Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN 978-1-84480-891-5 © 2010 Cengage Learning Chapter Two Principles of data mining."— Presentation transcript:

1 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Chapter Two Principles of data mining

2 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Chapter Overview The process of data mining Approaches of data mining Categories of data mining problems Information patterns to be discovered Overview of data mining solutions Importance of evaluation Undertaking a data mining task in Weka Review of basic concepts in statistics and probability

3 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Process Preparing Input Data Mining Patterns Post-processing Patterns Input Data Output Patterns A data mining stage Flow of control from one stage to the next stage Flow of control from one stage to the previous stage Repetition of the tasks at one stage

4 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Process Preparation Formatted Data set Formatted Data set Target Data set Pre-Processed Data set Original Data sets Collected Data set Integrating data Getting necessary data details Selecting relevant features Selecting relevant records Data cleaning Deal with unknown data Data transformation Formatting data into acceptable form by the mining tool

5 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Process Mining –Determining data mining tasks –Assigning roles for data for certain tasks –Selecting data mining solution(s) to each task –Setting necessary parameters for the solution –Collecting result patterns Formatted Data set Formatted Data set Solution 3 (w 1, w 2, …, w m ) Solution 2 (t 1, t 2, …, t r ) Solution 1 (p 1, p 2, …, p n ) Patterns Mining solutions Parameter settings

6 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Process Post-processing –Pattern evaluation –Pattern selection –Pattern interpretation Patterns Evaluation criteria reject Valid Patterns Valid Patterns Selection criteria Selected Patterns accept Pattern Interpretation Knowledge learnt

7 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Process Roles of participants in data mining –Participants include: Data miners / data analysts: main participant of a DM project Domain expert: main collaborators of DM project Decision makers: clients of a DM project –Risk of human bias in the discovery process –Important roles of domain expert Pattern interpretation (for usefulness) Pattern evaluation (for significance) Mining options (for suitable tasks, limited) Advisory on data pre-processing (for suitable operations, limited) –Balancing the strength of human and machine

8 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Approaches Hypothesis testing approach –Top-down lead by a hypothesis statement –Procedure: 1.Forming a hypothesis statement 2.Collecting and selecting data of relevance 3.Conducting data analysis and collecting patterns 4.Interpreting the patterns to accept/reject the hypothesis Discovery approach –Bottom-up without a hypothesis in mind –Procedure: 1.Collecting and preparing data of interest 2.Conducting data analysis and discovering possible patterns 3.Evaluating the importance and interestingness

9 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Approaches Discovery approach (cont’d) –Directed discovery (supervised learning): Certain aspects of the outcome, i.e. the goal, of the discovery have been specified. The discovery is to find those patterns satisfying the goal. e.g. patterns relating to the outcome of a class variable –Undirected discovery (unsupervised learning): There is no specification of the goal of the discovery. The discovery is to find those patterns of some kind of significance. e.g. associative links among some attribute values

10 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns Classification –Construct a classification model to determine the class of a given record Example Data Set Model Construction Method Classification Model Classification Model (a) Model Development Phase (b) Model Use Phase Unseen Data Record with undetermined class Data Record with the determined class

11 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns Various forms of classification models Instance space Neural networkDecision tree List of ordered classification rules Function ( linear regression ) Many more …

12 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns Cluster detection –Measure similarity among data objects and group them into clusters accordingly Cluster Memberships of Data Points Input data points Clustering Method

13 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns Forms of clustering results Clusters of various shapes Eclipse shaped clusters Hierarchical clustering results

14 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns Association rule mining –Discover significant relationships between data objects Association Mining Method X  YX  Y –Between values, e.g. Apple  Coke –Between categories of values, e.g. Food  Magazine –Between values of attributes, e.g. Married:yes  OwnHouse:yes –Over time period, e.g. year 1: Database  year 2: Data Mining Various associations

15 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining: Problems & Patterns An example Classification model?Clusters?Association rules?

16 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Solutions: An Overview Classification solutions –Decision tree e.g. ID3 –k nearest neighbour (kNN) e.g. PEBLS –Rulese.g. Sequential Cover –Bayesian theoreme.g. Naïve Bayes –Artificial neural network Clustering Solutions –Partition-basedmethodse.g. K-means –Hierarchical methodse.g. agglomeration –Density-based methodse.g. DBScan –Model-based methodse.g. Expectation-Maximisation –Graph-based methodse.g. Chameleon

17 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining Solutions: An Overview Association rule solutions –Greedy methods e.g. Apriori –Graph-based methods e.g. FP-Growth –Methods for various associations Boolean associations Generalised associations (multi-level associations) Quantitative associations (multidimensional associations) Sequential associations (sequential patterns) Since one type of data mining problems can be transformed to another type of data mining problems, some solutions for one type can also be applied to another type.

18 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Evaluation of Patterns Importance of evaluating result patterns –Classification model must be accurate enough to be creditable –Clusters must genuinely exist –Association rules must have enough strengths to be believed –Data descriptions must be general enough to cover a large part of the data set How do we evaluate the discovered patterns ?

19 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Evaluation of Patterns Possible measures of interestingness –Objective measures based on data and pattern Conciseness of pattern, e.g. minimum description length Coverage, e.g. coverage for classification rules Reliability, e.g. accuracy of a classification model Peculiarity, e.g. measures of difference from the norm Diversity, e.g. tendency of clusters –Subjective measures based on domain knowledge Novelty Surprisingness Usefulness Applicability

20 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Evaluation of Patterns Commonly used measures –Accuracy rate or error rate for classification models True positive False positive False negative (see section 6.5.1) –Quality of clusters Quality of a cluster Overall quality of all clusters (see section 4.5.1) –Strengths of associations Support Confidence Lift (see section and 8.6)

21 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Associate Tab page Data Mining in Weka Explorer The roadmap Preprocess Tab page (1) Cluster Tab page (2) Classify Tab page Tree Visualiser window (3)

22 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining in Weka Explorer Preprocess Open data set from different sources Generate random data setSave data set into a file Display & edit dataAttribute display, selection & removal from the opened data set Selected attribute summary Selected attribute visualisation Visualise all attributes Filters for pre-processing Feedback messages Data summary

23 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining in Weka Explorer Classify (as an example) Method selection & parameter setting Test option setting Task list. Menu of options available with right click. Result display window

24 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining in Weka Explorer Classify (as an example) Method ListSelecting a specific method Selecting & Changing parameters

25 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Data Mining in Weka Explorer Visualisation An Example Decision Tree Scatter plot of data object of different classes

26 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Where probability and statistics used? –Patterns found from data are probabilistic in nature –Used in various measures of evaluation, e.g. confidence measure of association rules –Used in data exploration stage for better understanding, e.g. maximum, minimum, mean, variance, skewness –Used during the mining process to assist the discovery of patterns, e.g. information gain for decision tree induction –Used as a part of patterns, e.g. naïve Bayes, Gaussian mixture model –Used in comparison of patterns, e.g. classification model with significantly better accuracy

27 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Probability and conditional probability –Probability of event P(E) and its meanings when: P(E) = 0, P(E) = 1 and 0 < P(E) < 1 –Probabilities of multiple events: P(E and F), P(E or F) = P(E) + P(F) – P(E and F) –Mutually exclusive events: P(E and F) = 0 and P(E and F) = P(E) + P(F) –Conditional probability of event E given event F: P(E|F) = P(E and F)/P(F) –Independent events: P(E and F) = P(E)  P(F), and P(E|F) = P(E)

28 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Probability & conditional probability (example)

29 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Probability distribution of random variables –Discrete random variable –Continuous random variable P(X = x) P(a  X < b) 68% 95%

30 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Basic Statistics –Sample mean, median and mode –Variance and standard deviation –Skewness

31 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Confidence interval estimate –Sample mean is only an estimate of the true mean for the data population. –Central limit theorem: sample means follows a normal distribution that: a.The mean is the true population mean  X b.The standard deviation is –Based on the central limit theorem and using the sample standard deviation to replace the true one, the following expression is used to estimate the interval for the true mean at confidence level of 1- 

32 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Confidence interval estimate (example) The interval is estimated as [21.347, ] at confidence level of 95% For this data set, n = 12, age = 26 and s age = At confidence level of 95%, i.e. 1 -  = 0.95 and  /2 = 0.025, n – 1 = 11, and therefore, t = The interval estimate is:

33 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Probability & Statistics: A Brief Review Hypothesis testing –As an introduction to statistical inference and statistic significance. –Procedure: a.Forming null and alternative hypotheses b.Deciding the level of significance p c.Determining a test statistic and calculating its value d.Comparing the calculated value against known value and deciding if the null hypothesis should be rejected

34 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Hypothesis testing (example) –Assuming  age = 25 –Hypotheses: Null: Alternative: –Calculating the statistic t as: Probability & Statistics: A Brief Review Less than t = for p/2 = and n – 1 = 11. –Conclusion: null hypothesis is not rejected, i.e. the difference between the sample mean and the population mean is insignificant.

35 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning Chapter Summary The data mining process involves preparation of data, mining of patterns and post-processing of the patterns. Top-down and bottom-up approaches are both useful. The discovery approach can be directed or undirected. Three main streams of data mining tasks and various forms of patterns and models are introduced. Specific solutions are required for specific types of problems The importance of evaluation of patterns must be appreciated. Normal procedure of conducting data mining in Weka is explained Some important basic concepts in probability and statistics are reviewed.

36 Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN © 2010 Cengage Learning References Read Chapter 2 of Data Mining Techniques and Applications Useful further references Han, J. and Kamber, M. (2006), Data Mining: Concepts and Techniques, 2 nd Edition, Morgan Kaufmann Publishers, Chapter 1 Berry, M. J. A. and Linoff, G. (2004), Data Mining Techniques: For Marketing, Sales and Customer Relationship Management, 2 nd ed. Wiley Computer Publishing, Chapters 1 – 2


Download ppt "Data Mining Techniques and Applications, 1 st edition Hongbo Du ISBN 978-1-84480-891-5 © 2010 Cengage Learning Chapter Two Principles of data mining."

Similar presentations


Ads by Google