Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Predictive Analytics

Similar presentations


Presentation on theme: "Introduction to Predictive Analytics"— Presentation transcript:

1 Introduction to Predictive Analytics
Tiber Training, 6/12/2015

2 Short Bio B.A. from Allegheny College
M.S. from Carnegie Mellon University Internship at the Department of Veterans Affairs July 2014 – Present: Senior Consultant, IBM Global Business Services Advanced Analytics and Optimization

3 Today’s Goals: Students Will Be Able To:
Identify the purpose of Data Mining Discuss major paradigms in Data Mining Apply paradigms to business problems Discuss major considerations in model performance Evaluate model performance Discuss the general functionality of some major classification algorithms Use RapidMiner software to load data, train a classifier, and evaluate Use RapidMiner to create clusters from data

4 Agenda Conceptual Material Hands-On Activities Data Mining Foundations
Problem Types and Model Paradigms Model Performance Considerations Recognizing a Strong Model Hands-On Activities K-Nearest Neighbors Classification Decision Tree Models K-Means Clustering

5 Data Mining Foundations – Data Mining Definition
BRAINSTORM Based on exposure to literature, conversations with clients, professional publications: What is data mining?

6 Data Mining Foundations – Several Definitions
To extract insight or information from data “Extracting useful knowledge from data to solve business problems” To discover interesting relationships within data * Provost & Fawcett, “Data Science for Business”

7 CRISP-DM Framework Iterative process
“Cross Industry Standard Practice for Data Mining” Iterative process Human stays in the loop throughout iterations Understanding of both the business and the data are paramount

8 System Design BI DASHBOARDS PROCESSED STATE (Data Warehouse?)
(Operational Data Store?) (Other form?) RESULTS REPORTING REPORTING RESULTS STORAGE SOURCE SYSTEMS DATA MINING RESULTS DASHBOARD It is assumed that some action will occur AD HOC, ONE-OFF DATA MINING AUTOMATIC DATA MINING

9 Data Format Name Age Occupation Purchase? Total Spent Jane Smith 38
Designer Yes 40,000 James Doe 19 Student No N/A Sally Q. Public 50 CEO 9,000 Rows, Also known as: Records Observations NOTE: Expects inputs and outputs to be at the same level (i.e. for an individual, for a visit to the store, for a webpage view, etc) Input Fields Labels/Output Fields, Can be: Binary Categorical Numeric Columns, Also known as: Fields Attributes Dimensions

10 Data Format in Different Disciplines
BRAINSTORM Based on experience within business intelligence and data warehousing: How does this data format expectation differ from other applications? Does it pose any unique challenges?

11 Problem Types and Paradigms
SUPERVISED UNSUPERVISED CLASSIFICATION REGRESSION CLUSTERING VISUALIZATION Supervised methods assume labels/outputs Unsupervised methods lack labels/outputs

12 Problem Types: Classification
The preceding slide had $ amount of purchase. Why not use it?! Problem Types: Classification Given a set of inputs, predict the most likely value for a categorical or binary output Examples: (From preceding slide) Every new customer must record their name, age, and occupation. Jimmy G. Generic 10 Student Will this new customer make a purchase? Given the dollar amount, GPS location of the store, category of the retailer, date, and time of a purchase – is it fraudulent? Given the pulse, respiration, and blood pressure – Will the patient have a complication?

13 Problem Types: Regression
Given a set of inputs, predict the most likely value for a real-valued output Examples: (From preceding slide) Every new customer must record their name, age, and occupation. Jimmy G. Generic 10 Student IF this customer makes a purchase, how much are they likely to spend? Today’s session has minimal coverage of regression. Why?

14 Problem Types: Clustering
Given a set of inputs, with or without outputs, assign a class label to each record based upon similarity to other records Examples: ` Days as a Salesperson ` ` Average $ Value Sales/Day

15 Problem Types: Visualization
Not necessarily a problem type like the others, but: Can sometimes create new insight for existing problems Social Network Visualization, Courtesy of Wikipedia What can we say about this? Would we necessarily glean the same from clustering?

16 Problem Types and Business
BRAINSTORM Now that we’ve introduced the major problem types in data mining: Form groups and create a list of at least five business use cases for each of the problem types.

17 Model Performance Considerations
PROBLEM SCENARIO: We have clickstream data on how long a customer has viewed an item (total) and how many times they’ve clicked it We pull 10 records from the database Build a model (which type of model? Which type of problem?) X = purchase O = nonpurchase X X X # of times clicked O O O O X O X Total time spent viewing

18 Model Performance Considerations
PROBLEM SCENARIO: We have clickstream data on how long a customer has viewed an item (total) and how many times they’ve clicked it We pull 10 records from the database Build a model (which type of model? Which type of problem) X = purchase O = nonpurchase X X X NON- PURCHASE # of times clicked PURCHASE O O O O X O X Time to cash the check! Right? q Total time spent viewing SUCCESS! If Total_Time >= q THEN Purchase ELSE Nonpurchase

19 Model Performance Considerations
We deploy the system, 10 new customers Suddenly … Our classifier looks much worse With more rules, using both dimensions, we can still have reasonable accuracy We can’t have a perfect classifier using these two dimensions … Can we imagine a third? And another rule? X X X X X X X X # of times clicked p O O O O O O X O X O O O q Total time spent viewing

20 Model Performance Considerations
UNDERFIT OVERFIT Which of these did we display in the original model for our clickstream customers? What if we kept adding rules, and kept adding dimensions?

21 10-fold cross validation is the most popular
Testing a Model Cross Validation lets us choose random subsets of the data to hold out from the model building stage 10-fold cross validation is the most popular

22 10-Fold Cross Validation Utility
Choose between different modeling algorithms Tune parameters of chosen algorithm to reduce bias or variance Create an estimate of model performance

23 Evaluating classifiers
Refer to handout – Confusion Matrix and Performance Metrics Time for the big bucks! I can deliver 99% accuracy for some problems, guaranteed! How? BRAINSTORM Why not just use accuracy?

24 Receiver Operating Characteristic (ROC) Curve
IMPORTANT Points on the ROC Curve each represent a specific threshold for a classifier. Time for the big bucks! I can deliver a 0% False Positive Rate, guaranteed! How?

25 Using ROC Curves to Compare Models
Generally, models closer to the top left are best, e.g. 100% true positive rate and 0% False Positive Rate Occasionally, one model will tend toward lower false positives while another favors higher true positives Leads to: Cost sensitive or unbalanced classifiers

26 K-Nearest Neighbors Non-parametric approach
(Does not actually extract model parameters, relies on having all examples in memory) Relies on a distance function between each point and others Each nearby point “votes” on the class label

27 K-Nearest Neighbors, Continued
Parameters can be tuned: K (number of neighbors to use as a vote) Distance metric What about 1 Nearest Neighbor? What about 100 Nearest Neighbors?

28 K-Nearest Neighbors Fairly interpretable (present example cases)
Can handle multiple output categories (not just binary) Can only handle numeric inputs Nonparametric – Memory intensive and slower to score, no time to train Distance metrics begin to break down as dimensionality increases

29 Decision Tree Models Recursive algorithm approach
Uses some measure of purity Given a certain split in an input, how “pure” is the output? Information Gain, based on Entropy, is a popular measure

30 Decision Trees, continued
Test Information Gain of the output variable given a split on each of the input variables First split – Maximum Information Gain Recursively repeat for each of the nodes, which correspond to split points of original input value Stop when the leaf nodes have reached an adequate level of “purity” Handle numeric fields with cutpoints (beyond the scope of this session)

31 Decision Trees – Entropy and Information Gain
Entropy = −p(yi) ∗ log2p(yi) Example: 7/10 people purchased, 3/10 did not H(y)= -[0.7*log2(0.7)+0.3*log2(0.3)] H(y) = -[0.7* * -1.74] H(y) = .88 How does .88 correspond to the chart at right?

32 Entropy, Another View (Best Regards to CMU’s Andrew Moore, who used a similar image) PILLOW STUFFING ENTROPY: LOW PILLOW STUFFING ENTROPY: HIGH

33 Decision Trees – Entropy and Information Gain, Continued
IG(T,a) = H(T) – H(T|a) In other words, take the entropy of the outcome and subtract the entropy after you’ve split on some criteria. For example, if H(y) = .88 for purchases on the preceding slide, what is the entropy if you first break it into income >= 50,000 and income <= 50,000?

34 Information Gain – Worked Example
IG, Shopped: Entropy(Purchased) = .88 Entropy(Purchased|Shopped = Y) = .88 P(shopped = y) = 1 IG = .88 – (1*.88) = 0 IG, Income: Entropy(Purchased|Income >= 50) = .72 Entropy(Purchased|Income < 50) = .97 P(Income >= 50) = .5 P(Income < 50) = .5 IG = .88 – (.5 * * .97) IG = .88 – ( ) IG = .035 Shopped? Income >= 50k? Purchased? Y N

35 Information Gain, Worked Example
Income > 50k Purchased Y N Income > 50k Purchased N Y

36 Why devote so much time to decision trees?
Many modern implementations, including ID3 and C4.5/C5.0 Intuitive, interpretable models Fast, scalable Can handle many dimensions, mixed categorical and numeric inputs Can handle multiple valued outputs, not just binary Parametric, requires longer to train, scores very quickly Sometimes called CART – Classification and Regression Trees. Can be used for Regression, beyond the scope of this session.

37 K-Means Clustering K is a given – the number of clusters
Place K cluster centers randomly throughout the space, assigning all points to one of the K clusters Calculate the centroid of the points assigned to each cluster (in all dimensions) and move the cluster centers to this new point Calculate the distance of all points to the new centroid, repeat step 3 Repeat steps 3 and 4 until convergence (cluster centroids move very little from one iteration to the next)

38 K-Means Clustering Example
Notice that the clusters are extremely imbalanced in Iteration 1 Imagine the green points on the bottom left pulling that centroid down and left Likewise with blue

39 Conclusion Topics introduced: Definition of Data Mining
Data format – Inputs, Outputs, Dimensions Problem Types and Paradigms Model Performance Considerations, Measuring and Comparing Models K-Nearest Neighbors non-parametric classification Decision Trees parametric classification K-Means clustering, unsupervised clustering

40 Topics Not Covered Much of classical statistics (p values, ANOVA, statistical representativeness, sampling plans), much of Bayesian Preprocessing data (Binning, normalizing, deriving new variables) Segmentation for model performance IT Infrastructure Concerns Bayesian methods (Naïve Bayes, Bayes Nets) Geospatial analytics Social Network Analysis and Visualization Recommender Systems Regression, Time Series Analysis Advanced Techniques (Two-Stage Modeling, etc) Many algorithms (Neural Nets, Support Vector Machines, etc) Operationalizing Data Mining


Download ppt "Introduction to Predictive Analytics"

Similar presentations


Ads by Google