BOAT - Optimistic Decision Tree Construction Gehrke, J. Ganti V., Ramakrishnan R., Loh, W.

Problem Efficient construction of decision trees As few passes of the database as possible Sample of dataset to give insight to the full database

Motivation Standard decision tree construction is not sufficient –For tree of height h, need h passes through entire database –To include new data, must rebuild the tree –For large databases, this is not feasible Need fast, scalable method

Intuition Begin with sample of data –Build decision tree on the sample –For numeric data, use a confidence interval for split Make limited passes of full data to both verify sampled tree and construct full tree –Only data that falls in confidence interval needs be rescanned to determine how to propagate

Criteria selection Use of impurity functions to generate the attribute to split on –Entropy, gini, index of correlation –Calculated for sample, could be wrong in the full dataset Minimize the impurity function in attribute selection and confidence interval

Confidence Interval Construct T trees If at node n, the splitting attribute is not the same in all trees, discard n and its subtree in all trees For categorical data, if the splitting subset is not identical in all trees, remove node n and its subtree in all trees

Confidence Interval Confidence interval on numeric attributes determined by the range of split points on the T trees Exact split point is likely to be between the min and max of the values of the split points on the T trees.

Verification Verifying predictions –Use a lower bound for the impurity function to determine if confidence interval and splitting attribute are correct –Discard node and its subtree completely if incorrect Rerun algorithm on any set of data related to a discarded node

Invalidated Predictions Discarded top nodes would result in resampling of entire database –No savings on full scans –Doesn’t usually happen –Basic probability distribution likely captured by sample –Error in the detail (low) level

Dynamic Environments No need to frequently rebuild the decision trees Store the confidence intervals Only need rebuild of tree if underlying probability distribution changes

Experimental Results Used tuple sample size Grew 20 trees on tuples drawn from pool Datasets of 1.5 million tuples Outperforms brute-force method by a factor of 2 or 3

Experimental Results Robust to noise –Noise affects detail-level probability distribution –Affected the lower levels, requiring rescans of small amounts of data Dynamic updating data –BOAT is much faster than brute-force

Weak Points May not be as useful on complex probability distributions –Failure at high level of tree means that most of the tree is discarded Hypotheses generate as simple as regular decision trees –Simply a way to speed generation

Suggested Improvements Clustering to give a better sample to draw from –Groups of datapoints with a measure of frequency of occurance –Would give better samples of the data and its underlying probability distribution

Suggested Improvements Extremely large datasets –For TB+ datasets, even two or three passes of DB may be too many –Use MCMC to draw many different samples –Estimate probability density function by resampling Would not guarantee tree accuracy

Conclusion Effective way to build scalable decision trees Much faster than the standard method Useful in large datasets