Download presentation

Presentation is loading. Please wait.

1
**Learning Rules from Data**

Olcay Taner Yıldız, Ethem Alpaydın Department of Computer Engineering Boğaziçi University

2
**Rule Induction? Derive meaningful rules from data**

Mainly used in classification problems Attribute types (Continuous, Discrete) Name Income Owns a house? Marital status Default Ali 25,000 $ Yes Married No Veli 18,000 $ No Married Yes

3
**Rules Disjunction: conjunctions are binded via OR's**

Conjunction: propositions are binded via AND's Proposition: relation on an attribute Attribute is Continuous (defines a subinterval) Attribute is Discrete (= one of the values of the attribute)

4
**How to generate rules? Rule Induction Techniques Via Trees**

C4.5Rules Directly from Data Ripper

5
**C4.5Rules (Quinlan, 93) Create decision tree using C4.5**

Convert the decision tree to a ruleset by writing each path from root to the leaves as a rule

6
**C4.5Rules q1 x1 > q1 x2 > q2 y = 0 y = 1 yes no q2 x2 : savings**

x1 : yearly-income q1 x1 > q1 x2 > q2 y = 0 y = 1 yes no OK DEFAULT q2 Rules: IF (x1 > q1) AND (x2 > q2) THEN Y = OK IF (x1 < q1) OR ((x1 > q1) AND (x2 < q2)) THEN Y = DEFAULT

7
**RIPPER (Cohen, 95) Learn rule for each class Two Steps**

Objective class is positive, other classes are negative Two Steps Initialization Learn rules one by one Immediately prune learned rule Optimization Since search is greeedy, pass k times over the rules to optimize

8
**RIPPER (Initialization)**

Split (pos, neg) into growset and pruneset Rule := grow conjunction with growset Add propositions one by one IF (x1 > q1) AND (x2 > q2) AND (x2 < q3) THEN Y = OK Rule := prune conjunction with pruneset Remove propositions IF (x1 > q1) AND (x2 < q3) THEN Y = OK If MDL < Best MDL + 64 Add conjunction Else Break

9
**Ripper (Optimization)**

Repeat K times For each rule IF (x1 > q1) AND (x2 < q3) THEN Y = OK Generate revisionrule by adding propositions IF (x1 > q1) AND (x2 < q3) AND (x1 > q3) THEN Y = OK Generate replacementrule by regrowing IF (x1 > q4) THEN Y = OK Compare current rule with revisionrule and replacementrule Take the best according to MDL

10
**Minimum Description Length**

Description Length of a Ruleset Description Length of Rules S = ||k|| + k log2 (n / k) + (n – k) log2 (k / (n – k)) Description Length of Exceptions S = log2 (|D| + 1) + fp (-log2 (e / 2C)) + (C – fp) (-log2 (1 - e / 2C)) + fn (-log2 (fn / 2U)) + (U – fn) (-log2 (1 - fn / 2U))

11
Ripper* Finding best condition is done by trying all possible split points (time-consuming) Shortcut: Linear Discriminant Analysis Split point is calculated analytically To be more robust Instances further than 3 are removed If number of examples < 20, shortcut not used

12
**Experiments 29 datasets from UCI repository are used**

10 fold cross-validation Comparison done using one-sided t test Comparison of three algorithms C4.5Rules, Ripper, Ripper* Comparison based on Error Rate Complexity of the rulesets Learning Time

13
Error Rate (I) Ripper and its variant have better performance than C4.5Rules Ripper* has similar performance compared to Ripper C4.5Rules has advantage when the number of rules are small (Exhaustive Search)

14
Error Rate (II) C4.5Rules Ripper Ripper* Total - 4 7 12 2 10 1 5 9

15
**Ruleset Complexity (I)**

Ripper and Ripper* produce significantly small number of rules compared to C4.5Rules C4.5Rules starts with an unpruned tree, which is a large amount of rule to start

16
**Ruleset Complexity (II)**

C4.5Rules Ripper Ripper* Total - 1 26 10 27 2 3 11

17
Learning Time (I) Ripper* better than Ripper, which is better than C4.5Rules C4.5Rules O(N3) Ripper O(Nlog2N) Ripper* O(NlogN)

18
Learning Time (II) C4.5Rules Ripper Ripper* Total - 2 23 25 13 27

19
Conclusion Comparison of two rule induction algorithms C4.5Rules and Ripper Proposed a shortcut in learning conditions using LDA (Ripper*) Ripper is better than C4.5Rules Ripper* improves learning time of Ripper without decreasing its performance

Similar presentations

OK

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on techniques to secure data Ppt on population census 2011 Ppt on hcl company profile Ppt on history of touchscreen technology Ppt on diode characteristics curve Download ppt on great indian mathematicians Ppt on social networking download Ppt on different kinds of birds Economics ppt on demand and supply Ppt on air pollution management