1. Richard Jensen, Chris Cornelis and Qiang Shen Dr. Chris Cornelis Ghent University, Belgium Chris.Cornelis@UGent.be Dr. Richard Jensen Aberystwyth University, UK rkj@aber.ac.uk Prof Qiang Shen Aberystwyth University, UK qqs@aber.ac.uk Hybrid Fuzzy-Rough Rule Induction and Feature Selection FUZZ-IEEE 2009

2. Richard Jensen, Chris Cornelis and Qiang Shen Outline IntroductionIntroduction Rough set theory (RST)Rough set theory (RST) Fuzzy-rough set theoryFuzzy-rough set theory Proposed method: QuickRulesProposed method: QuickRules ExperimentationExperimentation ConclusionConclusion

3. Richard Jensen, Chris Cornelis and Qiang Shen Introduction Rule induction has many advantages: e.g. understandability, accuracy, adding prior knowledgeRule induction has many advantages: e.g. understandability, accuracy, adding prior knowledge... but also limitations: scaling, dealing with noise, uncertainty...... but also limitations: scaling, dealing with noise, uncertainty... Pre-processing often usedPre-processing often used

4. Richard Jensen, Chris Cornelis and Qiang Shen Rough set theory Rx is the set of all points that are indiscernible with point x in terms of feature subset B UpperApproximation Set A LowerApproximation Equivalence class Rx

5. Richard Jensen, Chris Cornelis and Qiang Shen Discovering rules via RST Equivalence classesEquivalence classes Form the antecedent part of a ruleForm the antecedent part of a rule The lower approximation tells us if this is predictive of a given concept (certain rules)The lower approximation tells us if this is predictive of a given concept (certain rules) Typically done in one of two ways:Typically done in one of two ways: Overlaying reductsOverlaying reducts Building rules by considering individual equivalence classes (e.g. LEM2)Building rules by considering individual equivalence classes (e.g. LEM2) These require a discretization procedureThese require a discretization procedure

6. Richard Jensen, Chris Cornelis and Qiang Shen 6 Fuzzy rough sets implicatort-norm Fuzzy rough set Rough set

7. Richard Jensen, Chris Cornelis and Qiang Shen Fuzzy-rough sets Fuzzy-rough feature selectionFuzzy-rough feature selection Evaluation: function based on fuzzy-rough lower approximationEvaluation: function based on fuzzy-rough lower approximation Generation: greedy hill-climbingGeneration: greedy hill-climbing Stopping criterion: when maximal ‘goodness’ is reached (or to degree α)Stopping criterion: when maximal ‘goodness’ is reached (or to degree α) The fuzzy tolerance classes used during this process can be used to create fuzzy rulesThe fuzzy tolerance classes used during this process can be used to create fuzzy rules

8. Richard Jensen, Chris Cornelis and Qiang Shen QuickRules

9. Richard Jensen, Chris Cornelis and Qiang Shen Check

10. Richard Jensen, Chris Cornelis and Qiang Shen Experimentation 10-fold cross validation10-fold cross validation 6 fuzzy/rough set classifiers6 fuzzy/rough set classifiers 5 non fuzzy/rough set classifiers5 non fuzzy/rough set classifiers

11. Richard Jensen, Chris Cornelis and Qiang Shen Experimentation

12. Richard Jensen, Chris Cornelis and Qiang Shen Experimentation

13. Richard Jensen, Chris Cornelis and Qiang Shen Conclusion Proposed a rule induction method based on fuzzy-rough setsProposed a rule induction method based on fuzzy-rough sets Based on fuzzy-rough feature selection, using fuzzy tolerance classesBased on fuzzy-rough feature selection, using fuzzy tolerance classes Future workFuture work Post-processingPost-processing Other search mechanisms (from FS literature)Other search mechanisms (from FS literature) Other measures, e.g. VQRS positive region and dependencyOther measures, e.g. VQRS positive region and dependency

14. Richard Jensen, Chris Cornelis and Qiang Shen WEKA implementations of all fuzzy-rough classifiers and feature selectors can be downloaded from:WEKA implementations of all fuzzy-rough classifiers and feature selectors can be downloaded from:

15. Richard Jensen, Chris Cornelis and Qiang Shen