Lecture 3: CBR Case-Base Indexing Case Based Reasoning Lecture 3: CBR Case-Base Indexing

Outline Indexing CBR case knowledge Why might we want an index? Decision tree indexes C4.5 algorithm Summary

Why might we want an index? Efficiency Similarity matching is computationally expensive for large case-bases Similarity matching can be computationally expensive for complex case representations Relevancy of cases for similarity matching some features of new problem may make certain cases irrelevant despite being very similar Cases are pre-selected from case-base Similarity matching is applied to subset of cases

What to index? Case Features are: Indexed Unindexed Unindexed features Client Ref #: 64 Client Name: John Smith Address: 39 Union Street Tel: 01224 665544 Photo: Age: 37 Occupation: IT Analyst Income: £ 20000 … Case Features are: Indexed Unindexed Unindexed features Indexed features

Indexed vs Unindexed Features Indexed features are: used for retrieval are predictive of the case’s solution Unindexed feature are: not used for retrieval not predictive of the case’s solution provide valuable contextual information and lessons learned

Playing Tennis Example (case-base) Outlook Temperature Humidity Windy Play Sunny Hot High False No True Cloudy Yes Rainy Mild Cool Normal

Decision Tree (Index) for Playing Tennis outlook sunny rainy cloudy Yes windy humidity normal high true false No Yes No Yes

Choosing the Root Attribute outlook humidity temperature windy sunny rainy high low hot cold true false cloudy mild Yes No Yes Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Which attribute is best for the root of the tree? - the one that gives the best information gain - in this case outlook (as we are going to see)

Building Decision Trees – C4.5 Algorithm Based on the Information Theory (Shannon 1948) Divide and conquer strategy Choose attribute for root node Create branch for each value of that attribute Split cases according to branches Repeat process for each branch until all cases in the branch have the same class Assumption: simplest tree which classifies the cases is best

Entropy of a set of cases Playing Tennis Example: S is the set of 14 cases We want to classify the cases according to the values of “Play”, i.e., Yes and No in this example. the proportion of “Yes” cases is 9 out of 14: 9/14 = 0.64 the proportion of “No” cases is 5 out of 14: 5/14 = 0.36 The Entropy measures the impurity of S Entropy (S) = - 0.64(log2 0.64) – 0.36(log2 0.36) = -0.64(-0.644)-0.36(-1.474) = 0.41+0.53 = 0.94 Outlook Temperature Humidity Windy Play Sunny Hot High False No Cloudy Yes … “No” case “Yes” case 14 cases

Entropy of a set of cases S is a set of cases A is a feature Play in the example {S1 ... Si … Sn} are the partitions of S according to values of A Yes and No in the example {P1 ... Pi … Pn} are the proportions of {S1 ... Si … Sn} in S

Gain of an attribute Calculate Gain (S, A) for each attribute A expected reduction in entropy due to sorting on A Choose the attribute with highest gain as root of tree Gain (S, A) = Entropy(S) – Expectation(A) {S1, ..., Si, …, Sn} = partitions of S according to values of attribute A n = number of attributes A |Si| = number of cases in the partition Si |S| = total number of cases in S

Which attribute is root? If Outlook is made root of the tree There are 3 partitions of the cases S1 for Sunny, S2 for Cloudy, S3 for Rainy S1(Sunny)= {cases 1,2,8,9,11} |S1| = 5 In these 5 cases values for Play are 3 No and 2 Yes Entropy(S1) = - 2/5 (log2 2/5) – 3/5(log2 3/5) = 0.97 Similarly Entropy(S2)= 0 Entropy(S3)= 0.97 Outlook Temperature Humidity Windy Play Sunny Hot High False No True Cloudy Yes Rainy Mild Cool Normal

Choosing the Root Attribute outlook humidity temperature windy sunny rainy high low hot cold true false cloudy mild Yes No Yes Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Which attribute is best for the root of the tree? - the one that gives the best information gain - in this case outlook (as we are going to see)

Which attribute is root? Gain(S, Outlook) = Entropy(S) – Expectation(Outlook) = Gain(S, Outlook) = 0.94 – [5/14 * 0.97 + 4/14 * 0 + 5/14 * 0.97] = 0.247 Similarly Gain(S, Temperature)= 0.059 Gain(S, Humidity)= 0.051 Gain(S, Windy)= 0.048 Gain(S, Outlook) is the highest gain Outlook should be the root of the decision tree (index)

Repeat for Sunny Node outlook outlook sunny rainy cloudy sunny rainy Yes ? temperature Yes ? windy sunny rainy hot mild cold cloudy false true No Yes No Yes Yes ? humidity Yes No Yes No high normal No Yes

Repeat for Rainy Node outlook sunny rainy cloudy Yes Mild High False Yes Cool Normal False Yes Cool Normal True No Mild Normal False Yes Mild High True No humidity high normal No Yes

Decision Tree (Index) for Playing Tennis outlook sunny rainy cloudy Yes windy humidity normal high true false No Yes No Yes

Case Retrieval via DTree Index Typical implementation e.g. Case-Base indexed using a decision-tree Cases are “stored” in the index leaves… DTree created from cases Automated indexing of case-base

Summary Decision tree is built from cases Decision tree is often used for problem-solving In CBR, decision tree is used to partition cases Similarity matching is applied to cases in leaf node Indexing pre-selects relevant cases for k-NN retrieval BRING CALCULATOR on MONDAY