1. Case Based Reasoning Lecture 7: CBR Competence of Case-Bases

2. Outline The Utility Problem & Case Deletion A First Model of Case Competence Case Competence Categories Competence-Preserving Deletion A Second Model of Case Competence Competence Groups Reading

3. Case-Base Maintenance Case redundancy duplicates or unnecessary near neighbours in case-base may evolve during retain redundant cases may not harm decision making, but can slow down the system consult domain experts are potentially redundant cases harming performance? or may they be useful in future? Case utilisation statistics how many times is each case retrieved? if case never retrieved over a period of time may be redundant if case retrieved very frequently may indicate poor case coverage

4. The Utility Problem The utility problem occurs when cost of searching for relevant knowledge outweighs benefit of applying knowledge In CBR large case-bases mean expensive retrieval To cope with CBR utility problem delete any cases that do not affect the competence to solve problems the performance (time) i.e. lean case-bases

5. A First Case Competence Model All cases are not equal Case Competence Categories Pivotal cases contribute to competence Auxiliary cases contribute to performance Intermediate categories Spanning cases Support cases Competence-Preserving Deletion Categorise cases Order for deletion in terms of contribution to competence Smyth & Keane

6. Case Competence – The Basics Ideal measure of case coverage the set of target problems that it solves For a case c and a target problem t solves(c,t) means c solves t c is retrieved for t c can be adapted to solve t For a case c and a target problem set T coverage(c)={t T : solves(c,t)} Infeasible to generate set of all targets T space of target problems is too vast coverage( ) = { s} Target Case

7. Case Competence – The Basics Practical Measure of Case Coverage the set of cases in the case-base that it solves assumes case-base C is a representative sample of T coverage(c)={c C : solves(c,c)} the set of cases in the case-base C that c is retrieved for c can be adapted to solve Case coverage( ) = { s}

8. Case Competence – The Basics Reachability of a target problem t set of cases in C that provide a solution for it reachability(t) = {c C : solves(c,t)} Interested in reachability(c) for c C reachability(c)={c C : solves(c,c)} reachability( ) = { s} Case

9. Competence Categories: Pivotal A case is pivotal if it is reachable by no other case but itself pivotal (c) iff reachable(c) = {c} Pivotal cases are generally outliers too isolated to be solved by any other case Target problems falling within the region of a pivot can be solved only by that pivot Deletion of pivotal cases reduces competence Pivotal Auxiliary 1 4 2 3 Coverage Set

10. Competence Categories: Auxiliary A case is auxiliary if its coverage set is a subset of the coverage of one of its reachable cases auxiliary(c) iff c reachable(c) coverage (c) coverage (c) Auxiliary cases tend to lie in clusters of cases Deletion of auxiliary cases makes no difference If one is deleted then a nearby case can be used to solve any target that the deleted auxiliary could solve Pivotal Auxiliary 1 4 2 3

11. Competence Categories: Spanning Spanning Cases have coverage sets that link (span) other regions of the problem space coverage(2) spans coverage of 1 & 3 no more coverage than 1 & 3 but if 3 deleted, 2 is needed Spanning cases do not directly affect competence But if cases from linked regions deleted the spanning case may be necessary 1 2 3 Pivotal Spanning Auxiliary

12. Competence Categories: Support Support cases special kind of spanning case exist in groups each support case provides similar coverage to others in group Deletion of any case in support group does not reduce competence Deletion of all in group equivalent to deleting pivot 1 2 3

13. A Second Case Competence Model Competence Group collection of related cases Two cases belong to the same group if coverage sets overlap i.e., the two cases exhibit shared coverage Every case belongs to one and only one competence group Smyth & McKenna 1 3 4 Group 2 Group 1 Coverage(1)={1,2} Coverage(2)={1,2,3} Coverage(3)={3} Coverage(4)={4} 2

14. A Second Case Competence Model Group Coverage is proportional to size of group larger groups cover more target problems inversely proportional to density of cases denser groups cover smaller regions Case-base Coverage = Group Coverage Predicted competence = case-base coverage How does real competence relate to predicted competence?

15. Predicted vs True Competence Experiments 1000 different cases 300 chosen randomly as unseen problems Other 700 used to build case-bases True competence % accuracy on unseen problems compared with predicted competence

16. Competence Holes What is a competence hole? any uncovered region of the target space What makes a competence hole interesting? size of the hole relevance to target problems

17. Types of Competence Holes Type 1 - Lost coverage Insufficient cases within case-base. Type 2 - No lost coverage Due to domain constraints – impossible value combinations.

18. Identifying Interesting Holes Methodology Competence groups that are close may merge into a single group Missing cases are competence rich spanning cases Search for new spanning cases in the regions between nearby competence groups

19. Identifying Interesting Holes Boundary Cases Each pair of groups has pair of maximally similar cases g H and h G ( ) for G,H Each group has n-1 boundary cases corresponding to the n-1 other groups

20. Identifying Interesting Holes For each group search for new spanning cases between it and its nearest neighbour group New case is between boundary cases

21. Case Authoring Building new cases to fill the competence holes in the case-base Methodology Generate a new case from the feature values of boundary pair cases For Nominal Features Choose Most frequent value For Continuous Features Choose Mean value

22. Summary Competence Competence groups Competence holes Competence based maintenance Case deletion Case authoring Boundary cases Spanning cases between boundary cases Increasing the competence of case-bases

23. Reading Research papers B. Smyth, M.T. Keane. Remembering To Forget – A Competence-Preserving Case Deletion Policy for CBR Systems. In Proceedings of IJCAI, pp. 377-382, Canada, 1995. http://www.idi.ntnu.no/emner/it3704/lectures/papers/smyth- keane.pdf http://www.idi.ntnu.no/emner/it3704/lectures/papers/smyth- keane.pdf B. Smyth, E. McKenna. Building Compact Competent Case-bases. In proceedings of ICCBR, Munich, Germany. pp. 329-342. Springer Verlag, 1999. http://citeseer.ist.psu.edu/cache/... / smyth99building.pdf http://citeseer.ist.psu.edu/cache/... / smyth99building.pdf