1. Lectures 5,6 MACHINE LEARNING EXPERT SYSTEMS

2. Contents Machine learning Knowledge representation Expert systems

3. INDUCTION OF DECISION TREES FROM DATA

4. Sunny Rain Overcast Outlook HumidityWind HighNormalStrongWeak Decision trees Sport=NoSport=Yes

5. Data from credit history of loan applications

7. A simplified tree… But how to do it?

8. The induction algorithm ID3

9. Partially constructed decision trees STEP 1 STEP 2

10. A heuristic problem HOW TO SELECT THE BEST PROPERTY?

11. Approximate trees HighNormal Humidity 85% Outlook 75%100% Sunny Not Sunny

12. CLASSIFICATION SYSTEMS

13. A full classification system

14. Pattern recognition Patterns: –images, personal records, driving habits, etc. Representation: –vector of features (inputs to a neural network) Pattern classification: –Classify a pattern to one of the given classes

15. > classifier < Marks > classifier < not Marks > classifier < not Marks > classifier < not Marks > classifier < Marks > classifier < not Marks Classifier training

16. Classifier application > Classifier > Marks Note: The test image does not appear in the training data

17. LEARNING IN GENERAL

18. The data and the goals We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt) The goal is to infer a general definition that will allow the learner to recognize future instances of the class

19. Knowledge representation Positive and negative examples can be represented, e.g., in predicate calculus Two positive instances of the concept of “ball” can be expressed as follows: size(obj1,small)  color(obj1,red)  shape(obj1,round) size(obj2,large)  color(obj2,red)  shape(obj2,round) The general concept of “ball” could be defined by: size(X,Y)  color(X,Z)  shape(X,round) where any sentence that unifies with this general definition represents a ball

20. A general model of the learning process

21. A set of operations Given a set of training instances, the learner must construct a generalization, heuristic rule or plan that satisfies its goals

22. The concept space Representation language and the operations define a space of potential concept definitions The learner must search this space to find the desired concept

23. Heuristic search Learning programs must commit to a direction and order of search, as well as… …to the use of available data and heuristics to search efficiently

24. PATRICK WINSTON’S PROGRAM ON LEARNING CONCEPTS

25. Examples and near misses for the concept “arch”

26. Generalization of descriptions to include multiple examples (I)

27. Generalization of descriptions to include multiple examples (II)

28. Specialization of a description to exclude a near miss so that this can’t match Starting with the original we add special constraints

30. A BRIEF HISTORY OF AI REPRESENTATIONAL SCHEMES

31. Semantic network developed by Collins & Quillian in their research on human information storage and response times

32. Network representation of properties of… …snow and ice

33. …three definitions of the word “plant” Three planes representing…

34. Intersection path between “cry” and “comfort” (Quillian 1967)

35. Case frame representation of the sentence “Sarah fixed the chair with glue.”

36. Conceptual dependency theory of four primitive conceptualizations For example, all actions are assumed to reduce to one or more of the primitive ACTs listed below:

38. “John ate the egg” “John prevented Mary from giving a book to Bill”

39. Restaurant script (Schank and Abelson 1977)

40. Restaurant script (continued)

41. FRAMES

42. A frame includes: Frame identification information Its relationship to other frames Descriptors of requirements Procedural information on use of the structure described Frame default information New instance information

43. Relationship to other frames For instance, the “hotel phone” might be a special instance of “phone”, which might be an instance of a “communication device”

44. Descriptors of requirements For instance, a chair has its seat between 20 and 40 cm from the floor, its back higher than 60 cm, etc. These requirements may be used to determine when new objects fir the stereotype defined by the frame

45. Procedural information An important feature of frames is the ability to attach procedural code to a slot

46. Frame default information These are slot values that are taken to be true when no evidence to the contrary has been found For instance, chairs have four legs, telephones are pushbutton, hotel beds are made by the staff

47. New instance information Many frame slots may be left unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving For instance, the color of the bedspread may be left unspecified

48. Part of a frame description of a hotel room “Specialization” indicates a pointer to a superclass

49. Spatial frame for viewing a cube (Minsky 1975)

50. CONCEPTUAL GRAPHS: A NETWORK LANGUAGE

51. Conceptual relations of different arities

52. “Mary gave John the book” “A dog named emma is brown”

53. Examples of restriction…

54. …join, and simplify operations

55. Inheritance in conceptual graphs

56. “Tom believes that Jane likes pizza” This example shows the use of a propositional concept

58. RULE BASED EXPERT SYSTEMS

59. Architecture of a typical expert system for a particular problem domain

60. Guidelines to determine whether a problem is appropriate for expert system solution (1) The need for the solution justifies the cost and effort of building an expert system Human expertise is not available in all situations where it is needed The problem may be solved using symbolic reasoning

61. Guidelines to determine whether a problem is appropriate for expert system solution (2) The problem domain is well structured and does not require commonsense reasoning The problem may not be solved using traditional computing methods Cooperative and articulate experts exist The problem is of proper size and scope

62. Reasoning with a typical expert system

63. The role of mental or conceptual models in problem solving

64. A small system for analysis of automotive problems

65. The and/or graph searched in the car diagnosis example

66. The production system at the start of a consultation in the car diagnostic example Imagine that we want to get information about spark plugs

67. The production system after Rule 1 has fired

68. The system after Rule 4 has fired Note the stack-based approach to goal reduction

69. The following dialogue begins with the computer asking the user about the goals present in working memory

70. CASE BASED REASONING

71. Case based reasoners Share a common structure For each new problem they –Retrieve appropriate cases from memory –Modify a retrieved case so that it will apply to the current situation –Apply the transformed case –Save the solution, with a record of success or failure, for future use

72. Possible preference heuristics to help organize the storage and retrieval of cases (1) Goal-directed: Organize cases, at least in part, by goal descriptions; Retrieve cases that have the same goal as the current situation Salient-feature: Prefer cases that match the most important features or those matching the largest number of important features Recency: Prefer cases used most recently

73. Possible preference heuristics to help organize the storage and retrieval of cases (2) Specify: Look for as exact as possible matches of features before considering more general matches Frequency: Check first the most frequently matched cases Ease of adaptation: Use first cases most easily adapted to the current situation

74. Transformational analogy

75. Advantages of rule based approach

76. Disadvantages

77. Advantages of case based reasoning

78. Disadvantages

79. Combination of rule based and case based systems

80. 3 Robotic Planning

81. ROBOTIC PLANNING

82. The blocks world

83. Predicate based representation

84. A number of truth relations or rules for robot’s performance

85. Portion of the search space

86. Goal state for the blocks world

87. Descriptor triples Preconditions (P): Conditions the world must meet for an operator to be applied Add List (A): Additions to the state description that are a result of applying the operator Delete List (D): Items that are removed from a state description to create the new state when the operator is applied

88. Operators as triples of descriptions

89. A simple tree showing condition action rules

90. THANK YOU