1. Chapter 1: Introduction to Expert Systems

2. Contents Introduction Knowledge Representing
General Methods of Inference

3. Objectives Learn the meaning of an expert system
Understand the problem domain and knowledge domain
Learn the advantages of an expert system
Understand the stages in the development of an expert system
Examine the general characteristics of an expert system
Examine earlier expert systems which have given rise to today’s knowledge-based systems
Explore the applications of expert systems in use today
Examine the structure of a rule-based expert system

4. What is an expert system?
“An expert system is a computer system that emulates, or acts in all respects, with the decision-making capabilities of a human expert.”

5. Fig 1.1 Areas of Artificial Intelligence

6. Expert system technology may include:
Special expert system languages – CLIPS
Programs
Hardware designed to facilitate the implementation of those systems
Expert System Main Components:
Knowledge base – obtainable from books, magazines, knowledgeable persons, etc.
Inference engine – draws conclusions from the knowledge base

7. Figure 1.2 Basic Functions of Expert Systems

8. Problem Domain vs. Knowledge Domain
An expert’s knowledge is specific to one problem domain – medicine, finance, science, engineering, etc.
The expert’s knowledge about solving specific problems is called the knowledge domain.
The problem domain is always a superset of the knowledge domain.

9. Figure 1.3 Problem and Knowledge Domain Relationship

10. Advantages of Expert Systems
Increased availability
Reduced cost
Reduced danger
Performance
Multiple expertise
Increased reliability
Explanation
Fast response
Steady, unemotional, and complete responses at all times
Intelligent tutor
Intelligent database

11. 2. Knowledge Representing
The knowledge of an expert system can be represented in a number of ways, including
IF-THEN rules:
IF you are hungry THEN eat

12. Knowledge Engineering
The process of building an expert system:
The knowledge engineer establishes a dialog with the human expert to elicit knowledge.
The knowledge engineer codes the knowledge explicitly in the knowledge base.
The expert evaluates the expert system and gives a critique to the knowledge engineer.

13. Fig 1.4 Development of an Expert System

14. The Role of AI
An algorithm is an ideal solution guaranteed to yield a solution in a finite amount of time.
When an algorithm is not available or is insufficient, we rely on artificial intelligence (AI).
Expert system relies on inference – we accept a “reasonable solution.”

15. Uncertainty
Both human experts and expert systems must be able to deal with uncertainty.
It is easier to program expert systems with shallow knowledge than with deep knowledge.
Shallow knowledge – based on empirical and heuristic knowledge.
Deep knowledge – based on basic structure, function, and behavior of objects.

16. Limitations of Expert Systems
Typical expert systems cannot generalize through analogy to reason about new situations in the way people can.
A knowledge acquisition bottleneck results from the time-consuming and labor intensive task of building an expert system.
Early Expert Systems
DENDRAL – used in chemical mass spectroscopy to identify chemical constituents
MYCIN – medical diagnosis of illness
DIPMETER – geological data analysis for oil
PROSPECTOR – geological data analysis for minerals
XCON/R1 – configuring computer systems

17. Table 1.1 Broad Classes of Expert Systems

18. Problems with Algorithmic Solutions
Conventional computer programs generally solve problems having algorithmic solutions.
Algorithmic languages include C, Java, and C#.
Classical AI languages include LISP and PROLOG.

19. Considerations for Building Expert Systems
Can the problem be solved effectively by conventional programming?
Is there a need and a desire for an expert system?
Is there at least one human expert who is willing to cooperate?
Can the expert explain the knowledge to the knowledge engineer and he can understand it?
Is the problem-solving knowledge mainly heuristic and uncertain?

20. Languages, Shells, and Tools
Expert system languages are post-third generation.
Procedural languages (e.g., C) focus on techniques to represent data.
More modern languages (e.g., Java) focus on data abstraction.
Expert system languages (e.g. CLIPS) focus on ways to represent knowledge.

21. Expert systems & conventional programs I

22. Expert systems & conventional programs II

23. Expert systems & conventional programs III

24. Elements of an Expert System
User interface – mechanism by which user and system communicate.
Exploration facility – explains reasoning of expert system to user.
Working memory – global database of facts used by rules.
Inference engine – makes inferences deciding which rules are satisfied and prioritizing.
Agenda – a prioritized list of rules created by the inference engine, whose patterns are satisfied by facts or objects in working memory.
Knowledge acquisition facility – automatic way for the user to enter knowledge in the system by passing the explicit coding by knowledge engineer.
Knowledge Base – includes the rules of the expert system

25. Production Rules Knowledge base is also called production memory.
Production rules can be expressed in IF-THEN pseudo code format.
In rule-based systems, the inference engine determines which rule antecedents are satisfied by the facts.

26. Figure 1.5 Structure of a Rule-Based Expert System

27. Rule-Based ES

28. Example Rules

29. 3. Inference Engine Cycle

30. Foundation of Expert Systems

31. General Methods of Inference
Forward chaining (data-driven)– reasoning from facts to the conclusions resulting from those facts – best for prognosis, monitoring, and control.
Examples: CLIPS, OPS5
Backward chaining (query/Goal driven)– reasoning in reverse from a hypothesis, a potential conclusion to be proved to the facts that support the hypothesis – best for diagnosis problems.
Examples: MYCIN

32. Production Systems
Rule-based expert systems – most popular type today.
Knowledge is represented as multiple rules that specify what should/not be concluded from different situations.
Forward chaining – start w/facts and use rules do draw conclusions/take actions.
Backward chaining – start w/hypothesis and look for rules that allow hypothesis to be proven true.

33. Post Production System
Basic idea – any mathematical / logical system is simply a set of rules specifying how to change one string of symbols into another string of symbols.
these rules are also known as rewrite rules
simple syntactic string manipulation
no understanding or interpretation is required\also used to define grammars of languages
e.g BNF grammars of programming languages.
Basic limitation – lack of control mechanism to guide the application of the rules.

34. Reasoning Under Uncertainty
Chapter 2:
Reasoning Under Uncertainty

35. Contents
Introduction
Dealing with Uncertainty
Classical Probability

36. Objectives
Learn the meaning of uncertainty and explore some theories designed to deal with it
Find out what types of errors can be attributed to uncertainty and induction
Learn about classical probability, experimental, and subjective probability, and conditional probability
Explore hypothetical reasoning and backward induction
Examine temporal reasoning and Markov chains
Define odds of belief, sufficiency, and necessity
Determine the role of uncertainty in inference chains
Explore the implications of combining evidence
Look at the role of inference nets in expert systems and see how probabilities are propagated

37. How to Expert Systems Deal with Uncertainty?
Expert systems provide an advantage when dealing with uncertainty as compared to decision trees.
With decision trees, all the facts must be known to arrive at an outcome.
Probability theory is devoted to dealing with theories of uncertainty.
There are many theories of probability – each with advantages and disadvantages.

38. What is Uncertainty?
Uncertainty is essentially lack of information to formulate a decision.
Uncertainty may result in making poor or bad decisions.
As living creatures, we are accustomed to dealing with uncertainty – that’s how we survive.
Dealing with uncertainty requires reasoning under uncertainty along with possessing a lot of common sense.

39. 2. Dealing with Uncertainty
Deductive reasoning – deals with exact facts and exact conclusions
Inductive reasoning – not as strong as deductive – premises support the conclusion but do not guarantee it.
There are a number of methods to pick the best solution in light of uncertainty.
When dealing with uncertainty, we may have to settle for just a good solution.

40. Theories to Deal with Uncertainty
Bayesian Probability
Hartley Theory
Shannon Theory
Dempster-Shafer Theory
Markov Models
Zadeh’s Fuzzy Theory

41. Errors Related to Hypothesis
Many types of errors contribute to uncertainty.
Type I Error – accepting a hypothesis when it is not true – False Positive.
Type II Error – Rejecting a hypothesis (theory) when it is true – False Negative

42. Errors Related to Measurement
Errors of precision – how well the truth is known
Errors of accuracy – whether something is true or not
Unreliability stems from faulty measurement of data – results in erratic data.
Random fluctuations – termed random error
Systematic errors result from bias

43. Errors in Induction
Where deduction proceeds from general to specific, induction proceeds from specific to general.
Inductive arguments can never be proven correct (except in mathematical induction).
Expert systems may consist of both deductive and inductive rules based on heuristic information.
When rules are based on heuristics, there will be uncertainty.

44. Figure 2.1Deductive and Inductive Reasoning about Populations and Samples

45. Figure 2.2 Types of Errors

46. Table 2.1 Examples of Common Types of Errors

47. 3. Classical Probability
First proposed by Pascal and Fermat in 1654
Also called a priori probability because it deals with ideal games or systems:
Assumes all possible events are known
Each event is equally likely to happen
Fundamental theorem for classical probability is P = W / N, where W is the number of wins and N is the number of equally possible events.

48. Deterministic &Nondeterministic Systems
When repeated trials give the exact same results, the system is deterministic.
Otherwise, the system is nondeterministic.
Nondeterministic does not necessarily mean random – could just be more than one way to meet one of the goals given the same input.

49. Three Axioms of Formal Theory of Probability

50. Experimental and Subjective Probabilities
Experimental probability defines the probability of an event, as the limit of a frequency distribution:
Subjective probability deals with events that are not reproducible and have no historical basis on which to extrapolate.

51. Compound Probabilities
Compound probabilities can be expressed by:
S is the sample space and A and B are events.
Independent events are events that do not affect each other. For pair wise independent events,

52. Additive Law

53. Conditional Probabilities
The probability of an event A occurring, given that event B has already occurred is called conditional probability:

54. Figure 2.3 Sample Space of Intersecting Events

55. Bayes’ Theorem This is the inverse of conditional probability.
Find the probability of an earlier event given that a later one occurred.

56. Hypothetical Reasoning Backward Induction
Bayes’ Theorem is commonly used for decision tree analysis of business and social sciences.
PROSPECTOR (expert system) achieved great fame as the first expert system to discover a valuable molybdenum deposit worth $100,000,000.

57. Temporal Reasoning Reasoning about events that depend on time
Expert systems designed to do temporal reasoning to explore multiple hypotheses in real time are difficult to build.
One approach to temporal reasoning is with probabilities – a system moving from one state to another over time.
The process is stochastic if it is probabilistic.

58. Reasoning about events that depend on time
Transition matrix – represents the probabilities that the system in one state will move to another.
State matrix – depicts the probabilities that the system is in any certain state.
One can show whether the states converge on a matrix called the steady-state matrix – a time of equilibrium

59. Markov Chain Characteristics
The process has a finite number of possible states.
The process can be in one and only one state at any one time.
The process moves or steps successively from one state to another over time.
The probability of a move depends only on the immediately preceding state.

60. Figure 2.4 State Diagram Interpretation of a Transition Matrix

61. The Odds of Belief
To make expert systems work for use, we must expand the scope of events to deal with propositions.
Rather than interpreting conditional probabilities P(A!B) in the classical sense, we interpret it to mean the degree of belief that A is true, given B.
We talk about the likelihood of A, based on some evidence B.
This can be interpreted in terms of odds.

62. Sufficiency and Necessity
The likelihood of sufficiency, LS, is calculated as:
The likelihood of necessity is defined as:

63. Table 2.2 Relationship Among Likelihood Ratio, Hypothesis, and Evidence

64. Table 2.3 Relationship Among Likelihood of Necessity, Hypothesis, and Evidence

65. Uncertainty in Inference Chains
Uncertainty may be present in rules, evidence used by rules, or both.
One way of correcting uncertainty is to assume that P(H|e) is a piecewise linear function.

66. Figure 2.5 Intersection of H and e

67. Figure 2.6 Piecewise Linear Interpolation Function for Partial Evidence in PROSPECTOR

68. Combination of Evidence
The simplest type of rule is of the form:
IF E THEN H
where E is a single piece of known evidence from which we can conclude that H is true.
Not all rules may be this simple – compensation for uncertainty may be necessary.
As the number of pieces of evidence increases, it becomes impossible to determine all the joint and prior probabilities or likelihoods.
If the antecedent is a logical combination of evidence, then fuzzy logic and negation rules can be used to combine evidence.

69. Types of Belief
Possible – no matter how remote, the hypothesis cannot be ruled out.
Probable – there is some evidence favoring the hypothesis but not enough to prove it.
Certain – evidence is logically true or false.
Impossible – it is false.
Plausible – more than a possibility exists.

70. Figure 2.7 Relative Meaning of Some Terms Used to Describe Evidence

71. Propagation of Probabilities
The chapter examines the classic expert system PROSPECTOR to illustrate how concepts of probability are used in a real system.
Inference nets like PROSPECTOR have a static knowledge structure.
Common rule-based system is a dynamic knowledge structure.

72. Chapter 3
Fuzzy Logic

73. B. Inputs to fuzzy logic systems - fuzzification C. Fuzzy propositions
A. Introduction
B. Inputs to fuzzy logic systems - fuzzification
C. Fuzzy propositions
D. Fuzzy hedges
E. Computing the results of a fuzzy proposition given an
input
F. The resulting action

74. Fuzzy Logic Objectives 1. To define the basic notions of fuzzy logic
2. To introduce the logical operations and relations on fuzzy sets
3. To learn how to obtain results of fuzzy logical operations
4. To apply what we learn to GIS

75. A. Introduction (figure from Earl Cox)

76. Introduction Steps (Earl Cox based on previous slide):
1. Input – vocabulary, fuzzification (creating fuzzy sets)
2. Fuzzy propositions – IF X is Y THEN Z (or Z is A) … there are four types of propositions
3. Hedges – very, extremely, somewhat, more, less
4. Combination and evaluation – computation of the results given the inputs
5. Action - defuzzification

77. Input – vocabulary, fuzzification (creating a fuzzy set) by using our previous methods of frequency, combination, experts/surveys (figure from Earl Cox)

78. Input (figure from Klir&Yuan)

79. Fuzzy Propositions – types 1 and 2
GENERAL FORMS
1. Unconditional and unqualified
proposition: Q is P
Example: Temperature(Q) is high(P)
2. Unconditional and qualified
proposition: proposition(Q is P) is R
Example: That Coimbra and Catania are beautiful is very true.

80. Fuzzy Proposition – type 1 and 2 (from Earl Cox)

81. Fuzzy Propositions – type 1 and 2 (from Earl Cox)

82. Fuzzy Propositions – type 3
3. Conditional and unqualified
proposition: IF Q is P THEN R is S
Example: If Robert is tall, then clothes are large.
If car is slow, then gear is low.

83. Fuzzy Propositions – type 4
4. Conditional and qualified
proposition: IF Q is P THEN R is S is T
{proposition(IF Q is P THEN R is S )} is T

84. Fuzzy Hedges (from Earl Cox)

85. Fuzzy Hedges (from Earl Cox)

86. Illustrations of Fuzzy Propositions – Composition/Evaluation (from Klir&Yuan)

87. Illustrations of Fuzzy Propositions – Composition/Evaluation (Earl Cox)

88. Illustrations of Fuzzy Propositions – Composition/Evaluation (from Earl Cox)

89. Illustrations of Fuzzy Propositions Decomposition – Defuzzification/Action (from Earl Cox)

90. Defuzzification (from Earl Cox)

91. Defuzzification (from Earl Cox)

92. Chapter 4 Design of Expert Systems

93. Considerations when Building an Expert system
We will describe the necessary prerequisites when building an expert system.
The system should be a quality product.
The development should be cost effective and timely.
Designing of expert systems of part of a general field known as Knowledge Management (KM).

94. Selecting the Appropriate Problem
We need to answer the questions, “Why are we building this expert system?”.
Intellectual Property Agreement must be considered
Clearly identify the problem
Clearly identify the expert
Clearly identify the users
We need to know the payoff – money, efficiency, etc.

95. Selecting the Appropriate Problem
What tools will be available to build the expert system?
Check the Web for applications in existence
Know the language necessary to create a semantic net of relationships on which the system will be based
How much will the expert system cost?
A function of people, resources, time, etc.
How available is the knowledge?

96. Figure 4.1 Project Management Tasks

97. Stages in the Development
How will the system be developed?
This will depend on the resources provided
Stages:
Feasibility Study – see if the project is feasible
Rapid Prototype – demonstrate ideas / impress
Refined System – verification by knowledge engineers
Field Testable – system tested by selected users

98. Stages in the Development
Commercial quality system – validation / testing
Maintenance and evolution – repair bugs, enhance capabilities

99. Other Considerations How will the system be delivered?
Should be considered in earliest stages of development
Integration with existing programs
How will the system be maintained and evolve?
Performance is dependent on knowledge / expertise
Performance must be maintained
New knowledge will be acquired
Old knowledge will be modified

100. Figure 4.2 General Stages in the Development of an Expert System

101. Errors in Development Stages
Expert’s knowledge may be erroneous, propagating errors throughout the entire development process.
Formal procedures may be necessary to certify expert
Technique panels can scrutinize expert’s knowledge
Focus groups can also be used

102. Errors in the Development Stages
Knowledge may not be properly communicated to knowledge engineer, or knowledge may be misinterpreted.
Knowledge base may be corrupted by entering incorrect form of a rule or fact.

103. Errors in the Development Stages
Inference engine errors may result from errors in pattern matching, conflict resolution, and execution of actions.
Inference chain errors may be caused by erroneous knowledge, semantic errors, inference engine bugs, incorrect specifications of rule priorities, and strange interaction among rules.
Limits of ignorance – a problem common to all previous stages

104. Figure 4.3 Major Errors in Expert Systems and Some Causes

105. Software Engineering & Expert Systems
Expert systems are products like any other software product and require good standards for development.
Expert systems may have serious responsibilities – life and death.
High standards are a necessity and can be measured by “mean time between failures”.

106. Figure 4.4 Software Engineering Methodology

107. Expert System Life Cycle
Begins with the initial concept of the software and ends with its retirement from use.
Expert systems require more maintenance because they are based on knowledge that is:
Heuristic
Experiential
A number of life cycle models have been developed.

108. Waterfall Model
Each stage ends with a verification and validation activity to minimize any problems in that stage.
Arrows go back and forth only one stage at a time.
It is assumed that all information necessary for a stage is known.

109. Figure 4.5 Waterfall Model of the Software Life Cycle

110. Code-and-Fix Model
Some code is written and then fixed when it does not work correctly.
Usually the method of choice for new programming students in conventional and expert systems
This eventually led to the do-it-twice concept where a prototype then a final system was built.

111. Incremental Model
This is a refinement of the waterfall and top-down-approach.
The idea is to develop software in increments of functional capability.
Major increment – assistant  colleague  expert
Minor increment – expertise w/in each level
Micro increment – add/refining individual rules

112. Figure 4.6 Spiral Model
Each circuit of the spiral adds some functional capability to the system.

113. Detailed Life Cycle Model Linear Model
Planning Stage
The purpose of this stage is to produce a formal work plan for the expert system development – documents to guide and evaluate the development.

114. Table 4.1 Planning Stage Tasks

115. Linear Model
Knowledge Definition
The objective of this stage is to define the knowledge requirements of the expert system, which consists of two main tasks:
Knowledge source identification and selection
Knowledge acquisition, analysis, and extraction

116. Table 4.2 Knowledge Source / Identification

117. Table 4.3 Knowledge Acquisition, Analysis, and Extraction Tasks

118. Linear Model
Knowledge Design
The objective is to produce the detailed design for an expert system and involves:
Knowledge definition
Detailed design

119. Table 4.4 Knowledge Definition Tasks

120. Table 4.5 Detailed Design of Knowledge Tasks

121. Linear Model Code and Checkout
This begins the actual code implementation

122. Table 4.6 Code and Checkout Tasks

123. Linear Model Knowledge Verification
The objective here is to determine the correctness, completeness, and consistency of the system.
Formal tests
Test Analysis

124. Table 6.7 Formal Test Tasks of Knowledge Verification Stage

125. Table 6.8 Test Analysis Tasks

126. Linear Model
System Evaluation
This stage is for summarizing what has been learned with recommendations for improvements and corrections.

127. Table 4.9 System Evaluation Stage Tasks

128. Figure 4.7 Linear Model of Expert System Development Life Cycle

129. Summary
In this chapter, we have discussed a software engineering approach to the construction of an expert system.
Principles about good interviewing techniques were mentioned.
Now that expert systems are widely used, they must be quality products due to the sensitive nature of their applications.

130. Summary
Factors to be considered in the design of expert systems include problem selection, cost, and payoff.
Both managerial and technical aspects must be considered.
By consistently following the outlined “life cycle” it should be possible to construct quality software.