1. Introduction to Artificial Intelligence
Li Li

2. Textbook
(in English) Michael Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems (2nd ed.), Addison Wesley, 2005
(in Chinese) 人工智能— 智能系统指南, 顾力栩等译, 机械工业出版社, 2008

3. To Pass this unit To get at least 50% from three assignments;
To get at least 50% from the final exam; and
To get a total of these two parts at least 60%

4. Introduction to knowledge-based intelligent systems
Lecture 1
Introduction to knowledge-based intelligent systems
Intelligent machines, or what machines can do
The history of artificial intelligence or from the “Dark Ages” to knowledge-based systems
Summary

5. Intelligent machines, or what machines can do
Philosophers have been trying for over 2000 years to understand and resolve two Big Questions of the Universe: How does a human mind work, and Can non-humans have minds? These questions are still unanswered.
Intelligence is their ability to understand and learn things. 2 Intelligence is the ability to think and understand instead of doing things by instinct or automatically.
(Essential English Dictionary, Collins, London, 1990)

6. In order to think, someone or something has to have a brain, or an organ that enables someone or something to learn and understand things, to solve problems and to make decisions. So we can define intelligence as the ability to learn and understand, to solve problems and to make decisions.
The goal of artificial intelligence (AI) as a science is to make machines do things that would require intelligence if done by humans. Therefore, the answer to the question Can Machines Think? was vitally important to the discipline.
The answer is not a simple “Yes” or “No”.

7. Some people are smarter in some ways than others
Some people are smarter in some ways than others. Sometimes we make very intelligent decisions but sometimes we also make very silly mistakes. Some of us deal with complex mathematical and engineering problems but are moronic in philosophy and history. Some people are good at making money, while others are better at spending it. As humans, we all have the ability to learn and understand, to solve problems and to make decisions; however, our abilities are not equal and lie in different areas. Therefore, we should expect that if machines can think, some of them might be smarter than others in some ways.

8. One of the most significant papers on machine intelligence, “Computing Machinery and Intelligence”, was written by the British mathematician Alan Turing over fifty years ago . However, it still stands up well under the test of time, and the Turing’s approach remains universal.
He asked: Is there thought without experience? Is there mind without communication? Is there language without living? Is there intelligence without life? All these questions, as you can see, are just variations on the fundamental question of artificial intelligence, Can machines think?

9. Turing did not provide definitions of machines and thinking, he just avoided semantic arguments by inventing a game, the Turing Imitation Game.
The imitation game originally included two phases. In the first phase, the interrogator, a man and a woman are each placed in separate rooms. The interrogator’s objective is to work out who is the man and who is the woman by questioning them. The man should attempt to deceive the interrogator that he is the woman, while the woman has to convince the interrogator that she is the woman.

10. Turing Imitation Game: Phase 1

11. Turing Imitation Game: Phase 2
In the second phase of the game, the man is replaced by a computer programmed to deceive the interrogator as the man did. It would even be programmed to make mistakes and provide fuzzy answers in the way a human would. If the computer can fool the interrogator as often as the man did, we may say this computer has passed the intelligent behaviour test.

12. Turing Imitation Game: Phase 2

13. The Turing test has two remarkable qualities that make it really universal.
By maintaining communication between the human and the machine via terminals, the test gives us an objective standard view on intelligence.
The test itself is quite independent from the details of the experiment. It can be conducted as a two-phase game, or even as a single-phase game when the interrogator needs to choose between the human and the machine from the beginning of the test.

14. Turing believed that by the end of the 20th century it would be possible to program a digital computer to play the imitation game. Although modern computers still cannot pass the Turing test, it provides a basis for the verification and validation of knowledge-based systems.
A program thought intelligent in some narrow area of expertise is evaluated by comparing its performance with the performance of a human expert.
To build an intelligent computer system, we have to capture, organise and use human expert knowledge in some narrow area of expertise.

15. The history of artificial intelligence
The birth of artificial intelligence (1943 – 1956)
The first work recognised in the field of AI was presented by Warren McCulloch and Walter Pitts in They proposed a model of an artificial neural network and demonstrated that simple network structures could learn.
McCulloch, the second “founding father” of AI after Alan Turing, had created the corner stone of neural computing and artificial neural networks (ANN).

16. The third founder of AI was John von Neumann, the brilliant Hungarian-born mathematician. In 1930, he joined the Princeton University, lecturing in mathematical physics. He was an adviser for the Electronic Numerical Integrator and Calculator project at the University of Pennsylvania and helped to design the Electronic Discrete Variable Calculator. He was influenced by McCulloch and Pitts’s neural network model. When Marvin Minsky and Dean Edmonds, two graduate students in the Princeton mathematics department, built the first neural network computer in 1951, von Neumann encouraged and supported them.

17. The von Neumann architecture is a design model for a stored-program digital computer that uses a central processing unit (CPU) and a single separate storage structure ("memory") to hold both instructions and data. Such computers implement a universal Turing machine and have a sequential architecture.

19. Another of the first generation researchers was Claude Shannon
Another of the first generation researchers was Claude Shannon. He graduated from MIT and joined Bell Telephone Laboratories in Shannon shared Alan Turing’s ideas on the possibility of machine intelligence. In 1950, he published a paper on chess-playing machines, which pointed out that a typical chess game involved about possible moves (Shannon, 1950). Even if the new von Neumann-type computer could examine one move per microsecond, it would take 3  years to make its first move. Thus Shannon demonstrated the need to use heuristics in the search for the solution.

20. In 1956, John McCarthy, Marvin Minsky and Claude Shannon organised a summer workshop at Dartmouth College. They brought together researchers interested in the study of machine intelligence, artificial neural nets and automata theory. Although there were just ten researchers, this workshop gave birth to a new science called artificial intelligence.

21. The rise of artificial intelligence, or the era of great expectations (1956 – late 1960s)
The early works on neural computing and artificial neural networks started by McCulloch and Pitts was continued. Learning methods were improved and Frank Rosenblatt proved the perceptron convergence theorem, demonstrating that his learning algorithm could adjust the connection strengths of a perceptron.

22. One of the most ambitious projects of the era of great expectations was the General Problem Solver (GPS). Allen Newell and Herbert Simon from the Carnegie Mellon University developed a general-purpose program to simulate human-solving methods.
Newell and Simon postulated that a problem to be solved could be defined in terms of states. They used the mean-end analysis to determine a difference between the current and desirable or goal state of the problem, and to choose and apply operators to reach the goal state. The set of operators determined the solution plan.

23. However, GPS failed to solve complex problems
However, GPS failed to solve complex problems. The program was based on formal logic and could generate an infinite number of possible operators. The amount of computer time and memory that GPS required to solve real-world problems led to the project being abandoned.
In the sixties, AI researchers attempted to simulate the thinking process by inventing general methods for solving broad classes of problems. They used the general-purpose search mechanism to find a solution to the problem. Such approaches, now referred to as weak methods, applied weak information about the problem domain.

24. By 1970, the euphoria about AI was gone, and most government funding for AI projects was cancelled. AI was still a relatively new field, academic in nature, with few practical applications apart from playing games. So, to the outsider, the achieved results would be seen as toys, as no AI system at that time could manage real-world problems.

25. Unfulfilled promises, or the impact of reality
(late 1960s – early 1970s)
The main difficulties for AI in the late 1960s were:
Because AI researchers were developing general methods for broad classes of problems, early programs contained little or even no knowledge about a problem domain. To solve problems, programs applied a search strategy by trying out different combinations of small steps, until the right one was found. This approach was quite feasible for simple toy problems, so it seemed reasonable that, if the programs could be “scaled up” to solve large problems, they would finally succeed.

26. Many of the problems that AI attempted to solve were too broad and too difficult. A typical task for early AI was machine translation. For example, the National Research Council, USA, funded the translation of Russian scientific papers after the launch of the first artificial satellite (Sputnik) in Initially, the project team tried simply replacing Russian words with English, using an electronic dictionary. However, it was soon found that translation requires a general understanding of the subject to choose the correct words. This task was too difficult. In 1966, all translation projects funded by the US government were cancelled.

27. In 1971, the British government also suspended support for AI research
In 1971, the British government also suspended support for AI research. Sir James Lighthill had been commissioned by the Science Research Council of Great Britain to review the current state of AI. He did not find any major or even significant results from AI research, and therefore saw no need to have a separate science called “artificial intelligence”.

28. The technology of expert systems, or the key to success (early 1970s – mid-1980s)
Probably the most important development in the 70s was the realisation that the domain for intelligent machines had to be sufficiently restricted. Previously, AI researchers had believed that clever search algorithms and reasoning techniques could be invented to emulate general, human-like, problem-solving methods. A general-purpose search mechanism could rely on elementary reasoning steps to find complete solutions and could use weak knowledge about domain.

29. When weak methods failed, researchers finally realised that the only way to deliver practical results was to solve typical cases in narrow areas of expertise, making large reasoning steps.

30. DENDRAL
DENDRAL was developed at Stanford University to determine the molecular structure of Martian soil, based on the mass spectral data provided by a mass spectrometer. The project was supported by NASA. Edward Feigenbaum, Bruce Buchanan (a computer scientist) and Joshua Lederberg (a Nobel prize winner in genetics) formed a team.
There was no scientific algorithm for mapping the mass spectrum into its molecular structure. Feigenbaum’s job was to incorporate the expertise of Lederberg into a computer program to make it perform at a human expert level. Such programs were later called expert systems.

31. DENDRAL marked a major “paradigm shift” in AI: a shift from general-purpose, knowledge-sparse weak methods to domain-specific, knowledge-intensive techniques.
The aim of the project was to develop a computer program to attain the level of performance of an experienced human chemist. Using heuristics in the form of high-quality specific rules, rules-of-thumb , the DENDRAL team proved that computers could equal an expert in narrow, well defined, problem areas.
The DENDRAL project originated the fundamental idea of expert systems – knowledge engineering, which encompassed techniques of capturing, analysing and expressing in rules an expert’s “know-how”.

32. MYCIN
MYCIN was a rule-based expert system for the diagnosis of infectious blood diseases. It also provided a doctor with therapeutic advice in a convenient, user-friendly manner.
MYCIN’s knowledge consisted of about 450 rules derived from human knowledge in a narrow domain through extensive interviewing of experts.
The knowledge incorporated in the form of rules was clearly separated from the reasoning mechanism. The system developer could easily manipulate knowledge in the system by inserting or deleting some rules. For example, a domain-independent version of MYCIN called EMYCIN (Empty MYCIN) was later produced.

33. PROSPECTOR
PROSPECTOR was an expert system for mineral exploration developed by the Stanford Research Institute. Nine experts contributed their knowledge and expertise. PROSPECTOR used a combined structure that incorporated rules and a semantic network. PROSPECTOR had over 1000 rules.
The user, an exploration geologist, was asked to input the characteristics of a suspected deposit: the geological setting, structures, kinds of rocks and minerals. PROSPECTOR compared these characteristics with models of ore deposits and made an assessment of the suspected mineral deposit. It could also explain the steps it used to reach the conclusion.

34. A 1986 survey reported a remarkable number of successful expert system applications in different areas: chemistry, electronics, engineering, geology, management, medicine, process control and military science (Waterman, 1986). Although Waterman found nearly 200 expert systems, most of the applications were in the field of medical diagnosis. Seven years later (1993) a similar survey reported over 2500 developed expert systems (Durkin, 1994). The new growing area was business and manufacturing, which accounted for about 60% of the applications. Expert system technology had clearly matured.

35. However:
Expert systems are restricted to a very narrow domain of expertise. For example, MYCIN, which was developed for the diagnosis of infectious blood diseases, lacks any real knowledge of human physiology. If a patient has more than one disease, we cannot rely on MYCIN. In fact, therapy prescribed for the blood disease might even be harmful because of the other disease.
Expert systems can show the sequence of the rules they applied to reach a solution, but cannot relate accumulated, heuristic knowledge to any deeper understanding of the problem domain.

36. Expert systems have difficulty in recognising domain boundaries
Expert systems have difficulty in recognising domain boundaries. When given a task different from the typical problems, an expert system might attempt to solve it and fail in rather unpredictable ways.
Heuristic rules represent knowledge in abstract form and lack even basic understanding of the domain area. It makes the task of identifying incorrect, incomplete or inconsistent knowledge difficult.
Expert systems, especially the first generation, have little or no ability to learn from their experience. Expert systems are built individually and cannot be developed fast. Complex systems can take over 30 person-years to build.

37. How to make a machine learn, or the rebirth of neural networks (mid-1980s – onwards)
In the mid-eighties, researchers, engineers and experts found that building an expert system required much more than just buying a reasoning system or expert system shell and putting enough rules in it. Disillusions about the applicability of expert system technology even led to people predicting an AI “winter” with severely squeezed funding for AI projects. AI researchers decided to have a new look at neural networks.

38. By the late sixties, most of the basic ideas and concepts necessary for neural computing had already been formulated. However, only in the mid-eighties did the solution emerge. The major reason for the delay was technological: there were no PCs or powerful workstations to model and experiment with artificial neural networks.
In the eighties, because of the need for brain-like information processing, as well as the advances in computer technology and progress in neuroscience, the field of neural networks experienced a dramatic resurgence. Major contributions to both theory and design were made on several fronts.

39. Grossberg established a new principle of self-organisation (adaptive resonance theory), which provided the basis for a new class of neural networks (Grossberg, 1980).
Hopfield introduced neural networks with feedback – Hopfield networks, which attracted much attention in the eighties (Hopfield, 1982).
Kohonen published a paper on self-organising maps (Kohonen, 1982).
Barto, Sutton and Anderson published their work on reinforcement learning and its application in control (Barto et al., 1983).

40. But the real breakthrough came in 1986 when the back-propagation learning algorithm, first introduced by Bryson and Ho in 1969 (Bryson & Ho, 1969), was reinvented by Rumelhart and McClelland in Parallel Distributed Processing (1986).
Artificial neural networks have come a long way from the early models of McCulloch and Pitts to an interdisciplinary subject with roots in neuroscience, psychology, mathematics and engineering, and will continue to develop in both theory and practical applications.

41. Evolutionary computation
Simulating biological evolution – simply compete for survival
The fittest species have a greater chance to reproduce
It is based on the computation model of natural selection
Worth mentioning: genetic algorithms (Holland 1975)
A demo:

42. The new era of knowledge engineering, or computing with words (late 1980s – onwards)
Neural network technology offers more natural interaction with the real world than do systems based on symbolic reasoning. Neural networks can learn, adapt to changes in a problem’s environment, establish patterns in situations where rules are not known, and deal with fuzzy or incomplete information. However, they lack explanation facilities and usually act as a black box. The process of training neural networks with current technologies is slow, and frequent retraining can cause serious difficulties.

43. Classic expert systems are especially good for closed-system applications with precise inputs and logical outputs. They use expert knowledge in the form of rules and, if required, can interact with the user to establish a particular fact. A major drawback is that human experts cannot always express their knowledge in terms of rules or explain the line of their reasoning. This can prevent the expert system from accumulating the necessary knowledge, and consequently lead to its failure.

44. Very important technology dealing with vague, imprecise and uncertain knowledge and data is fuzzy logic.
Human experts do not usually think in probability values, but in such terms as often, generally, sometimes, occasionally and rarely. Fuzzy logic is concerned with capturing the meaning of words, human reasoning and decision making. Fuzzy logic provides the way to break through the computational bottlenecks of traditional expert systems.
At the heart of fuzzy logic lies the concept of a linguistic variable. The values of the linguistic variable are words rather than numbers.

45. Fuzzy logic or fuzzy set theory was introduced by Professor Lotfi Zadeh, Berkeley’s electrical engineering department chairman, in It provided a means of computing with words. However, acceptance of fuzzy set theory by the technical community was slow and difficult. Part of the problem was the provocative name – “fuzzy” – it seemed too light-hearted to be taken seriously. Eventually, fuzzy theory, ignored in the West, was taken seriously in the East – by the Japanese. It has been used successfully since 1987 in Japanese-designed dishwashers, washing machines, air conditioners, television sets, copiers, and even cars.

46. Benefits derived from the application of fuzzy logic models in knowledge-based and decision-support systems can be summarised as follows:
Improved computational power: Fuzzy rule-based systems perform faster than conventional expert systems and require fewer rules. A fuzzy expert system merges the rules, making them more powerful. Lotfi Zadeh believes that in a few years most expert systems will use fuzzy logic to solve highly nonlinear and computationally difficult problems.

47. Improved cognitive modelling: Fuzzy systems allow the encoding of knowledge in a form that reflects the way experts think about a complex problem. They usually think in such imprecise terms as high and low, fast and slow, heavy and light. In order to build conventional rules, we need to define the crisp boundaries for these terms by breaking down the expertise into fragments. This fragmentation leads to the poor performance of conventional expert systems when they deal with complex problems. In contrast, fuzzy expert systems model imprecise information, capturing expertise similar to the way it is represented in the expert mind, and thus improve cognitive modelling of the problem.

48. The ability to represent multiple experts: Conventional expert systems are built for a narrow domain. It makes the system’s performance fully dependent on the right choice of experts. When a more complex expert system is being built or when expertise is not well defined, multiple experts might be needed. However, multiple experts seldom reach close agreements; there are often differences in opinions and even conflicts. This is especially true in areas, such as business and management, where no simple solution exists and conflicting views should be taken into account. Fuzzy expert systems can help to represent the expertise of multiple experts when they have opposing views.

49. Although fuzzy systems allow expression of expert knowledge in a more natural way, they still depend on the rules extracted from the experts, and thus might be smart or dumb. Some experts can provide very clever fuzzy rules – but some just guess and may even get them wrong. Therefore, all rules must be tested and tuned, which can be a prolonged and tedious process. For example, it took Hitachi engineers several years to test and tune only 54 fuzzy rules to guide the Sendal Subway System.

50. In recent years, several methods based on neural network technology have been used to search numerical data for fuzzy rules. Adaptive or neural fuzzy systems can find new fuzzy rules, or change and tune existing ones based on the data provided. In other words, data in – rules out, or experience in – common sense out.

51. Summary
Expert, neural and fuzzy systems have now matured and been applied to a broad range of different problems, mainly in engineering, medicine, finance, business and management.
Each technology handles the uncertainty and ambiguity of human knowledge differently, and each technology has found its place in knowledge engineering. They no longer compete; rather they complement each other.

52. A synergy of expert systems with fuzzy logic and neural computing improves adaptability, robustness, fault-tolerance and speed of knowledge-based systems. Besides, computing with words makes them more “human”. It is now common practice to build intelligent systems using existing theories rather than to propose new ones, and to apply these systems to real-world problems rather than to “toy” problems.

53. Main events in the history of AI

59. Ch2 Rule-based expert systems
Ch3 Uncertainty management in rule-based expert systems
Ch4 Fuzzy expert systems
Ch5 Frame-based expert systems
Ch6 Artificial neural networks
Ch7 Evolutionary computation
Ch8 Hybrid intelligent systems
Ch9 Knowledge engineering and data mining

60. Rule-based expert systems
Lecture 2
Rule-based expert systems
Introduction, or what is knowledge?
Rules as a knowledge representation technique
The main players in the development team
Structure of a rule-based expert system
Characteristics of an expert system
Forward chaining and backward chaining
Conflict resolution
Summary

61. Introduction, or what is knowledge?
Knowledge is a theoretical or practical understanding of a subject or a domain. Knowledge is also the sum of what is currently known, and apparently knowledge is power. Those who possess knowledge are called experts.
Anyone can be considered a domain expert if he or she has deep knowledge (of both facts and rules) and strong practical experience in a particular domain. The area of the domain may be limited. In general, an expert is a skilful person who can do things other people cannot.

62. IF the ‘traffic light’ is red THEN the action is stop
The human mental process is internal, and it is too complex to be represented as an algorithm. However, most experts are capable of expressing their knowledge in the form of rules for problem solving.
IF the ‘traffic light’ is green
THEN the action is go
IF the ‘traffic light’ is red
THEN the action is stop

63. Rules as a knowledge representation technique
The term rule in AI, which is the most commonly used type of knowledge representation, can be defined as an IF-THEN structure that relates given information or facts in the IF part to some action in the THEN part. A rule provides some description of how to solve a problem. Rules are relatively easy to create and understand.
Any rule consists of two parts: the IF part, called the antecedent (premise or condition) and the THEN part called the consequent (conclusion or action).

64. AND antecedent 2 OR antecedent 2 . .
IF antecedent
THEN consequent
A rule can have multiple antecedents joined by the keywords AND (conjunction), OR (disjunction) or
a combination of both.
IF antecedent 1 IF antecedent 1
AND antecedent 2 OR antecedent 2
AND antecedent n OR antecedent n
THEN consequent THEN consequent

65. IF ‘age of the customer’ < 18 AND ‘cash withdrawal’ > 1000
The antecedent of a rule incorporates two parts: an object (linguistic object) and its value. The object and its value are linked by an operator.
The operator identifies the object and assigns the value. Operators such as is, are, is not, are not are used to assign a symbolic value to a linguistic object.
Expert systems can also use mathematical operators to define an object as numerical and assign it to the numerical value.
IF ‘age of the customer’ < 18
AND ‘cash withdrawal’ > 1000
THEN ‘signature of the parent’ is required

66. Rules can represent relations, recommendations, directives, strategies and heuristics:
IF the ‘fuel tank’ is empty
THEN the car is dead
Recommendation
IF the season is autumn
AND the sky is cloudy
AND the forecast is drizzle
THEN the advice is ‘take an umbrella’
Directive
IF the car is dead
AND the ‘fuel tank’ is empty
THEN the action is ‘refuel the car’

67. Strategy
IF the car is dead
THEN the action is ‘check the fuel tank’;
step1 is complete
IF step1 is complete
AND the ‘fuel tank’ is full
THEN the action is ‘check the battery’;
step2 is complete
Heuristic
IF the spill is liquid
AND the ‘spill pH’ < 6
AND the ‘spill smell’ is vinegar
THEN the ‘spill material’ is ‘acetic acid’

68. The main players in the development team
There are five members of the expert system development team: the domain expert, the knowledge engineer, the programmer, the project manager and the end-user.
The success of their expert system entirely depends on how well the members work together.

69. The main players in the development team

70. The domain expert is a knowledgeable and skilled person capable of solving problems in a specific area or domain. This person has the greatest expertise in a given domain. This expertise is to be captured in the expert system. Therefore, the expert must be able to communicate his or her knowledge, be willing to participate in the expert system development and commit a substantial amount of time to the project. The domain expert is the most important player in the expert system development team.

71. The knowledge engineer is someone who is capable of designing, building and testing an expert system. He or she interviews the domain expert to find out how a particular problem is solved. The knowledge engineer establishes what reasoning methods the expert uses to handle facts and rules and decides how to represent them in the expert system. The knowledge engineer then chooses some development software or an expert system shell, or looks at programming languages for encoding the knowledge. And finally, the knowledge engineer is responsible for testing, revising and integrating the expert system into the workplace.

72. The programmer is the person responsible for the actual programming, describing the domain knowledge in terms that a computer can understand. The programmer needs to have skills in symbolic programming in such AI languages as LISP, Prolog and OPS5 and also some experience in the application of different types of expert system shells. In addition, the programmer should know conventional programming languages like C, Pascal, FORTRAN and Basic.

73. The project manager is the leader of the expert system development team, responsible for keeping the project on track. He or she makes sure that all deliverables and milestones are met, interacts with the expert, knowledge engineer, programmer and end-user.
The end-user, often called just the user, is a person who uses the expert system when it is developed. The user must not only be confident in the expert system performance but also feel comfortable using it. Therefore, the design of the user interface of the expert system is also vital for the project’s success; the end-user’s contribution here can be crucial.

74. Structure of a rule-based expert system
In the early seventies, Newell and Simon from Carnegie-Mellon University proposed a production system model, the foundation of the modern rule-based expert systems.
The production model is based on the idea that humans solve problems by applying their knowledge (expressed as production rules) to a given problem represented by problem-specific information.
The production rules are stored in the long-term memory and the problem-specific information or facts in the short-term memory.

75. Production system model

76. Basic structure of a rule-based expert system

77. The knowledge base contains the domain knowledge useful for problem solving. In a rule-based expert system, the knowledge is represented as a set of rules. Each rule specifies a relation, recommendation, directive, strategy or heuristic and has the IF (condition) THEN (action) structure. When the condition part of a rule is satisfied, the rule is said to fire and the action part is executed.
The database includes a set of facts used to match against the IF (condition) parts of rules stored in the knowledge base.

78. The inference engine carries out the reasoning whereby the expert system reaches a solution. It links the rules given in the knowledge base with the facts provided in the database.
The explanation facilities enable the user to ask the expert system how a particular conclusion is reached and why a specific fact is needed. An expert system must be able to explain its reasoning and justify its advice, analysis or conclusion.
The user interface is the means of communication between a user seeking a solution to the problem and an expert system.

79. Complete structure of a rule-based expert system

80. Characteristics of an expert system
An expert system is built to perform at a human expert level in a narrow, specialised domain. Thus, the most important characteristic of an expert system is its high-quality performance. No matter how fast the system can solve a problem, the user will not be satisfied if the result is wrong.
On the other hand, the speed of reaching a solution is very important. Even the most accurate decision or diagnosis may not be useful if it is too late to apply, for instance, in an emergency, when a patient dies or a nuclear power plant explodes.

81. Expert systems apply heuristics to guide the reasoning and thus reduce the search area for a solution.
A unique feature of an expert system is its explanation capability. It enables the expert system to review its own reasoning and explain its decisions.
Expert systems employ symbolic reasoning when solving a problem. Symbols are used to represent different types of knowledge such as facts, concepts and rules.

82. Can expert systems make mistakes?
Even a brilliant expert is only a human and thus can make mistakes. This suggests that an expert system built to perform at a human expert level also should be allowed to make mistakes. But we still trust experts, even we recognise that their judgements are sometimes wrong. Likewise, at least in most cases, we can rely on solutions provided by expert systems, but mistakes are possible and we should be aware of this.

83. In expert systems, knowledge is separated from its processing (the knowledge base and the inference engine are split up). A conventional program is a mixture of knowledge and the control structure to process this knowledge. This mixing leads to difficulties in understanding and reviewing the program code, as any change to the code affects both the knowledge and its processing.
When an expert system shell is used, a knowledge engineer or an expert simply enters rules in the knowledge base. Each new rule adds some new knowledge and makes the expert system smarter.

84. Comparison of expert systems with conventional systems and human experts

85. Comparison of expert systems with conventional systems and human experts (Continued)

86. Forward chaining and backward chaining
In a rule-based expert system, the domain knowledge is represented by a set of IF-THEN production rules and data is represented by a set of facts about the current situation. The inference engine compares each rule stored in the knowledge base with facts contained in the database. When the IF (condition) part of the rule matches a fact, the rule is fired and its THEN (action) part is executed.
The matching of the rule IF parts to the facts produces inference chains. The inference chain indicates how an expert system applies the rules to reach a conclusion.

87. Inference engine cycles via a match-fire procedure

88. An example of an inference chain

89. Forward chaining
Forward chaining is the data-driven reasoning. The reasoning starts from the known data and proceeds forward with that data. Each time only the topmost rule is executed. When fired, the rule adds a new fact in the database. Any rule can be executed only once. The match-fire cycle stops when no further rules can be fired.

90. Forward chaining

91. Forward chaining is a technique for gathering information and then inferring from it whatever can be inferred.
However, in forward chaining, many rules may be executed that have nothing to do with the established goal.
Therefore, if our goal is to infer only one particular fact, the forward chaining inference technique would not be efficient.

92. Backward chaining
Backward chaining is the goal-driven reasoning. In backward chaining, an expert system has the goal (a hypothetical solution) and the inference engine attempts to find the evidence to prove it. First, the knowledge base is searched to find rules that might have the desired solution. Such rules must have the goal in their THEN (action) parts. If such a rule is found and its IF (condition) part matches data in the database, then the rule is fired and the goal is proved. However, this is rarely the case.

93. Backward chaining
Thus the inference engine puts aside the rule it is working with (the rule is said to stack) and sets up a new goal, a subgoal, to prove the IF part of this rule. Then the knowledge base is searched again for rules that can prove the subgoal. The inference engine repeats the process of stacking the rules until no rules are found in the knowledge base to prove the current subgoal.

94. Backward chaining

95. How do we choose between forward and backward chaining?
If an expert first needs to gather some information and then tries to infer from it whatever can be inferred, choose the forward chaining inference engine.
However, if your expert begins with a hypothetical solution and then attempts to find facts to prove it, choose the backward chaining inference engine.

96. Conflict resolution Rule 1: IF the ‘traffic light’ is green
Earlier we considered two simple rules for crossing a road. Let us now add third rule:
Rule 1:
IF the ‘traffic light’ is green
THEN the action is go
Rule 2:
IF the ‘traffic light’ is red
THEN the action is stop
Rule 3:

97. We have two rules, Rule 2 and Rule 3, with the same IF part
We have two rules, Rule 2 and Rule 3, with the same IF part. Thus both of them can be set to fire when the condition part is satisfied. These rules represent a conflict set. The inference engine must determine which rule to fire from such a set. A method for choosing a rule to fire when more than one rule can be fired in a given cycle is called conflict resolution.

98. In forward chaining, BOTH rules would be fired
In forward chaining, BOTH rules would be fired. Rule 2 is fired first as the topmost one, and as a result, its THEN part is executed and linguistic object action obtains value stop. However, Rule 3 is also fired because the condition part of this rule matches the fact ‘traffic light’ is red, which is still in the database. As a consequence, object action takes new value go.

99. Methods used for conflict resolution
Fire the rule with the highest priority. In simple applications, the priority can be established by placing the rules in an appropriate order in the knowledge base. Usually this strategy works well for expert systems with around 100 rules.
Fire the most specific rule. This method is also known as the longest matching strategy. It is based on the assumption that a specific rule processes more information than a general one.

100. Fire the rule that uses the data most recently entered in the database
Fire the rule that uses the data most recently entered in the database. This method relies on time tags attached to each fact in the database. In the conflict set, the expert system first fires the rule whose antecedent uses the data most recently added to the database.

101. Metaknowledge
Metaknowledge can be simply defined as knowledge about knowledge. Metaknowledge is knowledge about the use and control of domain knowledge in an expert system.
In rule-based expert systems, metaknowledge is represented by metarules. A metarule determines a strategy for the use of task-specific rules in the expert system.

102. Metarules
Metarule 1:
Rules supplied by experts have higher priorities than rules supplied by novices.
Metarule 2:
Rules governing the rescue of human lives have higher priorities than rules concerned with clearing overloads on power system equipment.

103. Advantages of rule-based expert systems
Natural knowledge representation. An expert usually explains the problem-solving procedure with such expressions as this: “In such-and-such situation, I do so-and-so”. These expressions can be represented quite naturally as IF-THEN production rules.
Uniform structure. Production rules have the uniform IF-THEN structure. Each rule is an independent piece of knowledge. The very syntax of production rules enables them to be self-documented.

104. Advantages of rule-based expert systems
Separation of knowledge from its processing. The structure of a rule-based expert system provides an effective separation of the knowledge base from the inference engine. This makes it possible to develop different applications using the same expert system shell.
Dealing with incomplete and uncertain knowledge. Most rule-based expert systems are capable of representing and reasoning with incomplete and uncertain knowledge.

105. Disadvantages of rule-based expert systems
Opaque relations between rules. Although the individual production rules are relatively simple and self-documented, their logical interactions within the large set of rules may be opaque. Rule-based systems make it difficult to observe how individual rules serve the overall strategy.
Ineffective search strategy. The inference engine applies an exhaustive search through all the production rules during each cycle. Expert systems with a large set of rules (over 100 rules) can be slow, and thus large rule-based systems can be unsuitable for real-time applications.

106. Disadvantages of rule-based expert systems
Inability to learn. In general, rule-based expert systems do not have an ability to learn from the experience. Unlike a human expert, who knows when to “break the rules”, an expert system cannot automatically modify its knowledge base, or adjust existing rules or add new ones. The knowledge engineer is still responsible for revising and maintaining the system.

107. 专家系统的开发工具 (1)
专家系统的开发工具主要有以下几类： ①面向人工智能的通用程序设计语言，如lisp、 prolog等 ②通用知识表示语言，如OPS-5、KPL-UNITS等 ③专家系统外壳，如EMYCIN、KAS和EXPERT等 ④通用计算机程序语言及开发环境，如VC++，Delphi，JBuilderX等

108. 专家系统的开发工具 (2)
比较而言，使用面向人工智能的通用程序设计语言可以大大减轻开发的难度，便于设计通用的专家系统；但是和使用通用知识表示语言以及专家系统外壳相类似，当推理方法等有较大改进时，这种开发工具不利于系统维护。选择通用计算机程序语言作为开发工具则具有较大的灵活性，能够较好的适应胜任系统维护和升级的工作。其中，Java 语言以其跨平台等特点，得到了越来越广泛的应用。

109. 专家系统的开发工具 (2)-cont.
目前采用Java语言开发的专家系统，大多是采用J2EE架构的Web服务程序，结合J2EE架构提供的多层分布式应用模型、组件重用策略、一致化的安全模型以及灵活的事务控制等特性，依靠网络环境，可以方便的实现专家系统的远程诊断等功能，系统的维护也比较方便。 但从应用看，J2EE架构的程序对专家系统的使用造成了一定障碍。实际应用中，往往要面对网络环境不完善，软硬件配置不同的环境，要在这样的环境下构建一个有一定适应性的故障诊断专家系统，选择Java 语言，基于J2SE架构进行编程，是比较合适的解决方案

110. Demonstration – media advisor ES
The system provides advice on selecting a medium for delivering a training program based on trainee’s job.
Construction of the KB

111. KB (1)

112. KB (2)

113. KB (3)

114. Media advisor (1) Using 6 linguistic objects environment
stimilus_situation
job
stimulus_response
feedback
medium

115. Media advisor (2)
Each object (e.g. environment) can take one of the following values
Papers
Manuals
Documents
Textbooks
Pictures
Illustrations
Photographs
Diagrams
Machines
Buildings
Tools
Numbers
Formulas
Computer programs

116. Media advisor (3) –DB An object + its value  a fact
All facts are placed in the DB

117. Media advisor (4) – forward chaining
The goal: to produce a solution to the problem based on the provided data (via dialogue)
medium is …?

118. Media advisor (4) –backward chaining
Goal: “ ‘medium’ is ‘workshop’”
(1) trying Rule 9 + Rule 9 stacked:
(2) trying Rule 3 + Rule 3 stacked:
ask environment?  machines
trying Rule 3:
(3) trying Rule 9 + Rule 9 stacked:
(4) trying Rule 6 + Rule 6 stacked:
ask job  repairing
trying Rule 6:
(5) trying Rule 9 + Rule 9 stacked:
ask feedback  required
trying Rule 9

119. Lecture 4 Fuzzy expert systems: Fuzzy logic
Introduction, or what is fuzzy thinking?
Fuzzy sets
Linguistic variables and hedges
Operations of fuzzy sets
Fuzzy rules
Summary

120. Introduction, or what is fuzzy thinking?
Experts rely on common sense when they solve problems.
How can we represent expert knowledge that uses vague and ambiguous terms in a computer?
Fuzzy logic is not logic that is fuzzy, but logic that is used to describe fuzziness. Fuzzy logic is the theory of fuzzy sets, sets that calibrate vagueness.
Fuzzy logic is based on the idea that all things admit of degrees. Temperature, height, speed, distance, beauty  all come on a sliding scale. The motor is running really hot. Tom is a very tall guy.

121. Boolean logic uses sharp distinctions
Boolean logic uses sharp distinctions. It forces us to draw lines between members of a class and non-members. For instance, we may say, Tom is tall because his height is 181 cm. If we drew a line at 180 cm, we would find that David, who is 179 cm, is small. Is David really a small man or we have just drawn an arbitrary line in the sand?
Fuzzy logic reflects how people think. It attempts to model our sense of words, our decision making and our common sense. As a result, it is leading to new, more human, intelligent systems.

122. Fuzzy, or multi-valued logic was introduced in the 1930s by Jan Lukasiewicz, a Polish philosopher. While classical logic operates with only two values 1 (true) and 0 (false), Lukasiewicz introduced logic that extended the range of truth values to all real numbers in the interval between 0 and 1. He used a number in this interval to represent the possibility that a given statement was true or false. For example, the possibility that a man 181 cm tall is really tall might be set to a value of It is likely that the man is tall. This work led to an inexact reasoning technique often called possibility theory.

123. Later, in 1937, Max Black published a paper called “Vagueness: an exercise in logical analysis”. In this paper, he argued that a continuum implies degrees. Imagine, he said, a line of countless “chairs”. At one end is a Chippendale. Next to it is a near-Chippendale, in fact indistinguishable from the first item. Succeeding “chairs” are less and less chair-like, until the line ends with a log. When does a chair become a log? Max Black stated that if a continuum is discrete, a number can be allocated to each element. He accepted vagueness as a matter of probability.

124. In 1965 Lotfi Zadeh, published his famous paper “Fuzzy sets”
In 1965 Lotfi Zadeh, published his famous paper “Fuzzy sets”. Zadeh extended the work on possibility theory into a formal system of mathematical logic, and introduced a new concept for applying natural language terms. This new logic for representing and manipulating fuzzy terms was called fuzzy logic, and Zadeh became the Master of fuzzy logic.

125. Why fuzzy?
As Zadeh said, the term is concrete, immediate and descriptive; we all know what it means. However, many people in the West were repelled by the word fuzzy, because it is usually used in a negative sense.
Why logic?
Fuzziness rests on fuzzy set theory, and fuzzy logic is just a small part of that theory.

126. Fuzzy logic is a set of mathematical principles for knowledge representation based on degrees of membership.
Unlike two-valued Boolean logic, fuzzy logic is multi-valued. It deals with degrees of membership and degrees of truth. Fuzzy logic uses the continuum of logical values between 0 (completely false) and 1 (completely true). Instead of just black and white, it employs the spectrum of colours, accepting that things can be partly true and partly false at the same time.

127. Range of logical values in Boolean and fuzzy logic

128. Fuzzy sets The concept of a set is fundamental to mathematics.
However, our own language is also the supreme expression of sets. For example, car indicates the set of cars. When we say a car, we mean one out of the set of cars.

129. Crisp set theory is governed by a logic that uses one of only two values: true or false. This logic cannot represent vague concepts, and therefore fails to give the answers on the paradoxes.
The basic idea of the fuzzy set theory is that an element belongs to a fuzzy set with a certain degree of membership. Thus, a proposition is not either true or false, but may be partly true (or partly false) to any degree. This degree is usually taken as a real number in the interval [0,1].

130. The classical example in fuzzy sets is tall men
The classical example in fuzzy sets is tall men. The elements of the fuzzy set “tall men” are all men, but their degrees of membership depend on their height.

131. Crisp and fuzzy sets of “tall men”

132. The x-axis represents the universe of discourse  the range of all possible values applicable to a chosen variable. In our case, the variable is the man height. According to this representation, the universe of men’s heights consists of all tall men.
The y-axis represents the membership value of the fuzzy set. In our case, the fuzzy set of “tall men” maps height values into corresponding membership values.

133. A fuzzy set is a set with fuzzy boundaries
Let X be the universe of discourse and its elements be denoted as x. In the classical set theory, crisp set A of X is defined as function fA(x) called the characteristic function of A
fA(x): X  {0, 1}, where
This set maps universe X to a set of two elements. For any element x of universe X, characteristic function fA(x) is equal to 1 if x is an element of set A, and is equal to 0 if x is not an element of A.

134. In the fuzzy theory, fuzzy set A of universe X is defined by function A(x) called the membership function of set A
A(x): X  [0, 1], where A(x) = 1 if x is totally in A;
A(x) = 0 if x is not in A;
0 < A(x) < 1 if x is partly in A.
This set allows a continuum of possible choices. For any element x of universe X, membership function A(x) equals the degree to which x is an element of set A. This degree, a value between 0 and 1, represents the degree of membership, also called membership value, of element x in set A.

135. How to represent a fuzzy set in a computer?
First, we determine the membership functions. In our “tall men” example, we can obtain fuzzy sets of tall, short and average men.
The universe of discourse  the men’s heights  consists of three sets: short, average and tall men. As you will see, a man who is 184 cm tall is a member of the average men set with a degree of membership of 0.1, and at the same time, he is also a member of the tall men set with a degree of 0.4.

136. Crisp and fuzzy sets of short, average and tall men

137. Representation of crisp and fuzzy subsets
Typical functions that can be used to represent a fuzzy set are sigmoid, Gaussian and pi. However, these functions increase the time of computation. Therefore, in practice, most applications use linear fit functions (shown on P18).

138. For example, the fuzzy set of ‘tall men’ shown on P18 can be represented as a fit-vector
Average men ={0/165, 1/175, 0/185}
Short men ={1/160, 0.5/165, 0/170}

139. Linguistic variables and hedges
At the root of fuzzy set theory lies the idea of linguistic variables.
A linguistic variable is a fuzzy variable. For example, the statement “John is tall” implies that the linguistic variable John takes the linguistic value tall.

140. In fuzzy expert systems, linguistic variables are used in fuzzy rules
In fuzzy expert systems, linguistic variables are used in fuzzy rules. For example:
IF wind is strong
THEN sailing is good
IF project_duration is long
THEN completion_risk is high
IF speed is slow
THEN stopping_distance is short

141. The range of possible values of a linguistic variable represents the universe of discourse of that variable. For example, the universe of discourse of the linguistic variable speed might have the range between 0 and 220 km/h and may include such fuzzy subsets as very slow, slow, medium, fast, and very fast.
A linguistic variable carries with it the concept of fuzzy set qualifiers, called hedges.
Hedges are terms that modify the shape of fuzzy sets. They include adverbs such as very, somewhat, quite, more or less and slightly.

142. Fuzzy sets with the hedge very

143. Representation of hedges in fuzzy logic

144. Representation of hedges in fuzzy logic (continued)

145. Operations of fuzzy sets
The classical set theory developed in the late 19th
century by Georg Cantor describes how crisp sets can
interact. These interactions are called operations.

146. Cantor’s sets

147. Complement Crisp Sets: Who does not belong to the set?
Fuzzy Sets: How much do elements not belong to the set?
The complement of a set is an opposite of this set. For example, if we have the set of tall men, its complement is the set of NOT tall men. When we remove the tall men set from the universe of discourse, we obtain the complement. If A is the fuzzy set, its complement A can be found as follows:
A(x) = 1  A(x)

148. Containment Crisp Sets: Which sets belong to which other sets?
Fuzzy Sets: Which sets belong to other sets?
Similar to a Chinese box, a set can contain other sets. The smaller set is called the subset. For example, the set of tall men contains all tall men; very tall men is a subset of tall men. However, the tall men set is just a subset of the set of men. In crisp sets, all elements of a subset entirely belong to a larger set. In fuzzy sets, however, each element can belong less to the subset than to the larger set. Elements of the fuzzy subset have smaller memberships in it than in the larger set.

149. Intersection Crisp Sets: Which element belongs to both sets?
Fuzzy Sets: How much of the element is in both sets?
In classical set theory, an intersection between two sets contains the elements shared by these sets. For example, the intersection of the set of tall men and the set of fat men is the area where these sets overlap. In fuzzy sets, an element may partly belong to both sets with different memberships. A fuzzy intersection is the lower membership in both sets of each element. The fuzzy intersection of two fuzzy sets A and B on universe of discourse X:
AB(x) = min [A(x), B(x)] = A(x)  B(x),
where xX

150. Union Crisp Sets: Which element belongs to either set?
Fuzzy Sets: How much of the element is in either set?
The union of two crisp sets consists of every element that falls into either set. For example, the union of tall men and fat men contains all men who are tall OR fat. In fuzzy sets, the union is the reverse of the intersection. That is, the union is the largest membership value of the element in either set. The fuzzy operation for forming the union of two fuzzy sets A and B on universe X can be given as:
AB(x) = max [A(x), B(x)] = A(x)  B(x),
where xX

151. Operations of fuzzy sets

152. Fuzzy rules
In 1973, Lotfi Zadeh published his second most influential paper. This paper outlined a new approach to analysis of complex systems, in which Zadeh suggested capturing human knowledge in fuzzy rules.

153. What is a fuzzy rule? IF x is A THEN y is B
A fuzzy rule can be defined as a conditional statement in the form:
IF x is A
THEN y is B
where x and y are linguistic variables; and A and B are linguistic values determined by fuzzy sets on the universe of discourses X and Y, respectively.

154. What is the difference between classical and fuzzy rules?
A classical IF-THEN rule uses binary logic, for
example,
Rule: 1
IF speed is > 100
THEN stopping_distance is long
Rule: 2
IF speed is < 40
THEN stopping_distance is short
The variable speed can have any numerical value between 0 and 220 km/h, but the linguistic variable stopping_distance can take either value long or short. In other words, classical rules are expressed in the black-and-white language of Boolean logic.

155. We can also represent the stopping distance rules in a fuzzy form:
IF speed is fast
THEN stopping_distance is long
Rule: 2
IF speed is slow
THEN stopping_distance is short
In fuzzy rules, the linguistic variable speed also has the range (the universe of discourse) between 0 and 220 km/h, but this range includes fuzzy sets, such as slow, medium and fast. The universe of discourse of the linguistic variable stopping_distance can be between 0 and 300 m and may include such fuzzy sets as short, medium and long.

156. Fuzzy rules relate fuzzy sets.
In a fuzzy system, all rules fire to some extent, or in other words they fire partially. If the antecedent is true to some degree of membership, then the consequent is also true to that same degree.

157. Fuzzy sets of tall and heavy men
These fuzzy sets provide the basis for a weight estimation model. The model is based on a relationship between a man’s height and his weight:
IF height is tall
THEN weight is heavy

158. The value of the output or a truth membership grade of the rule consequent can be estimated directly from a corresponding truth membership grade in the antecedent. This form of fuzzy inference uses a method called monotonic selection.

159. A fuzzy rule can have multiple antecedents, for example:
IF project_duration is long
AND project_staffing is large
AND project_funding is inadequate
THEN risk is high
IF service is excellent
OR food is delicious
THEN tip is generous

160. THEN hot_water is reduced; cold_water is increased
The consequent of a fuzzy rule can also include multiple parts, for instance:
IF temperature is hot
THEN hot_water is reduced;
cold_water is increased