1.  2003, G.Tecuci, Learning Agents Laboratory 1 Learning Agents Laboratory Computer Science Department George Mason University Prof. Gheorghe Tecuci 1. Introduction

2.  2003, G.Tecuci, Learning Agents Laboratory 2 Overview What is Machine Learning History of Machine Learning Basic bibliography and reading What is Artificial Intelligence Demo: Disciple learning agent

3.  2003, G.Tecuci, Learning Agents Laboratory 3 Artificial Intelligence is the Science and Engineering that is concerned with the theory and practice of developing systems that exhibit the characteristics we associate with intelligence in human behavior: perception, natural language processing, reasoning, planning and problem solving, learning and adaptation, etc. What is Artificial Intelligence

4.  2003, G.Tecuci, Learning Agents Laboratory 4 Central goals of Artificial Intelligence Understand the principles that make intelligence possible (in humans, animals, and artificial agents) Developing intelligent machines or agents (no matter whether they operate as humans or not) Formalizing knowledge and mechanizing reasoning in all areas of human endeavor Making the working with computers as easy as working with people Developing human-machine systems that exploit the complementariness of human and automated reasoning

5.  2003, G.Tecuci, Learning Agents Laboratory 5 What is an intelligent agent Intelligent Agent user/ environment output/ sensors effectors input/ An intelligent agent is a system that: perceives its environment (which may be the physical world, a user via a graphical user interface, a collection of other agents, the Internet, or other complex environment); reasons to interpret perceptions, draw inferences, solve problems, and determine actions; and acts upon that environment to realize a set of goals or tasks for which it was designed.

6.  2003, G.Tecuci, Learning Agents Laboratory 6 Characteristic features of intelligent agents Knowledge representation and reasoning Transparency and explanations Ability to communicate Use of huge amounts of knowledge Exploration of huge search spaces Use of heuristics Reasoning with incomplete or conflicting data Ability to learn and adapt

7.  2003, G.Tecuci, Learning Agents Laboratory 7 Overview What is Machine Learning History of Machine Learning Basic bibliography and reading What is Artificial Intelligence Demo: Disciple learning agent

8.  2003, G.Tecuci, Learning Agents Laboratory 8 Machine Learning is the domain of Artificial Intelligence which is concerned with building adaptive computer systems that are able to improve their competence and/or efficiency through learning from input data or from their own problem solving experience. What is Machine Learning

9.  2003, G.Tecuci, Learning Agents Laboratory 9 The architecture of a learning agent Ontology Rules/Cases/Methods Problem Solving Engine Learning Agent User/ Environment Output/ Sensors Effectors Input/ Knowledge Base Learning Engine Implements learning methods for extending and refining the knowledge base to improve agent’s competence and/or efficiency in problem solving. Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the input and provide an appropriate output. Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc.

10.  2003, G.Tecuci, Learning Agents Laboratory 10 What is Learning? Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more effectively the next time (Simon, 1983). Learning is making useful changes in our minds (Minsky, 1985). Learning is constructing or modifying representations of what is being experienced (Michalski, 1986). A computer program learns if it improves its performance at some task through experience (Mitchell, 1997).

11.  2003, G.Tecuci, Learning Agents Laboratory 11 So what is Learning? (1)Acquire, discover, and organize knowledge (by building, modifying and organizing internal representations of some external reality); (2)Acquire skills (by gradually improving their motor or cognitive skills through repeated practice, sometimes involving little or no conscious thought). Learning results in changes in the agent (or mind) that improve its competence and/or efficiency. Learning is a very general term denoting the way in which people and computers:

12.  2003, G.Tecuci, Learning Agents Laboratory 12 Two complementary dimensions for learning A system is improving its competence if it learns to solve a broader class of problems, and to make fewer mistakes in problem solving. A system is improving its efficiency, if it learns to solve the problems from its area of competence faster or by using fewer resources. Competence Efficiency

13.  2003, G.Tecuci, Learning Agents Laboratory 13 Main directions of research in Machine Learning Discovery of general principles, methods, and algorithms of learning Automation of the construction of knowledge-based systems

14.  2003, G.Tecuci, Learning Agents Laboratory 14 Learning strategies Rote learning Learning from instruction Learning from examples Explanation-based learning Conceptual clustering Quantitative discovery Abductive learning Learning by analogy Instance-based learning Reinforcement learning Neural networks Genetic algorithms and evolutionary computation Reinforcement learning Bayesian learning Multistrategy learning A Learning Strategy is a basic form of learning characterized by the employment of a certain type of inference (e.g. deduction, induction or analogy), a certain type of computational or representational mechanism (e.g. rules, trees, neural networks, etc.), and a certain type of learning goal (e.g. learn a concept, discover a formula, acquire new knowledge about an entity, refine an entity).

15.  2003, G.Tecuci, Learning Agents Laboratory 15 Overview What is Machine Learning History of Machine Learning What is Artificial Intelligence Basic bibliography and reading Demo: Disciple learning agent

16.  2003, G.Tecuci, Learning Agents Laboratory 16 History of Machine Learning Early enthusiasm (1955 - 1965) Learning without knowledge; Neural modeling (self-organizing systems and decision space techniques); Evolutionary learning; Rote learning (Samuel Checker’s player).

17.  2003, G.Tecuci, Learning Agents Laboratory 17 History of Machine Learning (cont.) Dark ages (1962 - 1976) To acquire knowledge one needs knowledge; Realization of the difficulty of the learning process and of the limitations of the explored methods (e.g. the perceptron cannot learn the XOR function); Symbolic concept learning (Winston’s influential thesis, 1972).

18.  2003, G.Tecuci, Learning Agents Laboratory 18 History of Machine Learning (cont.) Renaissance (1976 - 1988) Exploration of different strategies (EBL, CBR, GA, NN, Abduction, Analogy, etc.); Knowledge-intensive learning; Successful applications; Machine Learning conferences/workshops worldwide.

19.  2003, G.Tecuci, Learning Agents Laboratory 19 History of Machine Learning (cont.) Maturity (1988 - present) Experimental comparisons; Revival of non-symbolic methods; Computational learning theory; Multistrategy learning; Integration of machine learning and knowledge acquisition; Emphasis on practical applications.

20.  2003, G.Tecuci, Learning Agents Laboratory 20 Successful applications of Machine Learning Learning to recognize spoken words (all of the most successful systems use machine learning); Learning to drive an autonomous vehicle on public highway; Learning to classify new astronomical structures (by learning regularities in a very large data base of image data); Learning to play games; Automation of knowledge acquisition from domain experts; Learning agents.

21.  2003, G.Tecuci, Learning Agents Laboratory 21 Overview What is Machine Learning History of Machine Learning What is Artificial Intelligence Basic bibliography and reading Demo: Disciple learning agent

22.  2003, G.Tecuci, Learning Agents Laboratory 22 Disciple approach to agent development Disciple is a theory, methodology and agent shell for rapid development of end to end knowledge bases and agents, by subject matter experts, with limited assistance from knowledge engineers DISCIPLE RKF/COG Interface Problem Solving Learning Ontology + Rules The expert teaches Disciple in a way that resembles how the expert would teach a person. Disciple learns from the expert, building, verifying and improving its knowledge base

23.  2003, G.Tecuci, Learning Agents Laboratory 23 Vision on the evolution of the software development process Mainframe Computers Software systems developed and used by computer experts Personal Computers Software systems developed by computer experts and used by persons that are not computer experts Learning Agents Software systems developed and used by persons that are not computer experts

24.  2003, G.Tecuci, Learning Agents Laboratory 24 Vision on the use of Disciple in Education teaches Disciple Agent KB The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student. teaches Disciple Agent KB teaches Disciple Agent KB … Disciple tutors the student in a way that is similar to how the expert/teacher has taught it. teaches Disciple Agent KB

25.  2003, G.Tecuci, Learning Agents Laboratory 25 An intelligent agent for Center of Gravity analysis The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed. Carl Von Clausewitz, “On War,” 1832. If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. (Giles and Galvin, USAWC 1996).

26.  2003, G.Tecuci, Learning Agents Laboratory 26 Government Military People Economy Alliances Etc. Which are the critical capabilities? Are the critical requirements of these capabilities satisfied? If not, eliminate the candidate. If yes, do these capabilities have any vulnerability? Based on the concepts of critical capabilities, critical requirements and critical vulnerabilities, which have been recently adopted into the joint military doctrine of USA (Strange, 1996). Applied to current war scenarios (e.g. War on terror 2003, Iraq 2003) with state and non-state actors (e.g. Al Qaeda). Identify potential primary sources of moral or physical strength, power and resistance from: Test each identified COG candidate to determine whether it has all the necessary critical capabilities: Identification of COG candidatesTesting of COG candidates Approach to Center of Gravity (COG) analysis

27.  2003, G.Tecuci, Learning Agents Laboratory 27 leader people military communicate stay informed be protected have support be a driving force be influential be irreplaceable support the goal communicate desires to the highest level leadership receive communication from the highest level leadership have a positive impact support the highest level leadership be influential be indispensable exert power be deployable Critical capabilities needed to be a COG industrial capacity financial capacity ideology external support will of multi member force

28.  2003, G.Tecuci, Learning Agents Laboratory 28 Critical capability to communicate stay informed be protected have support be a driving force be influential be irreplaceable Leader who is a COG Corresponding critical requirement Have means to communicate with the government, the military and the people Have means to receive essential intelligence Have means to be protected from all threats Have means to secure continuous support from the government, the military and the people Have reasons and determination for pursuing the goal Have means to influence the government, the military and the people Be the only leader to maintain the goal

29.  2003, G.Tecuci, Learning Agents Laboratory 29 Critical capability to be protected Illustration: Saddam Hussein (Iraq 2003) Corresponding critical requirement Have means to be protected from all threats Means Republican Guard Protection Unit  loyalty not based on conviction and can be influenced by US-led coalition Vulnerabilities Complex of Iraqi Bunkers  location known to US led coalition  design known to US led coalition  can be destroyed by US-led coalition Iraqi Military  loyalty can be influenced by US-led coalition  can be destroyed by US-led coalition System of Saddam Doubles  loyalty of Saddam Doubles to Saddam can be influenced by US-led coalition 

30.  2003, G.Tecuci, Learning Agents Laboratory 30 Demonstration Disciple Demo Teaching Disciple how to determine whether a strategic leader has the critical capability to be protected.

31.  2003, G.Tecuci, Learning Agents Laboratory 31 Basic bibliography Mitchell T.M., Machine Learning, McGraw Hill, 1997. Shavlik J.W. and Dietterich T. (Eds.), Readings in Machine Learning, Morgan Kaufmann, 1990. Buchanan B., Wilkins D. (Eds.), Readings in Knowledge Acquisition and Learning: Automating the Construction and the Improvement of Programs, Morgan Kaufmann, 1992. Langley P., Elements of Machine Learning, Morgan Kaufmann, 1996. Michalski R.S., Carbonell J.G., Mitchell T.M. (Eds), Machine Learning: An Artificial Intelligence Approach, Morgan Kaufmann, 1983 (Vol. 1), 1986 (Vol. 2). Kodratoff Y. and Michalski R.S. (Eds.) Machine Learning: An Artificial Intelligence Approach (Vol. 3), Morgan Kaufmann Publishers, Inc., 1990. Michalski R.S. and Tecuci G. (Eds.), Machine Learning: A Multistrategy Approach (Vol. 4), Morgan Kaufmann Publishers, San Mateo, CA, 1994. Tecuci G. and Kodratoff Y. (Eds.), Machine Learning and Knowledge Acquisition: Integrated Approaches, Academic Press, 1995. Tecuci G., Building Intelligent Agents: An Apprenticeship Multistrategy Learning Theory, Methodology, Tool and Case Studies, Academic Press, 1998.

32.  2003, G.Tecuci, Learning Agents Laboratory 32 Recommended reading Mitchell T.M., Machine Learning, Chapter 1: Introduction, pp. 1-19, McGraw Hill, 1997. Tecuci G., Boicu M., Marcu D., Stanescu B., Boicu C., Comello J., Training and Using Disciple Agents: A Case Study in the Military Center of Gravity Analysis Domain, in AI Magazine, 24, 4, 2002, pp.51-68, AAAI Press, Menlo Park, California, 2002, http://lalab.gmu.edu/publications/default.htm