21 September 2005 Gheorghe Tecuci Learning Agents Center and Computer Science Department School of Information Technology and Engineering George Mason University

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

Conducts fundamental and experimental research on the development of knowledge- based learning and problem solving agents. Supports teaching in the areas of intelligent agents, machine learning, knowledge acquisition, artificial intelligence and its applications. Develops the Disciple theory, methodology and agent shells for building agents that can be taught how to solve problems by subject matter experts. Mission Basic Research Tools Applications Transitions

Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, Building an intelligent machine by programming is too difficult. “Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain.” Teaching as Alternative to Programming

How are Expert Systems Built and Why it is Hard Edward Feigenbaum, 1993: Rarely does a technology arise that offers such a wide range of important benefits.

Research Problem and Approach

Disciple’s Vision on the Future of Software Development Mainframe Computers Personal Computers Learning Agents

Vision on the Use of Disciple in Education teaches Disciple Agent KB teaches Disciple Agent KB teaches Disciple Agent KB … teaches Disciple Agent KB  2005, Learning Agents Center

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

 2005, Learning Agents Center The Overall Architecture of a Disciple Agent

Knowledge Base = Ontology + Rules ONTOLOGY FRAGMENT Main condition ?O1isPhD_advisor has_as_employer?O4 has_as_position?O5 ?O2isPhD_student ?O3isresearch_area ?O4isuniversity ?O5istenured_position Except when condition ?O1isperson is_likely_to_move_to?O6 ?O6isemployer IF: Determine whether ?O1 can be a PhD advisor for ?O2 in ?O3. THEN: Determine whether ?O1 would be a good PhD advisor for ?O2 in ?O3. REASONING RULE Determine whether John Smith can be a PhD advisor for Tom Even in Artificial Intelligence. PROBLEM SOLVING TASK

Model the reasoning of SME Create object ontology Define reasoning rules Verify and update rules KE SME Traditionally KE Agent SMEAgent SME Specify instances and features Learn ontological elements Import and develop initial ontology Agent Learn reasoning rules SME Agent Define and explain examples SME AgentSMEAgent Critique examples Refine rules Explain critiques SME Agent Develop reasoning trees SME KE Instruct SME to explain reasoning With Disciple Main Idea of the Disciple Approach Determine whether John Smith can be a PhD advisor for Tom Even in Artificial Intelligence.

Research Issues for Learning Agents Agent Architecture for Generality-Power Tradeoff Learning with an Evolving Representation Language Problem Solving Paradigm for Expert-Agent Collaboration Knowledge Base Structuring for Knowledge Reuse Plausible Reasoning with Partially Learned Knowledge Integrated Teaching and Learning Multistrategy Learning

S 1 S 11a S 1n S 11b1 S 11bm T 11bm T 11b1 T 1n T 11a … … T1T1 Q1Q1 S 11b T 11b A 1n S 11 A 11 … … A 11b1 A 11bm S 11b Q 11b Task reduction and solution composition guided by questions and answers Problem Solving Paradigm for Expert-Agent Collaboration

Learning with an Evolving Representation Language

Plausible Reasoning with Partially Learned Knowledge IF THEN … Plausible Lower Bound Condition Plausible Upper Bound Condition

Mixed-Initiative Problem Solving Creative solutions Inventive solutions Innovative solutions Routine solutions Problem Solution

Integrated Teaching and Learning Input knowledge Problem solving behavior Explicit learning guidance Explicit teaching guidance learning hints examples, facts, rules classification of examples, problem solutions questions

Rule Learning Method Example of a task reduction step Plausible version space rule analogy PLB PUB Knowledge Base Incomplete explanation Analogy and Hint Guided Explanation Analogy-based Generalization

Multistrategy Learning

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

Challenges for the Intelligence Analyst Overwhelmed by information Difficult to share intelligence A P Difficult to consider multiple hypotheses Difficult to collaborate with other analysts and experts Difficult to avoid the analytic mindset Difficult to analyze in reference to the culture of the data source Difficult to train new analysts Difficult to find time for critical analysis and AARs Difficult to acquire and retain expertise Knowledge Difficult to rigurously explain the analysis Intelligence analysis is very difficult H1H1 HnHn

An integrated approach to intelligence analysis research, education, and operations. Investigated Solution Develop a new type of intelligent agent that can rapidly acquire expertise in intelligence analysis, can train new intelligence analysts, and can assist the analysts to solve complex problems.

Vision: Integration of Research, Education, and Operations Agent Lifecycle DISCIPLE-LTA Building an agent shell 1 Knowledge engineer 2 Agent training by expert analyst DISCIPLE-LTA Expert analyst and knowledge engineer Rapid agent development DISCIPLE-LTA 6 Knowledge base optimization and re-use Knowledge engineer and expert analyst Agent optimization 3 DISCIPLE-LTA Intelligent tutoring Teaching new analysts Analyst Agent use and non-disruptive learning 4 DISCIPLE-LTA Analyst’s assistant (mixed-initiative analysis) Analyst After action review and agent personalization 5 DISCIPLE-LTA Analyst’s assistant (mixed-initiative learning) Analyst

Disciple-LTA Intelligent agent Disciple-LTA Vision: Use of Disciple-LTA Agents in an Operational Environment Disciple-LTA GLOBAL KNOWLEDGE BASE Disciple Client Libraries Knowledge Repositories Massive Databases SEARCH ENGINES

Intelligent Agents Research Military Research Military Education & Practice Disciple LTA Develop a systematic approach to military intelligence analysis Experimentation with Disciple-LTA in the 589 MAAI elective Agent development by expert analysts using learning agent technology Synergistic Integration of Research and Education  2005, Learning Agents Center Working closely with the expert analysts in a multi- disciplinary research Working closely with the end user to receive crucial and timely feedback

Live Experiment US Army War College Course 589 Military Applications of Artificial Intelligence: Intelligence Analysis

Assess whether Location-A is a training base for terrorist operations

What type of factors should be considered to assess the presence of a terrorist training base?

Political environment, physical structures, flow of suspected terrorists, weapons and weapons technology, other suspected bases in the region, and terrorist sympathetic population

Assess whether the political environment would support a training base for terrorist operations at Location-A Assess whether there is a flow of suspected terrorists in the region of Location-A Assess whether there are other suspected bases for terrorist operations in the region of Location-A Assess whether the physical structures at Location-A support the existence of a training base for terrorist operations Assess whether there are weapons and weapons technology at Location-A that suggest the presence of a training base for terrorist operations Assess whether there is terrorist sympathetic population in the region of Location-A

Assess whether the political environment would support a training base for terrorist operations at Location-A Assess whether there is a flow of suspected terrorists in the region of Location-A Assess whether there are other suspected bases for terrorist operations in the region of Location-A Assess whether the physical structures at Location-A support the existence of a training base for terrorist operations Assess whether there are weapons and weapons technology at Location-A that suggest the presence of a training base for terrorist operations Assess whether there is terrorist sympathetic population in the region of Location-A

REVIEWER #2: This is an innovative idea that could revolutionize the way we do business, enable us to be more efficient, more effective, more thorough. REVIEWER #1: a grand challenge to develop an intelligent agent capable of learning, tutoring and decision support … if implemented it would likely be pretty unique. Intelligence Experts Opinion: Quotations REVIEWER #3: a very important R&D area for next generation intelligence analysis. The work is well founded, and the execution of real software to implement the ideas is substantial. REVIEWER #4: I have seen a briefing on the work presented here last year and was impressed with the initial ease of use of capturing complex concepts. This could be excellent for use in both training analysts as well as capturing knowledge from more senior analysts.

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

Integrated KB KB1 Disciple-RKF Assistant Disciple-RKF Assistant Problem solver for a non-expert Tutor to a student Assistant of an expert KBn Disciple-RKF Assistant... Expert Successful experiments and transition to the US Army War College DARPA’s Rapid Knowledge Formation Program Develop the Disciple technology to enable teams of subject matter experts to build integrated knowledge bases and agents incorporating their problem solving expertise.

 2005, Learning Agents Center If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. P.K. Giles and T.P. Galvin US Army War College, Center of Gravity Analysis The center of gravity of an entity is its primary source of moral or physical strength, power or resistance. Joe Strange, Centers of Gravity & Critical Vulnerabilities, Marine Corps War College, 1996.

589jw Military Applications of Artificial Intelligence Students teach Disciple their COG analysis expertise, using sample scenarios (e.g. Iraq 2003, War on terror 2003, Arab-Israeli 1973) Students test the trained Disciple agent based on a new scenario (North Korea 2003) I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer Spring 2001 COG identification Spring 2002 COG identification and testing Spring 2003 COG testing based on critical capabilities Global evaluations of Disciple by officers during three experiments Use of Disciple at the US Army War College

 2005, Learning Agents Center Disciple Agent KB Problem solving Disciple was taught based on the expertise of Prof. Comello in center of gravity analysis. Disciple helps the students to perform a center of gravity analysis of an assigned war scenario. Teaching Learning The use of Disciple is an assignment that is well suited to the course's learning objectives Disciple should be used in future versions of this course Use of Disciple at the US Army War College 319jw Case Studies in Center of Gravity Analysis Disciple helped me to learn to perform a strategic COG analysis of a scenario Global evaluations of Disciple by officers from the Spring 05 course

Extended KB stay informed be irreplaceable communicate be influential Integrated KB Initial KB have support be protected be driving force 432 concepts and features, 29 tasks, 18 rules For COG identification for leaders 37 acquired concepts and features for COG testing COG identification and testing (leaders) Domain analysis and ontology development (KE+SME) Parallel KB development (SME assisted by KE) KB merging (KE) Knowledge Engineer (KE) All subject matter experts (SME) DISCIPLE-COG Training scenarios: Iraq 2003 Arab-Israeli 1973 War on Terror 2003 Team 1 Team 2Team 3Team 4Team 5 5 features 10 tasks 10 rules Learned features, tasks, rules 14 tasks 14 rules 2 features 19 tasks 19 rules 35 tasks 33 rules 3 features 24 tasks 23 rules Unified 2 features Deleted 4 rules Refined 12 rules Final KB: +9 features  478 concepts and features +105 tasks  134 tasks +95 rules  113 rules DISCIPLE-COG Testing scenario: North Korea 2003 Correctness = 98.15% 5h 28min average training time / team 3.53 average rule learning rate / team Parallel development and merging of KBs

KB INTEGRATION ASSISTANT Integration Team: Knowledge engineer + Subject matter experts KB Integration, Validation and Maintenance PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT Current Project Distributed Knowledge Acquisition, Validation, and Maintenance Copies of Disciple agents support users’ decision-making and all learn from these experiences. Knowledge acquired by the agents is validated and integrated into an improved Disciple Knowledge Base

PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT PROBLEM SOLVING AND LEARNING ASSISTANT Operational Use and Non-Disruptive Learning After Action Review and KB Refinement PROBLEM SOLVING AND LEARNING ASSISTANT Co-PI, SME Dr. Jerome Comello Experiments in 2005, 2006, 2007 Co-PI, SME Dr. Joseph Strange Experiments in 2006, 2007 Co-PI, SME Col Jeffrey Hightaian LtCol Todd Kemper Experiments in 2006, 2007 Experimentation Environment 2005, 2006, 2007 Air War College Army War College Marine Corps War College KB INTEGRATION ASSISTANT Integration Team: Knowledge engineer + Subject matter experts KB Integration, Validation and Maintenance George Mason University

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

 2005, Learning Agents Center Disciple-VE Virtual Experts for Multi-Domain Collaborative Planning User’s Assistant Virtual Team Manager Scenario Specification KB DISTRIBUTED KNOWLEDGE BASE KB Report Generator Plan Browser Assistant Training Modules Plan Abstraction Virtual Experts (VE) Library Disciple-VE Knowledge Management Local Knowledge Base OntologyRules Profile-based Team Selector Disciple-VE External- Expertise Agent Plan Grading Plan Brainstorming Collaborative Planner VE Training Modules Assumption-based Reasoning Knowledge Management Local Knowledge Base OntologyRules Indicators Identification Disciple-VE Team of Virtual Experts Disciple-VE User Sample scenario: Planning the response to an emergency situation involving a tanker truck leaking red-fuming nitric acid near a student residential area of GMU.

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

Rapid development and evaluation of a Course of Action critiquer DARPA’s HPKB Challenge Problem To what extent does this course of action conform to the principle of surprise?

DARPA’s HPKB Program: Evaluation High knowledge acquisition rate; Better results than the other evaluated systems; Better performance than the evaluating experts (many unanticipated solutions). 100%

Personal Cognitive Assistant for Intelligence Analysis Virtual Experts for Multi-domain Collaborative Planning Learning Agents Center: Research Vision Research Issues for Learning Agents Agent for Course of Action Critiquing Overview Agents for Centers of Gravity and Critical Vulnerabilities Final Remarks

 2005, Learning Agents Center Disciple’s Vision on the Future of Software Development Mainframe Computers Software systems developed and used by computer experts Personal Computers Software systems developed by computer experts and used by persons that are not computer experts Learning Agents Software systems developed and used by persons that are not computer experts

 2005, Learning Agents Center Vision on the Use of Disciple in Education teaches Disciple Agent KB The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student. teaches Disciple Agent KB teaches Disciple Agent KB … Disciple tutors the student in a way that is similar to how the expert/teacher has taught it. teaches Disciple Agent KB  2005, Learning Agents Center

The research performed in the Learning Agents Center was sponsored by several US government agencies including Defense Advanced Research Projects Agency, Air Force Office of Scientific Research, Air Force Research Laboratory, National Science Foundation, and Army War College. Acknowledgements

 2005, Learning Agents Center Questions