Presentation is loading. Please wait.

Presentation is loading. Please wait.

IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 1 Learning Hierarchical Task Networks by Analyzing Expert Traces Pat Langley.

Published byKevin Ritchie Modified over 10 years ago

Similar presentations

Presentation on theme: "IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 1 Learning Hierarchical Task Networks by Analyzing Expert Traces Pat Langley."— Presentation transcript:

1 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 1 Learning Hierarchical Task Networks by Analyzing Expert Traces Pat Langley Tolga Konik Negin Nejati Institute for the Study of Learning and Expertise Palo Alto, California

2 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 2 Formulation of the Learning Task Given: A set of domain operators with known effects A worked out problem solution that consists of The goal to be achieved in the problem A sequence of operator instances that achieves the goal A related sequence of intermediate problem states Find: A hierarchical task network that Reproduces the solution to the training problem Generalizes well to related problems in the domain

3 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 3 The I CARUS Architecture ConceptualMemory BeliefMemory Goal/IntentionMemory ConceptualInference SkillExecution Perception Environment PerceptualBuffer Skill Learning MotorBuffer Skill Retrieval and Selection Skill Memory

4 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 4 Representing Long-Term Structures Conceptual clauses: A set of relational inference rules with perceived objects or defined concepts in their antecedents; Skill clauses: A set of executable skills that specify: a head that indicates a goal the skill achieves; a single (typically defined) precondition; a set of ordered subgoals or actions for achieving the goal. These define a specialized class of hierarchical task networks in a syntax very similar to Nau et al.s SHOP2 formalism. Beliefs, goals, and intentions are instances of these structures. I CARUS encodes two forms of general long-term knowledge:

5 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 5 Representing Concepts (Axioms) ((in-rightmost-lane ?self ?clane) :percepts ((self ?self) (segment ?seg) (line ?clane segment ?seg)) :relations ((driving-well-in-segment ?self ?seg ?clane) (last-lane ?clane) (not (lane-to-right ?clane ?anylane))) ) ((driving-well-in-segment ?self ?seg ?lane) :percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)) :relations ((in-segment ?self ?seg) (in-lane ?self ?lane) (aligned-with-lane-in-segment ?self ?seg ?lane) (centered-in-lane ?self ?seg ?lane) (steering-wheel-straight ?self)) ) ((in-lane ?self ?lane) :percepts ((self ?self segment ?seg) (line ?lane segment ?seg dist ?dist)) :tests ((> ?dist -10) (<= ?dist 0)) ) Primitive Concepts Nonprimitive Concepts

6 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 6 ((in-rightmost-lane ?self ?line) :percepts ((self ?self) (line ?line)) :start ((last-lane ?line)) :subgoals ((driving-well-in-segment ?self ?seg ?line)) ) ((driving-well-in-segment ?self ?seg ?line) :percepts ((segment ?seg) (line ?line) (self ?self)) :start ((steering-wheel-straight ?self)) :subgoals ((in-segment ?self ?seg) (centered-in-lane ?self ?seg ?line) (aligned-with-lane-in-segment ?self ?seg ?line) (steering-wheel-straight ?self)) ) ((in-segment ?self ?endsg) :percepts ((self ?self speed ?speed) (intersection ?int cross ?cross) (segment ?endsg street ?cross angle ?angle)) :start ((in-intersection-for-right-turn ?self ?int)) :actions(( steer 1)) ) Primitive Skill Clauses Nonprimitive Skill Clauses Representing Skills (Methods)

7 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 7 Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. concepts skills I CARUS organizes both concepts and skills in a hierarchical manner. Hierarchical Structure of Memory

8 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 8 Hierarchical Structure of Memory For example, the skill highlighted here refers directly to the highlighted concepts. I CARUS interleaves its long-term memories for concepts and skills. concepts skills

9 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 9 Basic I CARUS Processes Concepts are matched bottom up, starting from percepts. Skill paths are matched top down, starting from intentions. I CARUS matches patterns to recognize concepts and select skills. concepts skills

10 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 10 Impasse-Driven Analytical Learning Skill Hierarchy Reactive Execution Analytical Learning Experts Primitive Skill Sequence … Effects of Primitive skills Learned Skills If Impasse Problem ? Initial State Goal

11 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 11 Learning HTNs by Trace Analysis concepts primitive skills

12 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 12 Skill Chaining concepts primitive skills Learning HTNs by Trace Analysis

13 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 13 Concept Chaining concepts primitive skills Learning HTNs by Trace Analysis

14 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 14 unstack C B on B A hand-empty putdown C putdownable C unstackable B A clear A unstack B A clear B unstackable C B A B C A B CAC B A B C Constructing an Explanation concepts primitive skills

15 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 15 A B C unstack C B on B A hand-empty putdown C putdownable C unstackable B A clear A unstack B A clear B unstackable C B A B C A B CAC B concepts primitive skills From an Explanation to an HTN

16 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 16 on ?y ?x hand-empty putdown ?z putdownable ?z unstackable ?y ?x clear ?x clear ?y unstackable ?z ?y unstack ?z ?y A B C A B CAC B A B C unstack ?y ?x From an Explanation to an HTN concepts primitive skills

17 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 17 Key Ideas of the Approach Constrained form of hierarchical task networks Each skill clause/method has a goal as its head Each method has one (possibly defined) precondition The resulting semi-lattice makes learning tractable Learning involves analyzing the expert trace Explanation draws on a form of goal regression Each step in the explanation becomes an HTN method Similar to explanation-based learning for planning but retains the explanation structure

18 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 18 Related Research Nonincremental, knowledge-lean approaches Behavioral cloning (Sammut, 1996; Urbancic & Bratko 1994) Relational induction from traces (e.g., Reddy & Tadepalli, 1997) Incremental, knowledge-intensive approaches Explanation-based learning (e.g., Shavlik, 1989; Mooney, 1990) Derivational analogy (e.g., Veloso & Carbonell, 1993) Programming by demonstration (e.g., Lau et al., 2003)

19 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 19 Plans for Future Research Extend framework to use and learn partial-order skills Augment approach to use known subtasks during learning Extend method to learn skills with negated goals and subgoals Modify approach to handle partially observable traces Extend system to learn skills with uncertain outcomes

20 IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 20 End of Presentation

Download ppt "IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 1 Learning Hierarchical Task Networks by Analyzing Expert Traces Pat Langley."

Similar presentations

2 Introduction A central issue in supporting interoperability is achieving type compatibility. Type compatibility allows (a) entities developed by various.

2 Introduction A central issue in supporting interoperability is achieving type compatibility. Type compatibility allows (a) entities developed by various.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Arizona State University and Institute for the Study of Learning and Expertise Expertise, Transfer, and Innovation in.

Pat Langley Arizona State University and Institute for the Study of Learning and Expertise Expertise, Transfer, and Innovation in.
Pat Langley Seth Rogers Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.

Pat Langley Seth Rogers Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.
Pat Langley Institute for the Study of Learning and Expertise Palo Alto, CA Cumulative Learning of Relational and Hierarchical Skills.

Pat Langley Institute for the Study of Learning and Expertise Palo Alto, CA Cumulative Learning of Relational and Hierarchical Skills.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Varieties of Problem Solving in a Unified Cognitive Architecture.

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Varieties of Problem Solving in a Unified Cognitive Architecture.
Pat Langley Dan Shapiro Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Dan Shapiro Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Extending the I CARUS Cognitive Architecture Thanks to D. Choi,

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Extending the I CARUS Cognitive Architecture Thanks to D. Choi,
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Dongkyu Choi Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.

Pat Langley Dongkyu Choi Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Mental Simulation and Learning in the I CARUS Architecture.

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Mental Simulation and Learning in the I CARUS Architecture.
General learning in multiple domains transfer of learning across domains Generality and Transfer in Learning training items test items training items test.

General learning in multiple domains transfer of learning across domains Generality and Transfer in Learning training items test items training items test.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Modeling Social Cognition in a Unified Cognitive Architecture.

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Modeling Social Cognition in a Unified Cognitive Architecture.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Information Processing Technology Office Learning Workshop April 12, 2004 Seedling Overview Learning Hierarchical Reactive Skills from Reasoning and Experience.

Information Processing Technology Office Learning Workshop April 12, 2004 Seedling Overview Learning Hierarchical Reactive Skills from Reasoning and Experience.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona Institute for the Study of Learning and Expertise Palo Alto, California.

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona Institute for the Study of Learning and Expertise Palo Alto, California.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Institute for the Study of Learning and Expertise Palo Alto,

Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Institute for the Study of Learning and Expertise Palo Alto,

Similar presentations

Ads by Google