1 ISLE Transfer Learning Team Main Technology Components The I CARUS Architecture Markov Logic Networks Executes skills in the environment Long-TermConceptualMemoryShort-TermConceptualMemory Short-TermGoal/SkillMemory ConceptualInference SkillExecution Perception Environment PerceptualBuffer Problem Solving Skill Learning MotorBuffer Skill Retrieval Long-Term Skill Memory Contains relational and hierarchical knowledge about relevant concepts Generates beliefs using observed environment and long term conceptual knowledge Creates internal description of the perceived environment Contains descriptions of the perceived objects Contains inferred beliefs about the environment Contains hierarchical knowledge about executable skills Finds novel solutions for achieving goals Acquires skills from successful problem solving traces Selects relevant skills based on beliefs and goals Contains goals and intentions The Soar Architecture The Companions Architecture Body Long-Term Memories Procedural Short-Term Memory Decision Procedure Chunking Episodic Learning Semantic Learning Semantic Reinforcement Learning PerceptionAction Markov Logic Weighted Satisfiability Markov Chain Monte Carlo Inductive Logic Programming Weight Learning Target Domain Source Domain

2 Year 1: Transfer in Three Testbeds Urban Combat is a first- person shooter game that involves spatial reasoning and reactive control General Game Playing covers a broad class of N- person games that involve strategic reasoning. ETS Physics involves finding answers to physics problems through a mixture of plausible inference and search Crawl under Climb over Urban Combat ETS Physics GGP Soar Companions I CARUS Our Year 1 efforts focused on Urban Combat (Soar and I CARUS ), but with some work on GGP (I CARUS ) and ETS Physics Companions) source target Year 1 emphasizes lower levels (1 to 8) of transfer learning

3 ISLE Team: Year 2 Integration Plans Alchemy (Washington) Long-TermConceptualMemoryShort-TermConceptualMemory Short-TermGoal/SkillMemory ConceptualInference SkillExecution Perception Environment PerceptualBuffer Problem Solving Skill Learning MotorBuffer Skill Retrieval Long-Term Skill Memory Reinforcement Learning (UT Austin) Weighted Satisfiability Markov Chain Monte Carlo Inductive Logic Programming Weight Learning Markov Logic β (A)A γ (S)S 1 2 I CARUS (ISLE) Hierarchical Task Networks (Maryland) Insert Cycorp Mountain here CYC (Cyrcorp) Year 2 integration will revolve around replacing existing I CARUS modules with software from other team members

4 Year 2: Deep Transfer in Three Testbeds In Urban Combat, we will demonstrate transfer from urban military missions to fire rescue operations In General Game Playing, we will show transfer across quite different games that use related strategies (e.g., forking moves) Crawl under Climb over Urban Combat ETS Physics GGP Soar Companions I CARUS In Year 2, We will evaluate each pair of architectures on at least one shared testbed and we will test them all on the General Game Playing testbed source target Year 2 will focus on higher levels (9 -10) of transfer learning In ETS Physics, we will show transfer from linear systems to rotational, thermal, hydraulic, and electrical systems source target

5 ISLE Team: Year 2 Evaluation Plans Comparison among architectures that use different mechanisms should reveal which approaches best support transfer learning. We will evaluate each pair of architectures on at least one shared testbed and will test them all on the General Game Playing testbed. Experiments will examine how well the frameworks support transfer in settings that emphasize reactive control, conceptual inference, and heuristic search. Year 2 evaluations will focus on high-level (9 and 10) transfer in each of the three testbeds. Urban Combat ETS Physics GGP Soar Companions I CARUS Architectures Testbeds