1. Integrated Episodic and Semantic Memory in Robotics Steve Furtwangler, sfurtwangler@soartech.com with Robert Marinier, Jacob Crossman

2. Introduction Robotics domain has some unique challenges General patterns or issues we encountered working in robotics Specifically, I will talk about Measuring similarity in semantic memory Using episodic and semantic memory together Required or prohibited query conditions Recreation of state 2

3. Using Episodic Memory for Partial Matches The agent creates a statistical model of its world The statistics are stored in semantic memory Long-term identifiers created for each thing we are modeling Statistics kept on these identifiers Sometimes need to find similar things Semantic memory doesn’t support partial matches We decided to leverage episodic memory to do this instead Example: If the agent has no/little statistical data for this exact situation Can ask if it was ever in a situation like this one If so, look up statistical data for that situation in semantic memory 3

4. Episode Representation - Unique Cues - Cell S1 cell C1 x 1 2 High Low y road trees Invariable Attributes Variable Attributes Q1 x 1 y 2 cue @Q5 x 1 y 2 id Find LTI with that cue, or create a new one if it is not found 4

5. Episode Representation - Unique Cues - Path S1 path P1 cell C1 High Low road trees Create one level deep cue for path, using unique-ids of cells and order of cells cell C2 cell C3 @Q1 id @Q2 id Paths which may have complex, deep working memory structures Q9 1 @Q1 @Q2 cue 2 @Q3 3 5

6. Measuring Similarity Low Med High V.Low V.High M.Low M.High 6 Cue Results Match Scores C1 feature Low C1 feature High C1 feature Med0 0 C1 feature1 Low M.Low feature2 C1 feature1 Med M.Low feature2 C1 feature1 High M.High feature2 1 0 One dimension doesn’t capture similarity Adding a second dimension helps

7. Episodic and Semantic Memory Conflicts The objects in memory are identified in semantic memory Some of the attributes on these objects (statistics) change over time These long-term identifiers are referenced on the topstate So they show up in episodic memory However, when episodic memory recreates the episode It recreates the attributes and values that the LTI had at the time 7

8. Example of Problem S1 @L1Old object value S1 @L1New object value Q1 query S1 @L1New OldR1 value object Q1 object query result value Problem: Cannot distinguish value in episode from current value Attribute “value” becomes a multivalued attribute 8

9. Q1 cell C1 road Med Low trees Episodic Memory Cue Solution: Long-Term Identifier Usage Pattern Problem Space S1 cell C1 x 1 2 High Low y road trees @Q5 x 1 y 2 id Episodic Memory result Q2 Semantic Memory cue id @Q6 Semantic Memory result id 2 success failure 12 Statistics not stored with episodes Conceptually, two kinds of LTIs 9

10. Required (or Prohibited) Query Conditions Queries to Episodic Memory often have two different kinds of conditions Things that have to exist in the episode (or should not exist) This tends to decide of the episode is even relevant or not Things which are optional, but should be as similar as possible Example: Query for a similar situation where the agent decided to go right In order to reason about what might happen if I turn right, now Result is a situation like the current situation, only the agent went left Has to be prohibited, until the agent gets a memory of going right Leads to a common pattern… 10

11. Solution: Episodic Memory Loop Pattern Construct Query Retrieve Episode Filter Episode Continue? Map ResultAnnotate Input 11

12. Time Spent Recreating State Often create episodic memory queries to answer a specific question “When I was last in this location, what time of day was it?” Retrieving the episode creates a lot of WMEs to recreate the whole state “Last time you were at this location, it was a Tuesday, it was raining, your fuel was at 90%... Yada yada yada… oh, and it was 5:35pm.” The time to recreate a state is, in part, based on size of that state We often look at one small piece of that result and throw it all way Causing all of those WMEs to immediately be removed The filtering loop may cause this to happen many times 12

13. Nuggets Reduced instances of repeated failure Agent doesn’t do the same dumb thing twice Constructed model for environment/plans Accuracy of estimations improve with experience Incorporated models of similar environments/plans Agent came to useful conclusions for new (untested) plans 13

14. Wish List (Coal) Partial matching for semantic memory Using episodic memory to achieve this is a hack Metric/Custom comparison functions Necessary for queries about similarity in space or to weight features Safeguards for episodic memory retrievals of long-term identifiers To reason about what a LTI looked like in the past as opposed to now Require/Prohibit queries in episodic and semantic memory Eliminate the epmem loop pattern used to filter out bad results Ability to specify sub-section of state to retrieve from episodic memory Often only care about a few key WMEs Reconstructing the entire state takes time 14