0. Soft Computing Lab. Yongjun Kim
A Gentle Introduction to Soar, an Architecture for Human Cognition: 2006 Update
Jill Fain Lehman, John Laird, Paul Rosenbloom
26th Mar., 2009
Soft Computing Lab.
Yongjun Kim

1. Outline Introduction Soar (State operator and result) Architecture
The Idea of Architecture
What Cognitive Behaviors have in common
Behavior as Movement through Problem Spaces
Tying the Content to the Architecture
Memory, Perception, Action, and Cognition
Detecting a Lack of Knowledge
Learning
Putting it all together: a Soar Model of Joe Rookie
Stepping back: the Soar Architecture in review
From Architecture to Unified Theories of Cognition
Discussion

2. Introduction
Many intellectual disciplines contribute to cognitive science:
Psychology, linguistics, anthropology, artificial intelligence, etc.
Each discipline provides expertise and contributes microtheories.
Descriptions of regularities in behavior
Theories that try to explain those regularities
How to know the contributions of each discipline fit in the big picture?
Go ahead and try to put the whole picture together.
Try to build unified theories of cognition (UTCs).
A set of general assumptions for cognitive models that account for all of cognition.
Soar was developed and used as a candidate UTC in 1980’s.
Try to find a set of computationally-realizable mechanisms and structures that can answer all the questions about cognitive behavior.

3. The Idea of Architecture
Architecture is the fixed set of mechanisms and structures.
An architecture stands as a theory of what is common among behavior.
Any complex system can be decomposed into architecture and content.
Architecture requires content to produce behavior.
Cognitive Architecture
A theory of the fixed mechanisms and structures that underlie human cognition.
Soar is used as a cognitive architecture.

4. What Cognitive Behaviors have in common
Soar theory assumes that cognitive behavior has at least the following characteristics:
It is goal-oriented.
It takes place in a rich, complex, detailed environment.
It requires a large amount of knowledge.
It requires the use of symbols and abstractions.
It is flexible, and a function of the environment.
It requires learning from the environment and experience.
Need to explore the architecture in terms of some particular content in order to see how the architecture contributes to behavior.
Have to be goal-oriented about something.
Need a scenario.

5. What Cognitive Behaviors have in common
A Simple Scenario from Baseball.

6. What Cognitive Behaviors have in common
A Simple Scenario from Baseball.
Behave in a goal-oriented manner:
Joe’s goal is to win the game.
A number of subgoals : get the batter out, strike the batter out with a curve ball, etc.
Operate in a rich, complex, detailed environment:
Many relevant aspects of Joe’s environment.
Positions of people, the number of balls and strikes, etc.
Use a large amount of knowledge:
Need to draw on statistics about his own team, his own pitching record, etc.
Behave flexibly as a function of the environment:
Need to respond to his own perceptions of the environment.
Is it windy?, is the batter left-or right-handed? etc.
Use symbols and abstractions:
Need to draw on his previous experience by abstracting away from this day and place.
Learn from the environment and experience:
Joe need to throw Sam a fast ball next time.

7. What Cognitive Behaviors have in common
A Simple Scenario from Baseball.
For Joe to act like a rookie pitcher, many different kinds of knowledge should be given to it.
Need to find some way to represent and process Joe’s knowledge in Soar.
Assume that there is an underlying structure to behavior and knowledge.
This structure provides a means for organizing knowledge as a sequence of decisions through a problem space.

8. Behavior as Movement through Problem Spaces
The Space of Possible Actions for Joe.
Must make his decisions with respect to the situation at the moment.
Support two points of view : a static / a dynamic view of Joe’s life.

9. Behavior as Movement through Problem Spaces
The Abstract Form of a Problem Space.
Consist of states, features, values and operators.
The state is a representation of all the aspects of the situation (internal and external).
Only one state exists at any time and prior states are not directly accessible.
f2 : batter status
v6 : out

10. Behavior as Movement through Problem Spaces
The Abstract Form of a Problem Space.
Movement could be entirely random.
To keep behavior goal-directed, the succession of operators and the resulting state transformations must be guided by the principle of rationality:
If an agent has knowledge that an operator application will lead to one of its goals then the agent will select that operator.
Tying the Content of Joe’s World to the Soar Architecture.
Map knowledge into goals, states and operators.
Determine what knowledge becomes part of the state and the operators.
Find how to know what an operator application will do.
Find how to know when the goal has been achieved.

11. Tying the Content to the Architecture
Knowledge Representation in the Architecture
Must be domain independent.
What is common across all domains and problems?
In Soar, it is the decomposition of knowledge into goals, problem spaces, states, and operators.

12. Tying the Content to the Architecture
Guidelines for Tying the Domain Content to the Architecture
The domain knowledge of the objects and people in the game (K1) => the features and values in the state
Knowledge of actions (K5 and K7) => operators
Knowledge about objectives (K4) => goals
Knowledge of the rules of the game (K3) + K1 + K4 + K5 + K7 => problem space
The Operator Selection
Operators that share common tests for goals and situations can be considered to be part of the same problem space.
Given the initial state and the goal in the game, the operators will be the various kinds of pitches.

13. Tying the Content to the Architecture
The Effect of Operators
Can be defined in two ways:
Defined by the execution of the operator in the external world using knowledge of physical actions (K7).
Defined by knowledge of abstract events or particular episodes (K2).
Goal Evaluation
Determining that the current state is a desired state relies on knowledge of the rules of the game (K3).
The environment can give signals of success and failure.
Umpire’s judge (“You’re out!”)
The Modification of Goal and Problem Space
Done by augmenting the state with goals and problem spaces.
Ex. Joe’s team is ahead in the fifth inning, but rain is on the horizon => out quickly.
Ex. A member of the opposing team on base => short windup.

14. Memory, Perception, Action, and Cognition
Soar’s Memory
Consist of long-term memory (LTM) and working memory (WM).
LTM has three different types: procedural, semantic, and episodic.

15. Memory, Perception, Action, and Cognition
Memory type
Long-term memory (LTM)
Procedural : knowledge about how and when to do things.
How to ride a bike, how to solve an algebra problem, etc.
Semantic : knowledge consists of facts about the world.
Bicycles have two wheels, a baseball game has nine innings, etc.
Episodic : knowledge consists of things you remember.
The time you fell off your bicycle and scraped your elbow.
LTM is not directly available, but must be searched to find what is relevant to the current situation.
Procedural knowledge is primarily responsible for controlling behavior and maps directly onto operator knowledge.
Working memory (WM)
Knowledge that is most relevant to the current situation.
In Soar, WM is represented as a set of the features and values that make up the current state (and substates).
Can be used to retrieve other knowledge from LTM.
Working memory elements in Soar arise in one of two ways:
Through perception.
Through retrievals from long-term memory.

16. Memory, Perception, Action, and Cognition
Examples of LTM procedural knowledge
There are dependencies between the rules. However, Soar doesn’t recognize them.
Rules are processed by the architecture in a general way.

17. Memory, Perception, Action, and Cognition
The Decision Cycle
Generate behavior out of the
content in LTM and WM.
Do its work in five phases:
Input
WM elements are created.
Elaboration
WM elements are matched against
the “if” parts of the rules in LTM.
Decision
Decide suggestions according to
preferences (symbolic/numeric).
Application
Output
Support limited parallelism.
Multiple actions can be packaged
together as a single operator.

18. Detecting a Lack of Knowledge
Impasse in the Decision Cycle
Happen when the decision cycle can’t decide a single operator due to lack of knowledge for preferences (e.g., without r5).
Soar automatically creates a substate.
The goal is to select between two operators for the original state.
Semantic and episodic memories are usually used in substates.
The reminding is goal-driven.

19. Detecting a Lack of Knowledge
Memory Search in Impasse
Assume that Joe has the following fact in episodic memory.
A Cue must be created that can be used to search the memory.
In some cases, no likely match will be returned. Then, the model can modify the cue.
The next three rules define the evaluate operator that creates preferences to resolve the tie.

20. Detecting a Lack of Knowledge
Resolving an Operator-Tie Impasse

21. Detecting a Lack of Knowledge
Working Memory Hierarchy
Working memory consists of a state/substate hierarchy.
The hierarchy grows as impasses arise and shrinks as impasses are resolved.
If multiple changes are suggested in different states, the change to the state that is highest in the hierarchy is selected.
If a change occurs to a context high up in the hierarchy, then all the substates below the changed state disappear.
Impasse Type in Soar
Other types of impasses can be occurred.
E.g., an operator tie, an operator no-change impasse, etc.
The full set of impasses defined in Soar is fixed and domain-independent.

22. Learning Four Learning Mechanisms
Chunking
Reinforcement learning
Episodic learning
Semantic learning
Three Questions for Learning Systems:
What do they learn?
Soar systems learn structures for its LTM: rules, declarative facts, and episodes.
What is the source of knowledge for learning?
Different between the learning mechanisms.
When do they learn?

23. Learning Chunking Reinforcement Learning
Is the most developed learning mechanism.
Is deductive and compositional.
Resolving impasses can lead to learn new rules (called chunks).
Reinforcement Learning
Knowledge source is feedback from the environment (reward).
Learn rules that generate preferences based on future expected rewards.
Two parts in Soar
Must learn rules that test the appropriate features of the states and operators.
Initially create rules based on the rules that propose operators, and specialize them to consistently predict the same value.
Must learn the appropriate expected rewards for each rule.
Done by comparing the prediction of a rule with what happens during the next decision.

24. Learning Episodic Memory Semantic Memory
Knowledge source is the stream of experience.
Episodes are recorded automatically as a problem is solved.
An episode consists of a subset of the WM elements that exist at the time of recording.
Comparison with chunking and reinforcement learning
Passive learning mechanism.
The contents of an episode are determined only indirectly by reasoning.
No distinction between conditions for retrieval and what should be retrieved.
Semantic Memory
Knowledge source is the co-occurrence of structures in WM.
Knowledge about the rules of baseball, what is a home run, etc.
When to store a structure in semantic memory is a research issue.
Deals with static structure instead of derivation-based rules.
Comparison with episodic memory
More general than episodes.
Place and time information is disassociated.

25. Putting it all together: a Soar Model of Joe Rookie
To Build a Full Model.
Specify the domain knowledge and which memories it is stored in.
Tie the domain knowledge to state structures and operators.
Specify the relationships between different levels by the impasses and the kinds of knowledge that will be missing, and learned.

26. Stepping back: the Soar Architecture in review
A Cognitive Architecture
Is a fixed set of mechanisms and structures that process content to produce behavior.
Is a theory, or point of view, about what cognitive behaviors have in common.
Soar Architecture
States and Operators
Working Memory
Long-term memory
The Perception/Motor Interface
The Decision Cycle
Impasses
Four Learning Mechanism

27. From Architecture to Unified Theories of Cognition
The model of Joe Rookie is a content theory.
Can explain why human needs change with time.
Can explain the specific factors that motivate people.
Soar has contributed a number of content theories to the field.
NTD-Soar, Instructo-Soar, IMPROV, TacAir-Soar, Soar MOUTBOT, etc.
Content theories model aspects of human language (or concept learning, or multi-tasking) within a framework.
The theory of the resulting model will be compatible with what is assumed to be architectural in the other content theories.
Content theories constitute a burgeoning unified theory of cognition (UTC).
Soar as a UTC.

28. Discussion
What is the similarity and difference between Soar and ACT-R?
Both try to model human.
Soar has focused on memory system, but ACT-R on brain image.
Do you think Soar can be a UTC?
A UTC must explain the following things.
How intelligent organisms flexibly react to stimuli from the environment.
How they exhibit goal-directed behavior and acquire goals rationally.
How they represent knowledge (or which symbols they use).
Learning.
It is known that there is efforts to model emotions into Soar. Do you think emotions can play a big role in cognition?