1. Subsumption Architecture.
Systems and Control
Subsumption Architecture.

2. Background Developed by Rodney Brooks at MIT in mid 80s.
A purely reactive task-based method.
Brooks argued that the Sense-Plan-Act paradigm used in first autonomous robots, like Shakey, was not practical.
Sense
Model
Plan
In his seminar paper, 1986, advocated the use of layered control system in a horizontal decomposition:
Act
Modify the World
Create Maps
Discover New Areas
Avoid Collisions
Move Around

3. The traditional approach to control was:
System Decomposition
The traditional approach to control was:
the Sense-Plan-Act (SPA) approach
inherently sequential
This produces deliberative architectures.
Subsumption Architecture was introduced as an alternative to SPA.
Subsumption produces inherently parallel systems.

4. Layers of Behaviors The Subsumption architecture is built in layers.
Each layer gives the system a set of pre-wired behaviors.
The higher levels build upon the lower levels to create more complex behaviors.
The behavior of the system as a whole is the result of many interacting simple behaviors.
The layers operate asynchronously.
Note: Behavior is a stimulus/response pair. For the subsumption architecture, intuitively means some type of software module designed to achieve one or more specific tasks.

5. Key Features of Subsumption.
Behavior is distributed rather than centralized.
Response to stimuli is reflexive:
the perception-action sequence is not modulated by cognitive deliberation.
Individual agents are inexpensive, allowing a domain to be populated by many simple agents rather than a few complex ones.
these simple agents individually consume little resources (such as power) and are expendable, making the investment in each agent minimal.

6. Key Features of Subsumption.
No explicit knowledge representation is used. Brooks often refers to this as "The world is its own best model".
The reactive tasks (agents) or behaviors are organized in a bottom-up fashion.
Thus, complex behaviors are fashioned from the combination of simpler, underlying ones.
But complex behavior need not necessarily be the product of a complex control system.
Intelligence is in the eye of the observer.
Note: Agent or software agent definition, on the next slide.

7. Agent
Asset
Environment
Sense
Learn
Act
Plan
An agency relationship is present whenever one party (the principle) depends on another party (the agent) to undertake some task on the principle’s behalf.
An agent is an entity which can be viewed as perceiving its environment through sensors and acting upon that environment through effectors.

8. Biological Foundations
Why Biology?
Airplanes don’t flap their wings
We don’t intend to copy the blue sheet of solution found by evolution
But we may get an inspiration
Why Biology
Closed world assumption isn’t realistic
The best World Model is only a model…
Lots of “stupid” creatures cope without closed world assumption
Maybe we can learn something
Agent
A self-contained independent intelligent system
Humans, Cats, Frog

9. Marr’s Computational Theory
Level 1:
What is the phenomena
we’re trying to represent?
(Proof it can be done)
for (i=nCol..
Level 2:
How it be represented as
a process with inputs/outputs?
(Flow charts)
Level 3:
How is it implemented?
(Specific details)

10. Level 1: Existence Proof
Goal: how to make line drawings of objects?
people can do this by age 10, computers should be able to do it
Level 1:
What is the phenomena
we’re trying to represent?

11. Level 2: Inputs, Outputs, Transforms
light
drawing
light
lines (edges)
drawing
retina
(gradient)
for (i=nCol..
Level 2:
How it be represented as
a process with inputs/outputs?

12. Level 3: Implementation+ -
G= |Gx|+|Gy|
Edge Detectors in the eye
Sobel Edge Detector
in computer vision
Level 3:
How is it implemented?

13. Behavior Definition
A behavior is a mapping of sensory inputs to a pattern of motor actions which then are used to achieve a task.
BEHAVIOR
Pattern
of Motor
Actions
Sensory
Input

14. Types of Behaviors Reflexive Reactive Conscious
stimulus-response, often abbreviated S-R
Reactive
learned or “muscle memory”
Conscious
deliberately stringing together

16. Sea Turtles
Born at night
Head to ocean to avoid prey

17. Arctic Terns
Arctic terns live in Arctic (black, white, gray environment, some grass) but adults have a red spot on beak
When hungry, baby pecks at parent’s beak, who regurgitates food for baby to eat
How does it know its parent?
It doesn’t, it just goes for the largest red spot in its field of view Only red thing should be an adult tern
Closer = large red

18. behavior template
BEHAVIOR
Pattern
of Motor
Actions
Sensory
Input

19. “the feeding behavior”
RED
Feeding
BEHAVIOR
PECK AT RED
Pattern
of Motor
Actions
Sensory
Input

20. Innate Releasing Mechanisms (INR)
Releaser
present? N Y
/dev/null
Sensory input
and/or
internal state
BEHAVIOR
Pattern
of Motor
Actions
Sensory
Input

21. the releaser template Releaser present? N /dev/null Y Sensory input
and/or
internal state

22. “the feeding releaser”
RED &
HUNGRY
sensory input
Releaser
internal state
present? N
/dev/null Y
RED
Feeding
BEHAVIOR
PECK AT RED

26. Example: Cockroach Hide
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
even if the lights are turned back off earlier
cockroaches!

27. Example: Hide Behavior
Programmed in C++, << 100 LOC
shows
taxis (oriented relative to light, wall, niche)
fixed action pattern (persisted after light was off)
reflexive (stimulus, response)
implicit sequencing
use of internal state
cockroaches!

28. Reflexive Behaviors S-R Implicit Chaining
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
even if the lights are turned back off earlier
Fixed Pattern Actions
cockroaches!
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
even if the lights are turned back off earlier

29. Exhibits Taxis (response is to move to a particular orientation)
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
even if the lights are turned back off earlier
to light
to wall
cockroaches!
to niche

30. Break into Behaviors light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
Flee
Follow-
wall
cockroaches!
hide

31. Ooops, need internal state:
LIGHT
present? N
Find Releasers Y
Flee
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
Wall/Blocked
present? N
Ooops, need internal state:
Scared
Follow-
wall
SCARED &
SURROUNDED
cockroaches!
present? N
hide

32. Internal State Set light goes on, the cockroach turns and runs
present? N
Internal State Set Y
Flee
SCARED
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
BLOCKED &
SCARED
present? N
Follow-
wall
SCARED &
SURROUNDED
cockroaches!
present? N
hide

33. Action light goes on, the cockroach turns and runs
present? N
Action
Steer and
drive forward Y
Flee
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
SCARED
BLOCKED &
SCARED
present? N
steer =F(dist to wall)
drive forward const.
Follow-
wall
SCARED &
SURROUNDED
cockroaches!
present?
steer =F(dist to wall)
drive forward const.
stop N
hide

34. Sensory Input light goes on, the cockroach turns and runsIR
LIGHT
present? N
Steer and
drive forward
encoders Y
Flee
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
SCARED
BLOCKED &
SCARED IR
present? N
steer =F(dist to wall)
drive forward const. IR
Follow-
wall
SCARED &
SURROUNDED IR
cockroaches!
present?
steer =F(dist to wall)
drive forward const.
stop N IR
hide

35. How Do You Link Them? light goes on, the cockroach turns and runsIR
LIGHT
present? N
Steer and
drive forward
encoders Y
light goes on, the cockroach turns and runs
when it gets to a wall, it follows it
when it finds a hiding place, goes in and faces outward
waits until not scared, then comes out
Flee
SCARED
BLOCKED &
SCARED IR
present? N
steer =F(dist to wall)
drive forward const. IR
Follow-
wall
SCARED &
SURROUNDED IR
what is the hide circuit burned out?
present?
steer =F(dist to wall)
drive forward const.
stop N IR
hide

36. Innate Releasing Mechaisms work on THREADS, not sequential processing!
Very simple modules
Nice building blocks since not directly linked
If one module (part of brain) fails, what happens?
Still works (the system), but with reduced performance

37. What happens when there’s a conflict from concurrent behaviors?
Equilbrium
Feeding squirrels-feed, flee: hesitate in-between
Dominance
Sleepy, hungry: either sleep or eat
Cancellation
Sticklebacks defend, attack: build a nest ?

38. Conflict Resolution. Complex actions are built from simpler behaviors.
The simple behavior’s action, being based on a specific sensory processing algorithm may be in conflict with other behavior’s output action.
Through specifically designed coordination mechanisms, called priority based arbitration may be enforced.
The process to solve the problem is called conflict resolution.

39. Triggering sensory input to action.
If a reactive system divides its perceptual world into mutually exclusive or unique situations, then only one of those situations can be triggered by any sensory input at any one time, and only one action will be activated as a result.
This is the simplest form of a reactive control system.
It is often too difficult to split up all possible situations this way, or it may require unnecessary encoding.
Consider the case of multiple sensors: to have mutually-exclusive sensory inputs, the controller must encode rules for all possible sensory combinations.
There is an exponential number of those.
— This is, in fact, the robot's entire sensory space. This space then needs to be mapped to all possible actions (the action space), resulting in the complete control space for that robot.
Although this mapping is done while the system is being designed, i.e., not at run-time, it can be very tedious, and it results in a large look up table which takes space to encode/store in a robot, and can take time to search, unless some clever parallel look up technique is used.

40. Arbitration
In general, this complete mapping is not used in hand-designed reactive systems.
Instead, specific situations trigger appropriate actions, and default actions are used to cover all other cases.
Human designers can effectively reduce the sensory space to only the inputs/situations that matter, map those to the appropriate actions, and thus greatly simplify the control system.
If the rules are not triggered by mutually-exclusive conditions, more than one rule can be triggered in parallel, resulting in two or more different actions being output by the system.
Deciding among multiple actions or behaviors is called arbitration, and is in general a difficult problem.

41. Arbitration priorities and multi-tasking.
Arbitration can be done based on:
a fixed priority hierarchy (processes have pre-assigned priorities)
a dynamic hierarchy (process priorities change at run-time)
learning (process priorities may be initialized or not, and are learned at run-time, once or repeatedly/dynamically)
If a reactive system needs to support parallelism, i.e., the ability to execute multiple rules at once, the underlying programming language must have the ability to multi-task, i.e., execute several processes/pieces of code in parallel.
The ability to multi-task is critical in reactive systems: if a system cannot monitor its sensors in parallel, but must go from one to another in sequence, it may miss some event, or at least the onset of an event, thus failing to react in time.

42. Coordination mechanisms can be divided into two main classes: arbitration and command fusion.
Classes of coordination mechanisms
Arbitration
Command Fusion
Priority based
Voting
Superposition
State based
Fuzzy
Winner take all
Multiple objective

43. Priority Based Coordination as conflict resolution.
במודל Subsumption מניעת סתירות מתבצע על ידי העדפת התנהגות מוכתבת על ידי שכבה עליונה יותר.

44. Other conflict resolution methods.
בחירת השיטה חייבת להיות תואמת למשימה הספציפית בצומת. דוגמה לבחירה לא נכונה יכולה לשמש המקרה בו רכב מתקרב למכשול הנמצא ישר לפנים. חיישן אחד יחליט שעל מנת למנוע התנגשות יש לפנות ימינה ואילו חיישן אחר שמאלה. אם גם עוצמות הסיגנלים יהיו שווים, חיבור וקטורי יגרום להתנגשות בטוחה!

45. Reactive Robotic Systems Characteristics
Behaviors serve as the basic building block for robotic actions.
A behavior typically consists of a simple sensor and motor pair
Use of explicit abstract representational knowledge is avoided in the generation of a response.
What you see is what you get.
Valuable in highly dynamic and hazardous environments, where unpredictability and potential hostility are inherent.
Animal models of behavior often serve as a basis for these systems.
Biology is providing an existence proof that many of the tasks we would like our robots to undertake are doable.
These systems are inherently modular from the software design perspective.
Enables the designer to expand competency by adding new behaviors without redesigning or discarding the old behaviors.

46. Situated Activity Based Design.
A robot’s actions are predicted upon the situation in which it finds itself.
Perception is reduced to recognizing what situation the robot is in and then choosing one action to undertake.
As soon as the robot finds itself in a new situation, it selects a new more appropriate action.

47. Arbitration in Subsumption
Arbitration: deciding who has control
Complex actions subsume simpler behaviors.
Inhibition: prevents output signals from reaching effectors.
Suppression: replaces input signal with the suppressing message.
The above two are the only mechanisms for coordination.
=> Results in priority-based arbitration.
the rule or layer with higher priority takes over, i.e., has control of the AFSM.

48. Augmented Finite State Machine as used within the subsumption architecture.

49. Designing in Subsumption
Qualitatively specify the overall behavior needed for the task.
Decompose that into specific and independent behaviors (layers).
The layers should be bottom-up and consisting of disjoint (incoherent) actions.
Ground low-level behaviors in the robot’s sensors and effectors.
Incrementally build, test, and add.

50. Networks of AFSMs
A Subsumption Architecture controller, using the AFSM-based programming language, is a network of AFSMs.
The network is divided into layers.
Once a basic competence is achieved (e.g., moving around safely), it is labeled layer 0.
The next layer (1) is added, and communication is done through wires.

51. Implementation
The original Subsumption Architecture was implemented using a language based on finite state machines (FSMs) augmented with a very small amount of state (AFSMs), themselves implemented in Lisp.
The underlying language can be anything, but the structure of FSMs will be convenient for incremental system building.
An AFSM can be in one state at a time, can receive one or more inputs, and send one or more outputs.
AFSMs are connected by communication wires, which pass input and output messages between them.
Start
Wander
Acquire
Finite State Acceptor (FSA) diagram for foraging
Halt
Retrieve

52. AFSMs for a simple three layer robot in Subsumption Architecture
Back-out-of-Tight Situations Layer
Lost
Collide
Reverse
Explore Layer
Wander
בדוגמה זאת ארכיטקטורה מבוססת על שלוש שכבות לביצוע משימה של איסוף (למשל איסוף פצצונות).
השכבה העליונה, רואה תמונת מצב שלימה ולכן יש לה היכולת להכתיב מהי הפעולה המתאימה ביותר להשלמת המשימה.
השכבה הנמוכה דואגת לתנועה ועקיפת מכשולים. השכבה האמצעית מחפשת פצצונות לאיסוף.
לכן הרובוט שואף לנוע קדימה, אבל השכבה השניה יכולה להנחות שינוי כיוון לצורך חיפוש.
Run Away

53. Implementation of Subsumption Architecture
Subsumption systems
grow from the bottom up, and
layers can keep being added, depending on the tasks of the robot.
How exactly layers are split up depends on the specifics of the robot, the environment, and the task.
There is no strict recipe, but some solutions are better than others, and most are derived empirically.

54. Implementation
The inspiration behind the Subsumption Architecture is the evolutionary process, which introduces new capabilities based on the existing ones.
Complete “creatures” are not thrown out and new ones created from scratch; instead, solid, useful “substrates” are used to build up to more complex capabilities.

55. Priority Based Coordination as conflict resolution in Subsumption Architecture

56. How this works? If we number the layers from 0 up,
we can assume that the 0th layer is constructed, debugged, and installed first.
As layer 1 is added, layer 0 continues to function, but may be influenced by layer 1, and so on up.
If layer 1 fails, layer 0 is unaffected.
When layer 1 is designed, layer 0 is taken into consideration and utilized, i.e., its existence is subsumed, thus the name of the architecture.
Layer 1 can inhibit (forbid) the outputs of layer 0 or suppress (replace) its inputs.

57. The Guiding Principles. (summary)
Systems are built from the bottom up.
Components are task-achieving actions/behaviors (not functional modules)
Components can be executed in parallel.
Components are organized in layers, from the bottom up.
Lowest layers handle most basic tasks.
Newly added components and layers exploit the existing ones.
Each component provides and does not disrupt a tight coupling between sensing and action.
There is no need for internal models: "the world is its own best model"

58. Module Independence
Strong coupling implies dependence between modules, which violates the modularity of the system.
If modules are interdependent, they are not as robust to failure.
In Subsumption, if higher layers fail, the lower ones remain unaffected.
Thus Subsumption has one-way independence between layers.
Two-way independence is not practical,
why? If higher level fails, it may still function based on low level behaviors.
But if lower level fails it is meaningless to let the higher levels continue to develop complex behavioral actions.

59. Coupling is done through the world.
Coupling between layers, and even between AFSMs, need not be through the system itself (i.e., not through explicit communication wires), but through the world.
This brings us back to a key principle of the Subsumption Architecture, use the world as its own best model.
The idea is that if the world can provide the information directly (through sensing), it is best to get it that way, rather than to store it internally in a representation (which may be large, slow, expensive, and outdated).

60. Subsumption Evaluation
Strengths:
Reactivity (speed, real-time nature)
Parallelism
Incremental design => robustness
Generality
Weaknesses:
Inflexibility at run-time
Expertise needed in design