1. System Architectures
Can you make the “architecture” of AI for robots more tangible?
What are the subsystems in a system architecture?
I know a technical architecture depends on the implementation, but what goes into a technical architecture?

2. Specific Learning Objectives
Be able to relate the functions in the canonical operational architecture to the 5 common subsystems
Classify a systems architecture as being either hierarchical, reactive, or hybrid deliberative/reactive based on 1) the relationship of the 3 AI robot primitives and 2) sensing handling
Be able to draw the Hybrid Deliberative/Reactive System Architecture
Objectives
Review
Systems
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

3. Systems Architectures: Outline
Review
3 ways of generally organizing systems
5 common subsystems
Why those 5? A historical evolution of Hierarchical, Reactive, Hybrid Deliberative/Reactive
Contributions to canonical system architecture
A little about technical architectures (technical)
Summary
Objectives
Review
Systems
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

4. Recall: Types of architectures [Levis, George Mason University]
Objectives
Review
Systems
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
operational architecture: describes what the systems does, not how it does it
systems architecture: describes how a system works in terms of major subsystems
technical architecture: implementation details, language

5. Types of architectures
Class will focus on these levels
And the algorithms used at this level
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
operational architecture: describes what the systems does, not how it does it
systems architecture: describes how a system works in terms of major subsystems
technical architecture: implementation details, language

6. The Most Abstract Canonical Operational Architecture
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
“Upper brain” or cortex
Reasoning over symbols (information) about goals
Interaction
Layer
Deliberative
Layer
“Middle brain”
Converting sensor data into symbols (information)
Reactive (or Behavioral)
Layer
Spinal Cord and “lower brain”
Skills and responses
Looks like Mardi Gras colors…

7. The Most Abstract Canonical Operational Architecture
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Interaction
Layer
Each layer has a different style of program organization
Not concerned with details of interaction layer
Deliberative
Layer
Reactive Layer
Behavioral
Layer

8. More Tangible Canonical Operational Architecture
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
monitoring
generating
selecting
implementing
PRESENT+PAST+FUTURE, SLOW
plan
World
model
PRESENT+PAST, FAST
PRESENT, VERY FAST
sense
act
Look at functions within the behavioral and deliberative layers, make it more tangible

9. World plan model sense act sense act sense act
monitoring
generating
World
model
plan
DELIBERATION
selecting
implementing
sense
act
sense
act
sense
act
REACTION
Reasoning over information about goals:
Promising results: Navigation, payload planning, contingency replanning
Open issues: Multi-agent replanning, fault recovery & reconfiguration, reasoning over multiple failures
BUT that doesn’t mean it is easy
Converting sensor data into information:
Promising results: ATR, single failure health monitoring
Open issues: creation of world models & situation awareness, monitoring & detection of new threats, exceptions, opportunities
Behaviors, skills and responses

10. Types of architectures [Levis, George Mason University]
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
operational architecture: describes what the systems does, not how it does it
systems architecture: describes how a system works in terms of major subsystems
technical architecture: implementation details, language
So we have layers, we have functions--- but not how it’s grouped for programing

11. System Architectures Provide…
“right” subsystems
Focus on good software engineering
Modular (object-oriented)
Abstraction and cohesion
Libraries of algorithms and data structures within a subsystem
Can pick from library to fit a specific niche
Platform neutral
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

12. What are the subsystems in a systems architecture?
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
What are the subsystems in a systems architecture?
Via…
A history of the three paradigms and example systems

13. Generally Accepted Subsystems (or Objects)
Navigation (Generating)
Planning about navigational goals, path planning
Cartographer, World Model, World Map (World Model)
Everything to do with world models and planning over maps
Planning (Generating mission, Implementing, Selecting, Monitoring)
Motor Schemas, Behaviors (executing motor commands)
Perception, Sensing, Perceptual Schemas (executing sensor input)
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
From Technology Development for Army Unmanned Ground Vehicles 2002, National Research Council
This is what they are, but interesting to see how they arose

14. Generally Accepted Subsystems
Navigation (Generating)
Planning about navigational goals, path planning
Cartographer, World Model, World Map (World Model)
Everything to do with world models and planning over maps
Planning (Generating mission, Implementing, Selecting, Monitoring)
Motor Schemas, Behaviors (executing motor commands)
Perception, Sensing, Perceptual Schemas (executing sensor input)
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Note the focus on Navigation- would this be sufficient for healthcare or entertainment applications? A robot with a manipulator? Or driving in heavy traffic with other cars?
From Technology Development for Army Unmanned Ground Vehicles 2002, National Research Council
The focus on navigation is because just moving was hard enough. But you generally see object libraries set up this way.

15. Describing Systems Architectures in AI Robotics
Subsystems can be thought of in terms of 2 attributes
(relationship) How 3 building blocks, or robot primitives, are arranged
(content) How sensing is handled
This leads to 3 paradigms of system architectures for AI robotics
Hierarchical
Reactive
Hybrid Deliberative/Reactive
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Now lets shift to how AI came up with those subsystems…

16. Recall: AI Primitives within an Agent
SENSE
PLAN
ACT
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
LEARN

17. Hierarchical (1967) PLAN SENSE ACT Objectives Review System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

18. Hierarchical (1967)
PLAN
SENSE
ACT
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Control people hated because didn’t “close the loop”
AI people hated because monolithic
Users hated because very slow

19. Hierarchical Paradigm for Operational Architectures
Advantages of Hierarchies (Albus, Mystel 01)
Natural way to organize
Not intrinsically rigid
Not intrinsically inefficient
not the same as centralized planning
priorities and goals are clear, therefore efficient
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

20. Hierarchical Paradigm: Sequence of 3 Primitives
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
PLAN
SENSE
ACT

21. Hierarchical Paradigm: Sensing is Centralized (global)
SENSE
PLAN
ACT
World model is a fused global data structure.
It combines:
A priori representation
Sensed info
Cognitive understanding
Could update the drawing and colors

22. Hierarchical Paradigm: Notable Systems Architectures
STRIPS/GPS (Nilsson)
Not used anymore, but did spawn planning industry
Shakey
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

23. Shakey
First AI robot
Built by SRI (Stanford Research Institute) for DARPA
Used Strips as main algorithm for controlling what to do
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

24. Hierarchical Paradigm: Notable Systems Architectures
STRIPS/GPS (Nilsson)
Not used anymore, but did spawn planning industry
Shakey
Nested Hierarchical Controller or NHC (Mystel)
Mostly theoretical, oriented towards navigation
Divided into subsystems: mission planner, navigator, pilot, world model, low-level controllers
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

25. Nested Hierarchical Controller (Meystel)
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

26. NHC Planner Objectives Review System Arch - 5 Subsystems - Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

27. Hierarchical Paradigm: Notable Systems Architectures
STRIPS/GPS (Nilsson)
Not used anymore, but did spawn planning industry
Shakey
Nested Hierarchical Controller or NHC (Mystel)
Mostly theoretical, oriented towards navigation
Divided into subsystems: mission planner, navigator, pilot, world model, low-level controllers
NIST Real-time Control System or RCS (Albus)
Integrated NHC and modified: sensory processing, world model, behavior generation, value judgment
Initially adopted by military in late 1980s, later versions try to retrofit to hybrid
Army UGV/XUV program
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

28. Demo III XUV
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Experimental Unmanned Vehicle in action at Ft. Indiantown Gap.
Photo courtesy of the Army Research Labs. Nov. 2001

29. Demo III Control Hierarchy
PLANNER
vehicle1 vehicle2
Section 10m
Vehicle 1m
VEHICLE PLANNER
communications plan AM plan RSTA plan
Subsystem 5s
COMMS PLANNER
message list
AM PLANNER
Driver Plan Gaze plan
RSTA PLANNER
gaze plan
Primitive 500ms
DRIVER PLANNER
Velocity Plan
GAZE PLANNER
Stereo Gaze Plan LADAR Gaze Plan
VELOCITY PLANNER
F Wheels R Wheels F Steer R Steer
Servo 50ms
F Wheel
R Wheel
F Steer
F Steer

30. Disadvantages of Hierarchical Organizational Architectures
While hierarchies have advantages, relying on a world model creates problems
Bottleneck on processing, particularly for control
Alternative is to create layers or hierarchies within the world model to match other subsystems (Kaebling, Simmons)
World model requires extensive representation which leads to two major problems:
Operates under the closed world assumption
Frame problem
In practice, implementations are planning-centric
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating c

31. Reactive (1986) 1 2 PLAN ACT SENSE Objectives Review System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating 2

32. Reactive (1986) 1 2 PLAN Users loved it because it worked
SENSE
PLAN 1
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Users loved it because it worked
AI people loved it, but wanted to put PLAN back in
Control people hated it because couldn’t rigorously prove it worked 2

33. Reactive Paradigm: Two primitives grouped into concurrent behaviors
SENSE-ACT
couplings are
“behaviors”
ACT
SENSE
PLAN
ACT
SENSE
ACT
SENSE
Behaviors are independent,
run in parallel, output is emergent

34. Reactive Paradigm: Sensing is behavior specific (local)
SENSE-ACT
couplings are
“behaviors”
ACT
SENSE
PLAN
ACT
SENSE
ACT
SENSE
Behaviors are independent,
run in parallel, output is emergent

35. Reactive Robots RELEASER SENSE ACT
behavior
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
SENSE
ACT
Behaviors (independent processes), released by perceptual or internal events (state)
No world models or long term memory
Highly modular, generic
Overall behavior emerges

36. Reactive Paradigm: Notable Systems Architectures
Subsumption (Brooks)
No subsystems, just layers of competence
AuRA (Arkin)
Potential field implementation
Will cover in great detail later
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

37. Seriously, what can you without planning?
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

38. Robomow Behaviors? Random Avoid Stop All active Avoid(bump=obstacle)
Avoid(wire=boundary)
Stop
Stop(tilt=ON)
All active
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

39. My Real Baby Behaviors? Touch-> Awake Upside down & Awake-> Cry
Awake & Hungry -> Cry
Awake & Lonely -> Cry
Note can get crying from multiple behaviors
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

40. Reactive Robots Among Us (not so obvious)
Disney
Animatronics
Animation! Hunchback of Notre Dame
Web search engines
Appliances
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

41. Hybrid Deliberative/Reactive (1990)
SENSE
PLAN
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

42. Hybrid Deliberative/Reactive (1990)
SENSE
PLAN
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Control people hated it because no models at the lowest level, but are getting over it
AI people loved it
Users loved it

43. Hybrid Deliberative/Reactive: Plan, then Sense-Act
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Plan, then sense-act until task is complete or need to change;
Note movement towards event-driven planning rather than continuous

44. Hybrid Paradigm: Sensing Organization is Shared
Deliberative functions
Can “eavesdrop”
Can have their own sensors
Have output which looks like a sensor output to a behavior(virtual sensor)
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

45. The Hybrid Deliberative/Reactive System Architecture: the Heart of the Canonical Operational and System Architecture
Social Interface
World
model
cartographer
Performance
& Problem
Solving
Mission
Planner
Sequencer
Sensing
Effectors
Resource Mgr
So if you put them together, you get something that fits nicely with our canonical architecture, just with some names from historical convention. Now, letsEach contributed to the
Behavior
Behavior
Behavior
Behavior
Behavior

46. Contributions of 3 Different Styles
Managerial (division of responsibilities looks like in business)
AuRA (Arkin)
Social Interface
World
model
cartographer
Performance
& Problem
Solving
Mission
Planner
Sequencer
Sensing
Effectors
Resource Mgr
Behavior
Behavior
Behavior
Behavior
Behavior

47. Contributions of 3 Different Styles
Managerial (division of responsibilities looks like in business)
AuRA (Arkin)
State Hierarchies (strictly by time scope)
3T (Bonasso, Kortenkamp, Firby, Gat)
Social Interface
World
model
cartographer
Performance
& Problem
Solving
Mission
Planner
Sequencer
Sensing
Effectors
Resource Mgr
Behavior
Behavior
Behavior
Behavior
Behavior

48. Contributions of 3 Different Styles
Managerial (division of responsibilities looks like in business)
AuRA (Arkin)
State Hierarchies (strictly by time scope)
3T (Bonasso, Kortenkamp, Firby, Gat)
Model-Oriented (models serve as virtual sensors, multiple world models
Saphira (Konolige), TCA (Simmons)
Social Interface
World
model
cartographer
Performance
& Problem
Solving
Mission
Planner
Sequencer
Sensing
Effectors
Resource Mgr
Behavior
Behavior
Behavior
Behavior
Behavior

49. Systems Architecture
Systems architectures have evolved around functions and around paradigm
The hybrid deliberative/reactive paradigm “won” the paradigm wars
Most extensible and reusable
More modular
Hierarchical systems are still favored by DoD and NASA because of perceived efficiency and deterministic
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

50. What goes into the technical architecture?
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
What goes into the technical architecture?

51. Technical Details…
Algorithms such as Hough transforms, genetic algorithms, neural networks, path planning, fuzzy logic, scheduling, resource allocation
Data structures such as potential fields, perceptual spaces, polar plots
Algorithms for coordination and control of software modules such as island driving, vector summation, subsumption, voting, production rules
Displays and interfaces such as natural language, gestures
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

52. Software-Service Flavor
“Software as a service” extension to object-oriented programming
Modules/functions are used rather than owned
Independent of programming language
Stateless
Practical ramifications
Libraries that
subsystems draw from dynamically
Contain logicially equivalent routines with methods to adapt them
Ability to store, distribute those libraries off-board (at a control center)
Proxy processing, centralized control
Use resources of other agents (group minds like Vinge’s Fire Upon the Deep)
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

53. Evaluating the Technical Architecture
support for modularity:
decomposition by functionality specified by the systems architecture
niche targetability:
Can be adapted to domains
ease of portability to other domains:
Infrastructure code should be reusable
robustness:
Often neglected beyond reactive system tendency to “do the right thing”
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

54. Summary and additional thoughts
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating
Summary and additional thoughts

55. Summary: Architectures
An architecture is the Big Picture of how to program an intelligent robot.
“Architecture” in AI robotics is often refers to
The operational architecture which captures the functionality
Three layers
Reactive
Deliberative
Interface
The system architecture developed by a researcher or research group
Three paradigms for the system architecture
Hierarchical
Hybrid deliberative/reactive
The technical architecture, usually a novel technique such as potential fields
The operational and system architecture influences the technical architecture
Objectives
Review
System Arch
- 5 Subsystems
- Paradigms
-Hierarchical
-Reactive
-Hybrid
Technical Arch
- evaluating

56. Summary Can you make the architecture more tangible?
Have functions, the hybrid deliberative/reactive architecture which gives 3 layers
What are the subsystems in a system architecture?
Navigation (Generating)
Cartographer, World Model, World Map (World Model)
Planning (Generating mission, Implementing, Selecting, Monitoring)
Motor Schemas, Behaviors (executing motor commands
Perception, Sensing, Perceptual Schemas (executing sensor input)
BUT THIS IS NOT ALL THAT IS NEEDED OR POSSIBLE
What goes into a technical architecture?
Specific algorithms, control/coordination, knowledge structures

57. Next Lecture
So far- AI is good with symbols (deliberative) and good with skills (reactive) and has ignored people (interface) - it’s the conversion of sensor data “in the middle” that is holding up progress
Telesystems
How do we put people “in the loop” to compensate for the problems in converting sensor data to symbols, i.e., telesystem as a workaround?
How intelligent does a robot need to be for people who want to work remotely through it, such as for telecommuting, telemedicine, surveillance, etc., i.e., telesystem for remote presence?
So far our ex;oration of architectures indicates that
… So see you for lecture 5!