1. Reactive Paradigm – Overview Subsumption Architecture By Ian Jonkers 159.734 Studies in Machine Learning: Intelligent Robotics

2. Reactive Paradigm Overview Two Representative Reactive Architectures: 1)Subsumption 2)Potential Fields Summation Reactive Paradigm emerged late 1980’s Reactive Paradigm still important for two reasons: 1)Robotic systems in limited task domain still being constructed 2)Forms the basis for the Hybrid Reactive-Deliberative Paradigm Reactive Paradigm grew out of the dissatisfaction of the hierarchical paradigm and with the influx of ideas from biological intelligence.

3. Horizontal Decomposition of Hierarchical Model SENSEPLANACT Sensors Extract Features Combine Features into Model Plan Tasks Task Execution Motor Control Actuators

4. Vertical Decomposition Instead, examination of ethological literature suggests that intelligence is layered in a vertical decomposition: 1.Agent starts with primitive survival behaviours. 2.Evolve new layers of behaviour which either: a)Reuse the lower, older behaviour b)Inhibit older behaviour c)Create parallel tracts of more advanced behaviours 3.Parallel tracks can be thought of as vertically stacked layers 4.Each layer has access to sensors and actuators independently of any other layer. 5.If anything happens to the more advanced layer, lower levels should still operate. i.e. human brain (breathing) continue independently of higher order functions (counting, face recognition).

5. Vertical Decomposition SENSEACT SENSEACT SENSEACT SENSEACT Sensors Actuators Avoid Collisions wander explore Build maps Vertical decomposition of tasks into a S-A Organisation, associated with the Reactive Paradigm

6. Attributes of Reactive Paradigm The fundamental attribute of reactive paradigm is all actions are accomplished through behaviours As in ethological systems, behaviours are a direct mapping of sensory inputs to a pattern of motor actions that are then used to achieve a task. Behaviour SENSEACT Behaviour SENSEACT Behaviour SENSEACT Behaviour SENSEACT S-A organisation of the Reactive Paradigm into multiple, concurrent behaviours

7. Attributes of Reactive Paradigm (continued) From mathematical perspective, behaviours are simply a transfer function. The Reactive Paradigm essential threw away the PLAN component. The SENSE & ACT are tightly coupled into behaviours & sequential or concurrent robotic activities emerge. Sensing is local to each behaviour, but sensors may be shared and is immediately available to the behaviour's perceptual schema which can be computationally inexpensive.

8. Characteristics of Reactive Behaviours Reactive robotic systems execute rapidly (tight coupling of senses permits real-time operation) Behaviours can be implemented directly in hardware circuits or low computational complexity algorithms (O(n)). Have no memory (limiting behaviours to stimulus-response reflexes) Main point “Behaviours controlled by what is happening in the world, duplicating the spirit of the innate releasing mechanisms, rather than the program storing & remembering what the robot last did” Five Characteristics of reactive Paradigm are: 1)Situated Agent (integrated part of world) Robots are situated agents operating in an ecological niche. i.e. when a robot acts, it changes the world, and receives immediate feedback about the world through sensing.

9. 2)Emergent Behaviours Behaviours serve as the basic building blocks for robotic action, and the overall behaviour of the robot is emergent. Behaviours are independent computational entities and operate concurrently, Hence there is no explicit “controller” module which determines what will be done, or which function call other functions. 3)Eco-centric Only local, behaviour specific sensing is permitted. i.e. does not matter that an obstacle is in the world at coordinates (x,y,z), only where it is relative to the robot. 4) Modular Behaviours These systems inherently follow good software design principles. The modularity of behaviours supports the decomposition of a task into component behaviours. 5) Biological Motivation Animal models of behaviour are often cited as a basis for these systems or a particular behaviour (unlike earlier AI days where a conscious effort not to mimic biological intelligence was made) Characteristics of Reactive Behaviours (continued)

10. Advantages of Programming by Behaviour Constructing a robotic system under Reactive Paradigm often referred to as programming by behaviour. Good Software Engineering since Behaviours are Modular. Robot becomes more intelligent by having more behaviours. Modules have Low Coupling  can function independently of each other with minimal connections or interfaces  promoting easy reuse. Modules have High Cohesion  Data and operations contained by a module relate only to the purpose of that module.

11. Reactive Paradigm Representative Architectures The overall action of the robot emerges from the multiple concurrent behaviours and the architecture must provide mechanisms for: 1)Triggering behaviours 2)Conflict resolution when multiple behaviours are active at any one time. The two most well known Reactive Architectures are 1)Potential Fields – Behaviours combined by summation of fields. 2)Subsumption – Decomposition into layers of task achieving behaviours.

12. Subsumption Architecture Rodney Brooks Subsumption Architecture most influential of the purely Reactive Paradigms. Many look like shoe-box sized insects (6 legs and antennae) Implementations quite often have the behaviours embedded directly in the hardware or small micro processors (unheard pre mid 1980’s) Robots could now walk, avoid collisions and climb over obstacles without the move-think-move-think pauses of Shakey. A behaviour is a network of sensing and acting modules which accomplish a task. The modules are Augmented Finite State Machines (AFSM), or finite state machines which have registers, timers & other enhancements to permit them to be interfaced with other modules.

13. Behaviours are released in a stimulus response way, without an external program explicitly coordinating and controlling them. Layers of Competence: The layers reflect a hierarchy of intelligence or competence.  lower layers encapsulate basic survival functions (collisions)  higher levels create more goal directed actions (mapping) Each of the layers can be viewed as an abstract behaviour for a particular task. Subsumption of lower layers: - Modules in a higher layer can override or subsume the output from behaviours in the next lower layers. - Behaviour layers operate concurrently and independently and hence need mechanism to handle potential conflicts  winner always takes all (always the highest layer) Subsumption Architecture (continued)

14. No World Model (Internal State) - No persistence representation of the world model since information comes directly from the world. - Dangerous to depend on internal state since may diverge from reality. - Some internal state needed for releasing behaviours (i.e. scared, hungry), but good design minimises this. Taskable - Tasks are accomplished by activating the appropriate layer, which then activate the lower layers. - In practice, Subsumption style systems are not easily taskable, that is, they can’t be ordered to do another task without being reprogrammed. Subsumption Architecture (continued)

15. Obstacle Avoidance Example Sensor: A SONAR module that gives the distance to the objects in polar coordinates. Internal Modules –COLLIDE  detects if front obstacle is too close (i.e. halt) –FEELFORCE  sensor reading acts as repulsive force field –RUNAWAY  provides direction to move Actuators –TURN  provides motor output to turn robot –FORWARD  switches forward motion on or off

16. Sonar Module “robo-centric” view of range readings 0 1 2 3 4 7 6 5 01345672 Unrolled into a plot  Sonar module reads the sonar ranges.  Polar plot range readings in polar coordinates (r, ) surrounding the robot.

17. Level 0 Architecture: obstacle avoidance

18. Level 1 Architecture: wander How does Turn know which module to take heading from? Wander - Can’t pass directly onto Turn since will sacrifice obstacle avoidance

19. Suppression – replaces all other inputs to the module with input coming from the “suppressing” module.

20. Inhibition – blocks output from the specified module for the defined time interval.

21. Level 2 Architecture: follow corridor Integrate - estimates how far robot has travelled off course  Supplies dangerous internal state

22. Evaluating Subsumption Architecture Modularity:  behaviours are modular, but generally favour a hardware implementation Niche targetability:  high due to direct perception. Portability:  limited to tasks which can be accomplished with reflexive behaviours  generally can’t be transferred where planning is needed Robustness:  offers graceful degradation if anything should disable a higher level behaviour, then the lower level can be left intact.

23. Summary Layers of abstract behaviours:  achieved by grouping schemas like modules into layers. Suppression and inhibition of lower layers:  Higher layers may subsume & inhibit behaviours in lower layers, but the behaviours in lower layers never rewritten or replaced.  Mimics biological evolution (i.e. frogs with large objects) Difficult to design:  More of a art form than a science. Behaviour Release Mechanisms  Behaviours are released by the presence of stimulus in the environment.

24. Summary (continued) Solution to the Frame Problem:  Solves the frame problem by eliminating the need to model the world.  Behaviours don’t remember the past. Direct Perception and Affordance  The release for a behaviour is almost always the percept for guiding the motor schema. Perception is Eco-centric & Distributed  i.e. sensing objects relative to the robot and the sensors can be shared among modules.

25. References Robin R. Murphy - Introduction to AI robotics - Cambridge, Mass. : MIT Press, 2000 Jonathan SIMPSON, Christian L. JACOBSEN and Matthew C. JADUD - Mobile Robot Control - The Subsumption Architecture and occam-pi - Communicating Process Architectures 2006 http://www.jonsimpson.co.uk/weblog/2006-09-24/mobile-robot-control-the- subsumption-architecture-and-occam-pi.html