1. Agent Based Software Development
Michael Luck, Ronald Ashri and Mark d’Inverno

2. Agent Architectures What kind of computational entity is an agent?
Why architecture?
to predict what an agent in its current state will do in its current environment
a methodology or blueprint for building agent systems

3. Different Agent Architectures
Single-agent (micro)
reactive agent architectures
deliberative agent architectures
Hybrid agent architectures
Multi-agent (micro and macro)
Macro – norms, protocols, languages
Micro – designing individual agents that can work effectively in a society

4. From the Traditional AI View to Reactive Agent Architectures
Symbolic (internal) representation
Manipulate this representation to work out what to do next
More recent View
Effective behaviour does not need representation and manipulation
Symbolic reasoning is very resource demanding
Agents must be situated and embodied with symbol “reflex” rules

5. Reactive Agent Architectures
behaviour directly coupled with the world
stimulus-response rules
Brooks was a pioneer
Subsumption Architecture
Maes also
Agent network architecture
“Intelligent behaviour is an emergent phenomenon arisiing from the interaction of societies of nonintelligent systems’’

6. Subsumption Architectures
A means of controlling behaviour of mobiles robots in real time – 3 basic requirements
Agents that can cope with multiple goals
Agents should have multiple sensors
Agents should be robust
Function when some sensors fail
Function in dynamic uncertain environments

7. Layered Architecture – Task Achieving Behaviours
Avoid contact with objects
Wander
Explore
Build a map and plan routes
Notice environment change
Identify objects, and perform certain object-related tasks
Build and execute plans to change world in some desirable way
Reason about the behaviour of other objects, and modify plans

8. Subsumption Architecture Operation
Layers unaware of layers above them
Layers able to examine details of layers below them
Each layer connected to all sensors but only extracts information relevant to that layer
Layers can suppress lower layers
suppressing inputs
inhibiting outputs

9. The Subsumption Architecture

10. Pros and Cons of the Subsumption Architecture
Showed decomposition by behaviour rather than function was possible
Modular structure and a clear control methdolology
Impact on other architectures
Cons
No explicit reasoning – how would it scale up?
No design methodology – built on intuition rather than a set of guiding principles
Difficult to perform any formal analysis and so predict or explain behaviour

11. Agent Network Architecture
Based on the notion that intelligent systems consist of societies of mindless interacting systems
Consists of competence modules
compete for behaviour using activation
Activation based on
whether a module is currently executable
perception of the environment
current goals

12. Agent Network Architecture Operation
Modules are linked
activated modules increase the activation of their successors
Non executable models activate their predecessors
All modules decrease activation of their conflictors
Architecture is distributed, modular and robust.
Has not been extended for designing multi-agent systems in general

13. Deliberative Agent Architectures
Maintain and manipulate representations of the world
To model rational or intentional agency use mental attitudes
Eg. Beliefs, goals, desires, knowledge, plans, motivations, intentions
3 Basic categories
Informative (about the world)
Motivational (what drives the agent)
Deliberative (directs agent behaviour)

14. Why Mental Attidues?
First, if an agent can be described in terms of what it knows, what it wants, and what it intends, then,since it is modeled on familiar concepts, it becomes possible for users to understand and predict its behavior.
Second, understanding the relationship between these different attitudes and how they affect behavior can provide the control mechanism for intelligent action in general.
Third, computational agents designed in this way may be able to interpret the behavior of others independently from any implementation.

15. BDI Theory and Architecture
Supports many deliberative architectures
Belief, desire and intention (theory)
Represented as data structures (beliefs, goals and intentions) which determine the operation of an agent
Cannot commit to all its goals as resource bounded so selects some which are the agents’ intentions

16. Success of BDI Many formal models Generic agent architectures
Basis for many previous and current agent architectures (including MAS)
Successfully applied to more than just toy problems or simulations
Air traffic control
Customer-service applications (Agentis)
First BDI system known as the Proceedural Reasoning System (PRS)

17. The PRS Architecture

18. PRS Beliefs, goals, intentions and plan library
Agent perceive the world through external events
Plans – proceedural knowledge
Recipes for action (tree labeled with actions and formulas which evaluate to a boolean)
Trigger (what an agent must perceive)
Context (what an agent must believe)
If plans cannot proceed they post internal events

19. From Plans to Intentions
Agents respond to Internal and External events by selcting an appropriate plan in ts plan based whose trigger and context is true
When a plan is adopted it becomes an intention
This intention become part of the agent’s intention structure

20. PRS Operation 1. Perceive the world, and update the set of events.
2. For each event, generate the set of plans whose trigger condition matches the event. These are known as the relevant plans of an event.
3. For each event, select the subset of relevant plans whose context condition is satisfied by the agent’s current beliefs. These plans are known as active plans.
4. From the set of active plans, select one for execution so that it is now an intention.
5. Include this new intention in the current intention structure either by creating a new intention stack or by placing it on the top of an existing stack.
6. Select an intention stack, take the topmost intention, and execute the next formula in it.

21. AgentSpeak(L)
Attempt to bridge the gap between BDI theories (highly abstract) and implemented BDI systems
It is an abstraction of an implemented BDI system
Captures essence of a PRS system

22. AgentSpeak(L) Data Structures and Operation
Beliefs
Intentions (sequences of plans)
Plans have a trigger, context and a sequence of actions and goals
Agents instantiate plans in response to internal events (goals of executing plans) and external events (which effect a belief change)
Agent execute the intentions

23. Hybrid Agent Architectures
Solely Reactive Systems
Cannot reason about their actions
Will never be able to achieve sophisticated behaviour
Solely Deliberative Systems
Will never be able to act “in time”
If agents are going to be able to survive in complex dynamic environments and achieve complex goals they will need to be If agents are going to be able to survive in complex dynamic environments and achieve complex goals they will need to be both reactive and deliberative

24. Touring Machines Based on the following observations:
Agents need the ability to deal with unexpected events in the real (physical or electronic) world, and do so at different levels of granularity.
Agents need to deal with the dynamism in the environment created by the actions of other agents.
Agents must pay attention to environmental change.
Agents need to reason about temporal constraints in the knowledge that computation is necessarily resource-bounded.
Agents must reason about the impact the short-term actions may have on long-term goals.

25. TouringMachines Architectural Layers
Reactive layer
Responds to events not explicitly programmed in other layers such as obstacle avoidance
Planning Layer
Generating, executing and modifying plans
Modelling Layer
Builds and maintains models of other agents in the environment

26. TouringMachine Architecture

27. TouringMachines
Each layer models world at different level of abstraction
Each directly connected to perception and action
Any two layers can communicate
Global rules used when conflicts between layers arise and can surpress perception or action
Important for showing how to build an agent which exhibited a range of behaviours from deliberative to reactive

28. InterRRaP Another layered architecture Based on BDI
It is vertically layered in constrast to TouringMachines which are horizontally layered
Horizontally layered – each layer can interact with every other layer and with perceptions and actions
Vertically layered – communication only possible between adjacent layers
InterRRaP has four such layers
The essential operation of the agent; in response to events in the environment, control spreads upwards until the appropriate level is reached.

29. InteRRaP Architecture

30. Distributed Agent Architectures
Agents described previously may have architecture that enables them to interact effectively with others at run time
Here the design is concerned with individual agents and the dimensions required – micro-level view
Macro-level view – considers a multiagent system from a holistic view, where interaction, coordination and cooperation between agents are designed in advance of run time

31. Contract Net Protocol Overview
Most commonly employed coordination mechanism for controlling the problem solving behavior of a collection of distributed agents
concerned with the dynamic configuration and coordination of agents to form hierarchies that can achieve complex, distributed tasks.
provides a mechanism by which nodes (or agents) can dynamically create relationships in response to the current processing requirements of the system as a whole, thereby enabling opportunistic task allocation.

32. Contract Net Protocol
1. A node decomposes a task into a number of subtasks and issues a task announcement describing what needs to be performed with eligibility requirements
2. Free nodes bid if they fulfil the eligibility specification
3. Once all bids have been received they are ranked and a contract is awarded to the highest ranked bidder
4. There is now a contract between the manager who made the task announcement and the bidder (the contractor). Over time, the manager monitors the problem solving and integrates partial results
5. The manager terminates a contract with a termination message.

33. Contract Net Protocol

34. The Contract Net Hierarchy
Subcontracting may take place

35. Contract Net Summary
The contract net protocol was a novel means for dynamically allocating tasks in distributed systems
Still commonly found in many agent systems.
However, it offers a very simple coordination mechanism
does not provide any strategies for global coherence
only suited to situations in which the tasks are easily decomposable into
only suitable if agents are not in conflict
does not support mechanisms for bargaining or negotiating
We use it to illustrate general principles involved when considering the design of systems from the macrolevel perspective.

36. Agentis: Agent Communication Languages
FIPA ACL
KQML
Encodes
Domain specific content
Set of addressess
Illocutionary
Typically do not encode how a particular message should be used in the context of a wider communication episode
Agentis – includes set of agent protocols for reliable concurrent request and provision of services and tasks from and to agents.

37. Agentis Agent Interaction Model
Includes the notions of Services and Tasks
Services are requested by human users
Services may call upon a series of tasks as part of a service contract

38. Other Approaches to MacroLevel Organisation
DECAF
Rapid design and execution of agents
Focus on defining generic behaviours that can be re-used
RETSINA
a multiagent infrastructure that comprises a set of services, conventions, and knowledge that enable complex social interactions such as negotiation to take place
TeamCore
based on principle that an agent integration infrastructure (a set of services and conventions) based on theoretical principles of coordination can automate robust coordination among heterogeneous agents in distributed systems

39. Other Agent Approaches: Agent-Oriented Programming
Another approach to building agents is to design a programming language with semantics based on some theory of rational or intentional agency, such as the BDI model, and to program the desired behavior of individual agents directly using mental attitudes.
Agent-Oriented Programming

40. Agent-Oriented Programming
program comprises a set of transition rules that specify how an agent in a given mental state will respond to an input, which may be a set of messages from other agents, by defining its new mental state and any outputs.
One example is PLACA based on AGENT0 which takes a societal view (agents interact with each other to achieve their own goals)

41. PLACA Program
Agent’s state consists of capabilities and its mental state, which comprises beliefs, intentions, and plans.
At every step, an agent collects messages that have been received from others and updates its mental state according to its defining program.
At the beginning of each execution step, those transition rules that are satisfied in the current state are identified, and applied to the current mental state and messages collected from the input buffer.
Once the mental state is updated, messages that need to be sent are placed in the output buffer, and actions that need to be performed are recorded and executed in the next step.
If there is sufficient time before the next tick of the clock, the planner may construct and refine current plans for satisfying intentions.

42. PLACA Interpreter

43. Concurrent MeTaTeM Also takes a societal view
Agents specified using a temporal logic that is directly executable
Agents specified using temporal logic formula
An interpreter attempts to construct a model in which the formula is true
Dynamically changing environment also alters the model so responses must be made to what has just been made true or untrue in the model

44. Specification of a Controller agent in Concurrent MetaTeM
1. If an agent asks, then eventually give the resource to that agent;
2. Do not give to anyone unless they have asked since you last gave to them;
3. If you give to two people, then they must be the same person.

45. Summary Deliberative vs Reactive Micro vs Macro
design for decisions at run time - design occurs before run time
Micro vs Macro