1. Ontologies Reasoning Components Agents Simulations Introduction to Intelligent Agents Jacques Robin

2. Outline  What are intelligent agents?  Characteristics of agents  Characteristics of agents’ environments  Agent architectures

3. What is an Agent? General Minimal Definition  Any entity (human, animal, robot, software):  Situated in an environment (physical, virtual or simulated) that  Perceives the environment through sensors (eyes, camera, socket)  Acts upon the environment through effectors (hands, wheels, socket)  Possess its own goals, i.e., preferred states of the environments (explicit or implicit)  Autonomously chooses its actions to alter the environment towards its goals based on its perceptions and prior encapsulated information about the environment  Processing cycle: 1.Use sensor to perceive P 2.Interprets I = f(P) 3.Chooses the next action A = g(I,G) to perform to reach its goal G 4.Use actuator to execute A Artificial Intelligence Agents Distributed Systems Software Engineering

4. What is an Agent? Autonomous Reasoning Agent Environment Sensors Effectors Goals Action Choice: A = g(I,O) A Perception Interpretation: I = f(P) P 1.Environment percepts 2.Self-percepts 3.Communicative percepts 1.Environment altering actions 2.Perceptive actions 3.Communicative actions

5. Agent x Object  Intentionality: Encapsulate own goals (even if implicitly) in addition to data and behavior  Decision autonomy:  Pro-actively execute behaviors to satisfy its goals  Can negate request for execution of a behavior from another agent  More complex input/output: percepts and actions  Temporal continuity: encapsulate an endless thread that constantly monitors the environment  Coarser granularity:  Encapsulate code of size comparable to a package or component  Composed of various objects when implemented using an OO language  No goal  No decision autonomy:  Execute behaviors only reactively whenever invoked by other objects  Always execute behavior invoked by other objects  Simpler input/output: mere method parameters and return values  Temporally discontinuous: active only during the execution of its methods

6. Intelligent Agent x Simple Software Agent Environment Sensors Effectors Goals Percept Interpretation: I = f(P) Action Choice: A = g(I,O) Conventional Processing Conventional Processing AI

7. Intelligent Agent Environment Sensors Effectors Goals Percept Interpretation Action Choice AI Situated Agent Reasoning Input Data Output Data GoalDisembodiedAISystem AI Classical AI System

8. What is an Agent? Other Optional Properties  Reasoning Autonomy:  Requires AI, inference engine and knowledge base  Key for: embedded expert systems, intelligent controllers, robots, games, internet agents...  Adaptability:  Requires IA, machine learning  Key for: internet agents, intelligent interfaces,...  Sociability:  Requires AI + advanced distributed systems techniques:  Standard protocols for communication, cooperation, negotiation  Automated reasoning about other agents’ beliefs, goals, plans and trustfulness  Social interaction architectures  Key for: multi-agent simulations, e-comerce,...

9. What is an Agent? Other Optional Properties  Personality:  Requires AI, attitude and emotional modeling  Key for: Digital entertainment, virtual reality avatars, user-friendly interfaces...  Temporal continuity and persistence:  Requires interface with operating system, DBMS  Key for: Information filtering, monitoring, intelligent control,...  Mobility:  Requires:  Network interface  Secure protocols  Mobile code support  Key for: information gathering agents,...  Security concerns prevented its adoption in practice

10. Welcome to the Wumpus World! Agent-Oriented Formulation:  Agents: gold digger  Environment objects:  caverns, walls, pits, wumpus, gold, bow, arrow  Environment’s initial state  Agents’ goals:  be alive cavern (1,1) with the gold  Perceptions:  Touch sensor: breeze, bump  Smell sensor: stench  Light sensor: glitter  Sound sensor: scream  Actions:  Legs effector: forward, rotate 90º  Hands effector: shoot, climb out

11. Wumpus World: Abbreviations 1 2 3 4123 4 start S A B P W B B S S, B, G P P B B G A - Agent W - Wumpus P - Pit G - Gold X? – Possibly X X! – Confirmed X V – Visited Cavern B – Breeze S – Stench G – Glitter OK – Safe Cavern

12. Perceiving, Reasoning and Acting in the Wumpus World  Percept sequence: 1 2 3 4123 4 A ok t=0 nothing t=2 breeze 1 2 3 4123 4 ok A V P? b  Wumpus world model maintained by agent:

13. 1 2 3 4123 4 ok A VV b ok W! s ok P! stench t=7: Go to (2,1), Sole safe unvisited cavern  Percept sequence:  Wumpus World Model: Perceiving, Reasoning and Acting in the Wumpus World t=11: Go to (2,3) to find gold 1 2 3 4123 4 ok A S ok VV b ok P! W! V ok V S B G P? ok {stench, breeze, glitter}  Action Sequence:

14. Classification Dimensions of Agent Environments  Agent environments can be classified as points in a multi-dimensional spaces  The dimensions are:  Observability  Ramification Type  Determinism  Dynamicity  Mathematical domains of the variables  Episodic or not  Multi-agency  Size  Diversity

15. Observability  Fully observable (or accessible):  Agent sensors perceive at each instant all the aspects of the environment relevant to choose best action to take to reach goal  Partially observable (or inaccessible, or with hidden variables)  Sources of partial observability:  Realm inaccessible to any available sensor  Limited sensor scope  Limited sensor sensitivity  Noisy sensors

16. Determinism  Deterministic:  Deterministic: all occurrence of executing a given action in a given situation always yields same result  Non-deterministic (or stochastic):  Non-deterministic (or stochastic): action consequences partially unpredictable  Sources of non-determinism:  Inherent to the environment: quantic granularity, games with randomness  Other agents with unknown or non-deterministic goal or action policy  Noisy effectors  Limited granularity of effectors or of the representation used to choose the actions to execute

17. RamificationType  Non-ramifying: actions of agent(s) have only the direct effect of achieving their intended goal  action a executed at instant t so that action-sensitive environment property f (called fluent) holds at instant t+1 to achieve goal g(t) has no other effects  i.e., all other fluents that held at instant t still hold at instant t+1 and all fluents that did not hold at instant t still don’t at time t+1 )  Boundedly ramifying: actions of agent(s) also have indirect (or side) effects but in finite number and of finite time horizon  action a executed at instant t so that action-sensitive environment property f (called fluent) holds at instant t+1 to achieve goal g(t) also has other indirect effects  due to causal synchronic or diachronic relationships between f and other fluents  e.g., holds(f,t)   hold(f+,t)  holds(f+,t)  holds(f,t)  hold(f-,t)   holds(f-,t)  holds(f,t)   hold(f+,t)  holds(f+,t+1)  holds(f,t)  hold(f-,t)  holds(f-,t+1)  Unboundely ramifying: actions of agent(s) also have indirect, synchronic or diachronic effects of in potentially infinite numbers  e.g., momentum of an accelerated body in absolute void

18. Dynamicity: Static and Sequential Environments  Static: Single perception-reasoning-action cycle during which environment is static  Sequential: Sequence of perception-reasoning-action cycles during each of which the environment changes only as a result of the agent’s actions Percept Static Environment Agent Action State 1State 2 Reasoning Percept Sequential Environment Agent Action State 1 Reasoning PerceptAction State 2 Reasoning PerceptAção State 3 Reasoning State N...

19.  Concurrent synchronous: Environment can change on its own between one action and the next perception of an agent, but not during its reasoning  Concurrent asynchronous: Environment can change on its own at any time, including during the agent’s reasoning Dynamicity: Concurrent Synchronous and Asynchronous... Percept Synchronous Concurrent Environment Agent Action State 1 Reasoning PerceptAction State 2 Reasoning State 4State 5 State 3... Percept Asynchronous Concurrent Environment Agent Action State 1 Reasoning State 2 State 4 State 3 PerceptAction State 5 Reasoning State 6

20.  Stationary: The underlying laws or rules that govern state changes in the environment are fixed and immutable; they remain the same during the entire lifetime of the agent  ex, a soccer game is asynchronous, yet stationary  Non-Stationary: The underlying laws or rules that govern state changes in the environment are themselves subject to dynamic changes (meta-level changes) during the lifetime of the agent  ex, an accounting agent acts in a non-stationary environment, since the tax laws are subject to changes from one year to the next Dynamicity: Stationary and Non-Stationary

21. Multi-Agency  Sophistication of agent society:  Number of agent roles and agent instances  Multiplicity and dynamicity of agent roles and groups  Communication protocols, cooperation and negotiation schemas  Main classes:  Mono-agent  Multi-agent cooperative  Multi-agent competitive  Multi-agent cooperative and competitive  With static or dynamic coalitions

22. Mathematical Domain of Variables Binary Dichotomical Boolean Qualitative Nominal Ordinal Quantitative Interval Fractional Discrete Continuous R [0,1]  MAS variables:  Parameters of agent percepts, actions and goals  Attributes of environment objects  Arguments of environment relations, states, events and locations

23. Mathematical Domain of Variables  Binary:  Boolean, ex, Male  {True,False}  Dichotomic, ex, Sex  {Male, Female}  Nominal (or categorical)  Finite partition of set without order nor measure  Relations: only = ou   ex, Brazilian, French, British  Ordinal (or enumerated):  Finite partition of (partially or totally) ordered set without measure  Relations: only =, , , >  ex, poor, medium, good, excellent  Interval:  Finite partition of ordered set with measure m defining distance d:  X,Y, d(X,Y) = |m(X)-m(Y)|  No inherent zero  ex, Celsius temperature  Fractional (or proportional):  Partition with distance and inherent zero  Relations: anyone  ex, Kelvin temperature  Continuous (or real)  Infinite set of values

24. Other Characteristics  Episodic:  Agent experience is divided in separate episodes  Results of actions in each episode, independent of previous episodes ex.: image classifier is episodic, chess is not soccer tournament is episodic, soccer game is not  Open environment:  Partially observable, Non-deterministic, Non-episodic, Continuous Variables, Concurrent Asynchronous, Multi-Agent.  ex.: RoboCup, Internet, stock market

25. Size and Diversity  Size, i.e.,  Size, i.e., number of instances of:  Agent percepts, actions and goals  Environment agents, objects, relations, states, events and locations  Dramatically affects scalability of agent reasoning execution  Diversity,  Diversity, i.e., number of classes of:  Agent percepts, actions and goals  Environment agents, objects, relations, states, events and locations  Dramatically affects scalability of agent knowledge acquisition process