Sept 14, 2015 Introduction to Intelligent Agent Dr. Hao WANG, R&L Group, CS, NJU

2 What is an intelligent agent? Fields that motivate agent technologies  Artificial intelligence  AI is the intelligence by machines or software  Major AI researchers and textbooks define this field as "the study and design of intelligent agents" [wikipedia: artificial intelligence]artificial intelligence Artificial Intelligence: Foundations of Computational Agents [link], Cambridge University Press, 2010link "AI is the filed that studies the synthesis and analysis of computational agents that act intelligently. " Agent is the carrier of artificial intelligence. Topics in the book: reasoning, learning, planning, etc.

3 What is an intelligent agent? Fields that motivate agent technologies  Software engineering  SE is the study and an application of engineering to the design, development, and maintenance of software. [wikipedia: software engineering]software engineering M. Wooldridge [Wooldridge 1998] System Environment inputoutput [Wooldridge 1998] M. Wooldridge, Agent and software engineering, in: AIIA, OS, physical world,... functional, reactive,... * Image from Aug 4, * What is a good software / system? Complexities come from: environment, interaction, system specification, …

4 What is an intelligent agent? Fields that motivate agent technologies  Distributed system  A software system in which components located on networked computers communicate and coordinate their actions by passing messages [wikipedia: distributed computing]distributed computing 战区高空防御导弹战区防空与导弹防御 军事战略战术中继卫星 联合监测目标攻击雷达系统 天基红外系统

5 What is an intelligent agent? Fields that motivate agent technologies  Economics  In economics, an agent is an actor and more specifically a decision maker in a model of some aspect of the economy. [wikipedia: agent (economics)]agent (economics) [wikipedia: program trading]program trading [wikipedia: algorithmic trading]algorithmic trading [wikipedia: high-frequency trading]high-frequency trading

6 What is an intelligent agent? Fields that motivate agent technologies  Economics  In economics, an agent is an actor and more specifically a decision maker in a model of some aspect of the economy. [wikipedia: agent (economics)]agent (economics) paid listing gaming / bidding

7 Gradual changes lead to crisis Big data  Volume  Velocity  Variety  Veracity  Example: find the top-k frequent integers in…  an array, a file, a database, a distributed archive, a stream * Image from Aug 6, * With the growth of complexity, there will be a point where we need brand new tools and even brand new ways of thinking.

8 Agent: general definitions Merriam-Webster  A person who does business for another person: a person who acts on behalf of another  A person or thing that causes something to happen [Russell&Norvig 2009] S. Russell & P. Norvig, Artificial Intelligence: A Modern Approach (3 rd edition), Prentice Hall, 2009.Artificial Intelligence: A Modern Approach (3 rd edition) S. Russell & P. Norvig  An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors. [Russell&Norvig 2009]

9 [Maes 1995] P. Maes, Artificial life meets entertainment: lifelike autonomous agents, Communications of the ACM, 38(11): , 1995.Artificial life meets entertainment: lifelike autonomous agents P. Maes  Autonomous agents are computational systems that inhabit some complex, dynamic environment, sense and act autonomously in this environment, and by doing so realize a set of goals or tasks for which they are designed. [Maes 1995] Agent: general definitions D. Smith, A. Cypher & J. Spohrer  Let us define an agent as a persistent software entity dedicated to a specific purpose.  “Persistent” distinguishes agents from subroutines; agents have their own idea about how to accomplish tasks, their own agendas. “Specific purpose” distinguishes them from entire multifunction applications; agents are typically much smaller. [Smith et al. 1994] [Smith et al. 1994] D. Smith, A. Cypher & J. Spohrer, KidSim: programming agents without a programming language, Communications of the ACM, 37(7): 54-67, 1994.KidSim: programming agents without a programming language

10 [Hayes-Roth 1995] B. Hayes-Roth, An architecture for adaptive intelligent systems, Artificial Intelligence, 72(1): , 1995.An architecture for adaptive intelligent systems B. Hayes-Roth  “Intelligent agents” continuously perform three functions: perception of dynamic conditions in the environment; action to affect conditions in the environment; and reasoning to interpret perceptions, solve problems, draw inferences, and determine actions.  Conceptually, perception informs reasoning and reasoning guides action, although in some cases perception may drive action directly. [Hayes-Roth 1995] Agent: general definitions

11 [Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: , 1995.Intelligent agents: theory and practice … the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:  Autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state;  Reactivity: agents perceive their environment, and respond in a timely fashion to changes that occur in it; [Wooldridge&Jennings 1995] Agent: a weak notion

12 [Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: , 1995.Intelligent agents: theory and practice … the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:  Proactiveness: agents do not simply act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative;  Social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language. [Wooldridge&Jennings 1995] Agent: a weak notion

13 [Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: , 1995.Intelligent agents: theory and practice Stronger notions = weak notions + …  Mentalistic notions: conceptualized or implemented using notions such as  Knowledge  Belief  Intention  Obligation  Emotional notions  human-like [Wooldridge&Jennings 1995] Agent: stronger notions * Image from Aug 10, * ** Image from Aug 10, **

14 [Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: , 1995.Intelligent agents: theory and practice [Wooldridge&Jennings 1995] Agent: other properties Other properties  Mobility: agent is able to move around an electronic network (to perform local processing of information);  Veracity: agents will not knowingly communicate false information;  Benevolence: agents do not have conflicting goals, and that every agent will therefore always try to do what is asked of it;  Rationality: an agent will act in order to achieve its goals, and will not act in such a way as to prevent its goals being achieved.

15 An agent inhabits in its environment, which is typically dynamic & complex. An agent perceives its environment, and affects its environment by its actions, and from its perception to its action there is typically a decision-making process (reasoning, planning, …). An agent usually has the so-called weak characteristics: autonomy, reactivity, proactiveness, and social abilities. An agent may have stronger characteristics, being mentalistic or even emotional. In some applications, an agent may be designed with emphases on other characteristics, such as mobility, veracity, benevolence, and rationality. Agent: summary

16 Synonyms of the term “intelligent agent”  Robot, software agent, softbot, knowbot, taskbot, userbot, … Chinese translations of the term “intelligent agent”  “ 智能代理 ”  A person who does business for another person: a person who acts on behalf of another  “ 智能体 ” 、 “ 智能主体 ”  A thing that causes something to happen; the carrier of AI  “ 智能艾真体 ”  “ 以音译为基础，又兼顾了一定的物理含义，而且 ‘ 艾 ’ 含有漂亮的 意思。 ” Synonyms and Chinese Translations [Cai&He 2002, Cai 2003?] [Cai&He 2002] 蔡自兴, 贺汉根, 智能科学发展的若干问题, 自动化学报, 28(suppl): , [Shi 2000] [Shi 2000] 史忠植, 智能主体及其应用, 科学出版社, [Cai 2003?] 蔡自兴, 对 Agent 中文译法的讨论, Available online: Accessed: Aug 6,

17 Examples of agents Air-conditioner / any control system  Thermostat Software daemon  Print server  Http server

18 Think… Find some other examples of agents (not necessarily intelligent); in particular, specify:  1. The environment that the agent inhabits;  2. Possible actions of the agent, and how each of them will change the environment;  3. The goal or design purpose of the agent.

19 Think… If a traffic light together with its control system is designed as an intelligent agent, what are the properties that this agent should exhibit?  1. Weak properties? (autonomous, reactive, proactive, social)  2. Strong properties? (mental, emotional)  3. Other properties? (mobile, veracious, benevolent, rational)

20 The environment Why is the ENVIRONMENT important when we are interested in AGENTS?  From the perspective of problem solving, the environment, together with the goal of the agent, defines the PROBLEM;  Understanding the environment can largely guide the design of the intelligent agent.

21 The environment [Sutton&Barto 1998] The thing the agent interacts with, comprising everything outside the agent is called the environment. Agent-environment boundary is not often the same as the physical boundary of a robot’s body.  Motors, mechanical linkages, sensing hardware Anything that cannot be changed arbitrarily by the agent should be considered part of its environment.

22 Properties of environments Accessibility: accessible vs. inaccessible Determinism: deterministic vs. nondeterministic Dynamism: static vs. dynamic Continuity: discrete vs. continuous [Russell&Norvig 2009] S. Russell & P. Norvig, Artificial Intelligence: A Modern Approach (3 rd edition), Prentice Hall, 2009.Artificial Intelligence: A Modern Approach (3 rd edition) [Russell&Norvig 2009]

Inaccessible (partially observable)  The agent needs to keep track of the environment through its internal state.  Example: observe the current weather  in a windowless room  at the South Pole of Pluto  Accessible + noise = inaccessible 23 Accessibility Accessible (fully observable)  An accessible environment is one in which can obtain complete, accurate, up-to-date information about the environment’s state.  Example: observe the weather, is it sunny / rainy / cloudy / …? Most real-world environments are inaccessible. The more accessible an environment is, the simpler it is to build agents.

Nondeterministic (stochastic)  Nondeterminism captures the facts that…  Agents have at best partial control over their environment  Actions may fail to have the desired result 24 Determinism Deterministic  A deterministic environment is one in which any action has single guaranteed effect.  There is no uncertainty about the state that will result from performing an action.  A deterministic environment may exhibit nondeterminism if it’s sufficiently complex  Lack of a good environment model  Lack of complete sensor coverage

25 Dynamism Static  A static environment is one that can be assumed to remain unchanged except by the performance of actions by the agent.

Continuous  A continuous environment is one that may be in uncountably many states.  The information a discrete-state agent uses in order to select an action in a continuous environment will be made on the basis of information that is inherently approximate. 26 Continuity Discrete  A discrete environment is one that can be guaranteed to only ever be in a finite number of discrete states. Most complex general class of environments  Inaccessible, nondeterministic, dynamic, and continuous  a.k.a. open environments

 Simplest case  The agent knows (1) the maze and (2) the goal position.  Actions of the agent have deterministic effects.  The maze is fixed.  accessible, deterministic, static, and discrete 27 Examples of environments: maze games Maze games are common testbeds for intelligent agents N S E W agent goal position obstacles

 If actions are performed via motors…  There are chances of mechanical failure  accessible, nondeterministic, static, and discrete 28 Examples of environments: maze games N S E W agent goal position obstacles  If, in addition, the sight is limited…  inaccessible, nondeterministic, static, and discrete

29 Examples of environments: maze games  If there is another agent, or the maze is random…  inaccessible, nondeterministic, dynamic, and discrete  If the maze becomes more practical…  inaccessible, nondeterministic, dynamic, and continuous

30 Think… Determine the types of the following environments Mine- sweeper E- shopping PokerChess Deterministic? Accessible? Discrete? Static?

31 Agents vs. functional systems [Recall] An intelligent agent is:  Proactive: goal-directed  Reactive: perceive and respond to its environment  Sociable: interact with other agents / human Functional system  Precondition + postcondition  Example: design by contract (DbC)design by contract * * Image from (license: CC0 1.0), Aug 11, accessible, deterministic, static, discrete inaccessible, nondeterministic, dynamic, continuous

32 Agents vs. objects Object  Computational entities that encapsulate some state, are able to perform actions, or methods on this state, and communicate by message passing  Combination of variables, functions, and data structures [Wooldridge&Ciancarini 2001] M. Wooldridge & P. Ciancarini, Agent-oriented software engineering: the state of the art, in: Proceedings of the 1 st International Workshop on Agent-oriented Software Engineering, 2001.Agent-oriented software engineering: the state of the art [Wooldridge&Ciancarini 2001] [Wikipedia: Object (computer science)]Object (computer science) a set of assignments to variables W ALL ·E move_north( ) move_south( ) move_east( ) move_west( ) (x, y)

33 Agents vs. objects Differences between agents and objects  The level of autonomy  Objects have control over its internal state (encapsulation: private variables) W ALL ·E move_north( ) move_south( ) move_east( ) move_west( ) (x, y)  Objects invoke other objects’ methods, whereas agents can only request other agents to perform actions. Objects do it for free; agents do it for money.  Objects have no control over its behavior: once a method is made public, objects cannot decide whether or not that method is executed.

34 Agents & OOP OOP: object-oriented programming  Standard OOP does not require agent-like properties.  In standard OOP, there is a single thread of control, whereas in principle each agent should have its own thread of control.  Agent is similar to active object. W ALL ·E move_north( ) move_south( ) move_east( ) move_west( ) (x, y) run( )

35 Agents vs. expert systems Expert system  Computer system that emulates the decision-making ability of human expert  Designed to solve complex problems by reasoning about knowledge, represented primarily as if-then rules rather than through conventional procedural code Key difference  Expert systems are not coupled to their environment;  Expert systems are not designed for reactive, proactive behavior;  Expert systems do not consider social ability. [Wooldridge 2009] M. Wooldridge, An Introduction to Multiagent Systems (2 nd edition), John Wiley & Sons, [Woooldridge 2009] [Wikipedia: expert system]expert system

36 Distributed Artificial Intelligence (DAI) DAI is a subfield of AI Dedicated to the development of distributed solutions for complex problems regarded as requiring intelligence Main areas of DAI – Distributed problem solving (DPS) Centralized control and distributed data – Multi-agent system (MAS) Distributed control and distributed data [Wikipedia: distributed artificial intelligence]distributed artificial intelligence

37 DAI considers… Agent granularity (agent size) Heterogeneity of agents (agent type) Methods of distributed control (among agents) Communication MAS – Coarse agent granularity – High-level communication distributed computing distributed AI AI MAS DPS

38 Motivation behind MAS To solve problems too large for centralized agents – e.g., economic system To allow interconnection and interoperation of multiple legacy systems – e.g., web crawling To provide solutions to inherently distributed systems To provide solutions when expertise is distributed To provide conceptual clarity and simplicity of design

39 Benefits of MAS Faster problem solving Decreasing communication – Higher semantics of communication (speech-act level) Flexibility Increasing reliability

40 Heterogeneity degrees in MAS Low – Identical agents, different resources Medium – Different agent expertise High – Share only interaction protocol (e.g. FIPA or KQML)

41 Cooperative and self-interested MAS Cooperative – Agents are designed by interdependent designers – Agents act for the common good of the system – Focus on the performance of the entire system, not the interests of individual agents Self-interested – Agents are designed by independent designers – Agents have their own agenda and motivation – Concerned with the benefit of each agent (‘individualistic’) Which is more realistic in an Internet-setting?

42 Our categorization of MAS Cooperative MAS (teamwork) – Agents share a common objective Competitive MAS – Each agent has its own objective, and these objectives are contradictory Coopetitive ( 竞合 ) MAS – Each agent has its own objectives; these objectives are typically competitive, but there is still possibility that agents may cooperate to achieve their common good

43 Summary Agent has general definitions, weak definitions, and strong definitions Classification of the environment Differences between agents and functional systems, objects, expert systems, etc. A brief introduction to multi-agent systems