1. Lecture 2: Intelligent Agents Heshaam Faili hfaili@ece.ut.ac.ir University of Tehran What is an intelligent agent? Structure of intelligent agents Environments Examples

2. 2 Intelligent agents: their environment and actions

3. 3 Concepts Environment Sensor Actuator( effectors ) Agent Functions Agent Program Percept and Percept Sequence Rational Agent Performance measure

4. 4 Ideal rational agents For each possible percept sequence, an ideal rational agent should take the action that is expected to maximize its performance measure, based on evidence from the percept sequence and its built-in knowledge. Key concept: mapping from perceptions to actions Different architectures to realize the mapping

5. 5 Structure of intelligent agents Agent program: a program that implements the mapping from percepts to actions: The JOB of AI Architecture: the platform to run the program (note: not necessarily the hardware!) Agent = architecture + program Examples: medical diagnosis- part-picking robot satellite image analysis- interactive tutor refinery controller- flight simulator

6. 6 An Example(Vacuum Cleaner) What is good Performance measure? amount of dirt cleaned up in a single eight-hour shift ?? … Clean and dump it again, … reward the agent for having a clean floor. For example, one point could be awarded for each clean square at each time step As a general rule, it is better to design performance measures according to what one actually wants in the environment, rather than according to how one thinks the agent should behave.

7. 7 Task Environment PEAS: Performance, Environment, Actuator, Sensors

8. 8 More Examples

9. 9 Properties of environments Fully Observable or Partially Observable: is the state of the world fully know at each step? Deterministic to Stochastic: how much is the next state determined by the current state? If the environment is deterministic except for the actions of other agents, we say that the environment is strategic. Episodic to sequential: how much state memory?

10. 10 Properties of environments Static to dynamic: how much independent change? SemiDynamic: If the environment itself does not change with the passage of time but the agent's performance score does Discrete to continuous: how clearly are the actions and percepts differentiated? Single Agent or multi Agent: Competitive or Co- operative

11. 11 Examples of environments

12. 12 Rational or OmniScience An omniscient agent knows the actual outcome of its actions and can act accordingly; but omniscience is impossible in reality. Rationality is not the same as perfection Rationality maximizes expected performance, while perfection maximizes actual performance.

13. 13 Table-Driven Agents function Table-driven-agent(percept) returns action static: percepts, a sequence, initially empty table, indexed by percept sequences (given) append percept to the end of percepts action := LOOKUP(percepts, table) return action Keeps a list of all percepts seen so far Table too large takes too long to build might not be available TAXI DRIVER: need table with 10 250,000,000,000 entries for 1 hour driving (30frames per second 640*480 * 24 color) CHESS: 10 150

14. 14 Simple Reflex Agent (1) e.g. if car-in-front-is-braking then initiate-braking.

15. 15 Simple Reflex Agent (2) function Simple-Reflex-Agent(percept) returns action static: rules, a set of condition-action rules state := Interpret-Input (percept) rule := Rule-Match(state, rule) action := Rule-Action[rule] return action No memory, no planning

16. 16 Simple Reflex example

17. 17 Partially Observable Simple Reflex Vacuum cleaner with only one sensor: Only get Status (Dirty/Clean) Infinite loops unavoidable in partially observable Randomized Simple Reflex

18. 18 Model-based Reflex Agents The most effective way to handle partial observability is to keep track of the part of the world it can't see now. Store some of internal nodes Change internal nodes based on Environments and Agent Actions Model the Environment by some nodes

19. 19 Model-based Reflex Agents

20. 20 Model-based Reflex Agents function Reflex-Agent-State(percept) returns action static: rules, a set of condition-action rules state, a description of the current state state := Update-State (state, percept) rule := Rule-Match(state, rules) action := Rule-Action[rule] state := Update-State (state, action) return action still no longer-term planning

21. 21 Goal-based Agents Knowing the current state is not always enough to decide what to do. For example, the taxi can turn left, turn right, or go straight on. The correct decision depends on where the taxi is trying to get to. GOAL-BASED

22. 22 Goal-based Agents (1)

23. 23 Goal-based Agents (2) function Goal-Based-Agent(percept, goal) returns action static: rules, a set of condition-action rules state, a description of the current state state := Update-State (state, percept) rule := Plan-Best-Move(state, rules, goal) action := Rule-Action[rule] state := Update-State (state, action) return action longer term planning, but what about cost?

24. 24 Utility-based Agents (1) Goals alone are not enough to generate high-quality behavior There are many action sequences that will get the taxi to its destination Add utility evaluation: not only how close does the action take me to the goal, but also how useful it is for the agent Other aspects to be considered: uncertainty in perceptions and actions incomplete knowledge environment characteristics

25. 25 Utility-based Agents (2)

26. 26 Learning Agent How the agent programs come into being? Turing (1950) considers the idea of actually programming his intelligent machines by hand. But he also proposed to build learning machines and then to teach them (Learning). Learning allows the agent to operate in initially unknown environments and to become better and better…

27. 27 Learning Agent - Learning element: responsible for making improvements -Performance elements: responsible for selecting external actions - critic : learning elements uses feedback from Critic -Problem generator: It is responsible for suggesting actions that will lead to new and informative experiences.

28. 28 Learning Agent The learning element can make changes to any of the "knowledge" components shown in the agent diagrams learning directly from the percept sequence. Observation of pairs of successive states ("How the world evolves“) observation of the results of its actions ("What my actions do.“)

29. 29 ?