1. Chapter 2 Hande AKA

2. Outline Agents and Environments Rationality The Nature of Environments Agent Types

3. Agents and Environments An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators Human agent Eyes, ears,and other organs for sensors Hands,legs,mouth,and other body parts for actuators Robotic agent Cameras and infrared range finders for sensors and various motors for actuators

4. Agent function & Agent program The term percept means the agent’s perceptual inputs at any given instant. The agent’s percept sequence is the complete history of everything the agent has ever perceived. The agent function maps any given percept sequence to an action.(The mathematical description : [f: P*  A ] ) The agent program is the concrete implementation, running on the agent architecture.

5. A vacuum-cleaner example It perceives which square it is and whether there is a dirt in the square. (ex: [A,dirty] ) Possible actions : move right, move left, suck up the dirt or do nothing.

6. Rational agents A rational agent is one that does the right thing.Right actions will cause the agent to be most successful. How can we measure the success?

7. Performance Measures An objective criterion for success of an agent's behavior. Performance measure of a vacuum-cleaner agent could be amount of dirt cleaned up, amount of time taken, amount of electricity consumed, amount of noise generated, etc.

8. Rationality What is rational at any given time depens on ;  The performance measure  The agent’s prior knowledge of the environment  The actions that the agent can perform  The agent’s percept sequence

9. Definition of a rational agent For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

10. Rationality ≠ Omniscience An omniscient agent knows the actual outcome of its actions, but omniscience is impossible in reality. Rationality is not the same as perfection.

11. Rationality Our definition requires;  Information gathering/exploration -Doing actions in order to modify future percepts  Learning -Extending prior knowledge  Autonomy - Compensate for incorrect prior knowledge

12. The nature of environments To design a rational agent we must specify its task environment. PEAS description of the environment  Performance  Environment  Actuators  Sensors

13. Automated taxi driver example PEAS description of the environment Performance -Safety, destination, profits, legality, comfort Environment - Streets/freeways, other traffic, pedestrians, weather Actuators -Steering, accelerating, brake, horn, speaker/display Sensors - Video, sonar, speedometer, engine sensors, keyboard, GPS

14. Examples of agent types and their PEAS description

15. Environment Types 1. Fully observable vs. partially observable An environment is fully observable when the sensors can detect all aspects that are relevant to the choice of action. An environment might be partially observable because some parts of the state are missing from the sensor data. ex: A vacuum agent with only a local dirt sensor cannot tell wheter there is a dirt in other squares.

16. 2. Deterministic vs. stochastic If the next state of the environment is determined by the current state and the action executed by the agent, then, the environment is deterministic, otherwise, it is stochastic.  Taxi driving is stochastic, because one can never predict the behaviour of the traffic exactly. If the environment is deterministic except for the actions of other agents, the environment is strategic. (playing chess )

17. 3. Episodic vs. sequential In an episodic environment the agent’s experience can be divided into atomic steps where the agents perceives and then performs a single action. The next episode does not depend on the actions taken in previous episodes. ex: Imagine, an agent that has to spot defective parts on an assembly line. It’s current decision doesn’t affect whether the next part is defective. In sequential environments,current decision could affect future decisions. Chess is sequential.

18. 4. Static vs. dynamic If the environment can change while the agent is choosing an action, the environment is dynamic, otherwise it is static.If the environment itself doesn’t change, but the agent’s performance score does, then it is semidynamic.  Taxi driving -> dynamic  Chess with a clock ->semidynamic  Crossword puzzles -> static

19. 5. Discrete vs Continuous Chess game is a discrete environment,because it has a finite number of distinct states.It has also discrete set of percepts and actions. Tavi driving is continuous ; the speed and the location of the taxi and the other vehicles changes through a range of continuous values.

20. 5. Single agent vs. multiagent If the environment contain other agents who are also maximizing some performance measure that depends on the current agent’s actions,then it is multiagent. (chess, taxi driving )

21. Examples of task environments and their characteristics

22. Agent Types All agents have the same skeleton  Input= current percepts  Output = actions  Program= manipulates input to produce output

23. Agent Types Four basic kind of agent programs;  Simple reflex agents  Model-based reflex agents  Goal-based agents  Utility-based agents All these can be turned into learning agents

24. Simple reflex agents  These agents select actions on the basis of the current percept,ignoring the rest of the percept history.  Eg. Vacuum agent is a simple reflex agent because its decisions based only on the current location amd whether that contains dirty  Implemented through condition-action rules  If dirty then suck

25. Simple reflex agents

26. Model-based reflex agents  Agents should maintain some internal state  E.g. internal state can be the previous frame of the camera.  Over time update state using world knowledge How does the world change? How do actions affect world?

27. Model-based reflex agents UPDATE-STATE is responsible for creating the new internal state description

28. Goal-based agents  Knowing about the current state of the environment is not always enough to decide what to do.  The agents needs a goal. E.g. being at passenger’s destination is the goal of an taxi driver.

29. Utility-based agents  Certain goals can be reached in different ways.  Some are better, have a higher utility.  E.g. There are many action sequences that will get the taxi to its destination but some are quicker,safer or cheaper than others.

30. Learning agents All previous agent-programs describe methods for selecting actions.  Learning mechanisms can be used to perform this task.

31. Learning agents  Learning element –responsible for making improvements  Performance element- Selecting actions based on percepts  Critic- Learning element uses feedback from the critic on how the agent is doing and determines how the performance element should be modified to do better.  Problem generator- Suggests actions that will lead to new and informative experiences.