2 Chapter 2 Intelligent Agents What is an agent ?An agent is anything that perceiving its environment through sensors and acting upon that environment through actuatorsExample:Human is an agentA robot is also an agent with cameras and motorsA thermostat detecting room temperature.
5 Simple Terms Percept Percept sequence Agent’s perceptual inputs at any given instantPercept sequenceComplete history of everything that the agent has ever perceived.
6 Agent function & program Agent’s behavior is mathematically described byAgent functionA function mapping any given percept sequence to an actionPractically it is described byAn agent programThe real implementation
7 Vacuum-cleaner world Perception: Clean or Dirty? where it is in? Actions: Move left, Move right, suck, do nothing
9 Program implements the agent function tabulated in Fig. 2.3 Function Reflex-Vacuum-Agent([location,statuse]) return an actionIf status = Dirty then return Suckelse if location = A then return Rightelse if location = B then return left
10 Concept of Rationality Rational agentOne that does the right thing= every entry in the table for the agent function is correct (rational).What is correct?The actions that cause the agent to be most successfulSo we need ways to measure success.
11 Performance measure Performance measure An objective function that determinesHow the agent does successfullyE.g., 90% or 30% ?An agent, based on its percepts action sequence :if desirable, it is said to be performing well.No universal performance measure for all agents
12 Performance measure A general rule: E.g., in vacuum-cleaner world Design performance measures according toWhat one actually wants in the environmentRather than how one thinks the agent should behaveE.g., in vacuum-cleaner worldWe want the floor clean, no matter how the agent behaveWe don’t restrict how the agent behaves
13 Rationality What is rational at any given time depends on four things: The performance measure defining the criterion of successThe agent’s prior knowledge of the environmentThe actions that the agent can performThe agents’s percept sequence up to now
14 Rational agent For each possible percept sequence, E.g., an exam an rational agent should selectan action expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent hasE.g., an examMaximize marks, based onthe questions on the paper & your knowledge
15 Example of a rational agent Performance measureAwards one point for each clean squareat each time step, over time stepsPrior knowledge about the environmentThe geography of the environmentOnly two squaresThe effect of the actions
16 Example of a rational agent Actions that can performLeft, Right, Suck and NoOpPercept sequencesWhere is the agent?Whether the location contains dirt?Under this circumstance, the agent is rational.
17 Omniscience An omniscient agent An example Knows the actual outcome of its actions in advanceNo other possible outcomesHowever, impossible in real worldAn examplecrossing a street but died of the fallen cargo door from 33,000ft irrational?
18 Omniscience Based on the circumstance, it is rational. As rationality maximizesExpected performancePerfection maximizesActual performanceHence rational agents are not omniscient.
19 Learning Does a rational agent depend on only current percept? No, the past percept sequence should also be usedThis is called learningAfter experiencing an episode, the agentshould adjust its behaviors to perform better for the same job next time.
20 AutonomyIf an agent just relies on the prior knowledge of its designer rather than its own percepts then the agent lacks autonomyA rational agent should be autonomous- it should learn what it can to compensate for partial or incorrect prior knowledge.E.g., a clockNo input (percepts)Run only but its own algorithm (prior knowledge)No learning, no experience, etc.
21 Software Agents Sometimes, the environment may not be the real world E.g., flight simulator, video games, InternetThey are all artificial but very complex environmentsThose agents working in these environments are calledSoftware agent (softbots)Because all parts of the agent are software
22 Task environments Task environments are the problems While the rational agents are the solutionsSpecifying the task environmentPEAS description as fully as possiblePerformanceEnvironmentActuatorsSensorsIn designing an agent, the first step must always be to specify the task environment as fully as possible.Use automated taxi driver as an example
23 Task environments Performance measure How can we judge the automated driver?Which factors are considered?getting to the correct destinationminimizing fuel consumptionminimizing the trip time and/or costminimizing the violations of traffic lawsmaximizing the safety and comfort, etc.
24 Task environments Environment A taxi must deal with a variety of roads Traffic lights, other vehicles, pedestrians, stray animals, road works, police cars, etc.Interact with the customer
25 Task environments Actuators (for outputs) Sensors (for inputs) Control over the accelerator, steering, gear shifting and brakingA display to communicate with the customersSensors (for inputs)Detect other vehicles, road situationsGPS (Global Positioning System) to know where the taxi isMany more devices are necessary
26 Task environmentsA sketch of automated taxi driver
27 Properties of task environments Fully observable vs. Partially observableIf an agent’s sensors give it access to the complete state of the environment at each point in time then the environment is effectively and fully observableif the sensors detect all aspectsThat are relevant to the choice of action
28 Partially observableAn environment might be Partially observable because of noisy and inaccurate sensors or because parts of the state are simply missing from the sensor data.Example:A local dirt sensor of the cleaner cannot tellWhether other squares are clean or not
29 Properties of task environments Deterministic vs. stochasticnext state of the environment Completely determined by the current state and the actions executed by the agent, then the environment is deterministic, otherwise, it is Stochastic.-Cleaner and taxi driver are:Stochastic because of some unobservable aspects noise or unknown
30 Properties of task environments Episodic vs. sequentialAn episode = agent’s single pair of perception & actionThe quality of the agent’s action does not depend on other episodesEvery episode is independent of each otherEpisodic environment is simplerThe agent does not need to think aheadSequentialCurrent action may affect all future decisions-Ex. Taxi driving and chess.
31 Properties of task environments Static vs. dynamicA dynamic environment is always changing over timeE.g., the number of people in the streetWhile static environmentE.g., the destinationSemidynamicenvironment is not changed over timebut the agent’s performance score does
32 Properties of task environments Discrete vs. continuousIf there are a limited number of distinct states, clearly defined percepts and actions, the environment is discreteE.g., Chess gameContinuous: Taxi driving
33 Properties of task environments Single agent VS. multiagentPlaying a crossword puzzle – single agentChess playing – two agentsCompetitive multiagent environmentChess playingCooperative multiagent environmentAutomated taxi driverAvoiding collision
34 Properties of task environments Known vs. unknownThis distinction refers not to the environment itslef but to the agent’s (or designer’s) state of knowledge about the environment.-In known environment, the outcomes for all actions aregiven. ( example: solitaire card games).- If the environment is unknown, the agent will have to learn how it works in order to make good decisions.( example: new video game).
37 Structure of agents Agent = architecture + program Architecture = some sort of computing device (sensors + actuators)(Agent) Program = some function that implements the agent mapping = “?”Agent Program = Job of AI
38 Agent programs Input for Agent Program Input for Agent Function Only the current perceptInput for Agent FunctionThe entire percept sequenceThe agent must remember all of themImplement the agent program asA look up table (agent function)
39 Agent programsSkeleton design of an agent program
40 Agent Programs P = the set of possible percepts T= lifetime of the agentThe total number of percepts it receivesSize of the look up tableConsider playing chessP =10, T=150Will require a table of at least entries
41 Agent programs Despite of huge size, look up table does what we want. The key challenge of AIFind out how to write programs that, to the extent possible, produce rational behaviorFrom a small amount of codeRather than a large amount of table entriesE.g., a five-line program of Newton’s MethodV.s. huge tables of square roots, sine, cosine, …
42 Types of agent programs Four typesSimple reflex agentsModel-based reflex agentsGoal-based agentsUtility-based agents
43 Simple reflex agents It uses just condition-action rules The rules are like the form “if … then …”efficient but have narrow range of applicabilityBecause knowledge sometimes cannot be stated explicitlyWork onlyif the environment is fully observable
46 A Simple Reflex Agent in Nature percepts(size, motion)RULES:(1) If small moving object,then activate SNAP(2) If large moving object,then activate AVOID and inhibit SNAPELSE (not moving) then NOOPneeded forcompletenessAction: SNAP or AVOID or NOOP
47 Model-based Reflex Agents For the world that is partially observablethe agent has to keep track of an internal stateThat depends on the percept historyReflecting some of the unobserved aspectsE.g., driving a car and changing laneRequiring two types of knowledgeHow the world evolves independently of the agentHow the agent’s actions affect the world
48 Example Table Agent With Internal State IFTHENSaw an object ahead, and turned right, and it’s now clear aheadGo straightSaw an object on my right, turned right, and object ahead againHaltSee no objects aheadSee an object aheadTurn randomly
49 Example Reflex Agent With Internal State: Wall-Following startActions: left, right, straight, open-doorRules:If open(left) & open(right) and open(straight) thenchoose randomly between right and leftIf wall(left) and open(right) and open(straight) then straightIf wall(right) and open(left) and open(straight) then straightIf wall(right) and open(left) and wall(straight) then leftIf wall(left) and open(right) and wall(straight) then rightIf wall(left) and door(right) and wall(straight) then open-doorIf wall(right) and wall(left) and open(straight) then straight.(Default) Move randomly
50 Model-based Reflex Agents The agent is with memory
52 Goal-based agentsCurrent state of the environment is always not enoughThe goal is another issue to achieveJudgment of rationality / correctnessActions chosen goals, based onthe current statethe current percept
53 Goal-based agents Conclusion Goal-based agents are less efficient but more flexibleAgent Different goals different tasksSearch and planningtwo other sub-fields in AIto find out the action sequences to achieve its goal
55 Utility-based agents Goals alone are not enough to generate high-quality behaviorE.g. meals in Canteen, good or not ?Many action sequences the goalssome are better and some worseIf goal means success,then utility means the degree of success (how successful it is)
57 Utility-based agents it is said state A has higher utility If state A is more preferred than othersUtility is therefore a functionthat maps a state onto a real numberthe degree of success
58 Utility-based agents (3) Utility has several advantages:When there are conflicting goals,Only some of the goals but not all can be achievedutility describes the appropriate trade-offWhen there are several goalsNone of them are achieved certainlyutility provides a way for the decision-making
59 Learning Agents After an agent is programmed, can it work immediately? No, it still need teachingIn AI,Once an agent is doneWe teach it by giving it a set of examplesTest it by using another set of examplesWe then say the agent learnsA learning agent
60 Learning Agents Four conceptual components Learning element Making improvementPerformance elementSelecting external actionsCriticTells the Learning element how well the agent is doing with respect to fixed performance standard.(Feedback from user or examples, good or not?)Problem generatorSuggest actions that will lead to new and informative experiences.