1. Introduction to CSE 591: Autonomous agents - theory and practice. Chitta Baral Professor Department of Computer Sc. & Engg. Arizona State University

2. About the course It is an advanced graduate level course. Prerequisites –None formally, but be concerned (and discuss with me) If not taken an AI class before. If afraid of writing proofs. If you don’t know what NP-completeness is. The course will be rigorous.

3. Grading Scale: A+, A, B+, B, C+, C, F. Grade weights –Home work + small assignments 20-30% –2 Exams 50% –Paper 20-30% One exam in mid-October and another on the last day of class (no exams on the day of finals)

4. A formal approach to design and analysis of autonomous agents Lots of hype about agents. Used in different contexts, no uniform framework. Multiple answers to the question: What is an agent? No systematic approach to design or analyze an agent exists.

5. Central thesis Give a formal approach to design and analysis of ‘autonomous’ agents. –Similar to the design of relational databases. Use this approach in different applications. –For example: network agents, embedded agents, diagnostic agents, remote (space) agents, data-consistency agents (active databases), viewing living cells as agents.

6. What are ‘autonomous’ agents? Agents –One that acts or has the power or authority to act. –One empowered to act for or represent another. –A means by which something is done or caused. –A force or substance that causes a change. –…

7. What are autonomous agents? Cont. Autonomous –Not controlled by others or by outside forces –Independent in mind or judgment or government; self directed. –Self governing

8. Our view An agent is an entity (software module, piece of embedded hardware) that acts and causes change. Autonomy: –Not micro-managed –Can take high-level directives and goals and can figure out what actions it needs to take and execute those actions.

9. What ability an autonomous agent must possess? Must understand high level directives Must have knowledge about its abilities –What actions it can perform –How these actions affect the world Must be able to reason and chalk out its plan of action (and revise them if things change)

10. Designing agents: the steps Representing actions and environment. – Objects in the environment. –Relationship between the objects. –Actions that the agent can perform. –Actions that are beyond the control of the agent. (exogenous actions) –How does the actions affect the environment. –*Main issue* -- an appropriate language (syntax and semantics) for representing the above.

11. Designing agents (cont.) What is the goal of the agent? –Achievement goals. –Maintenance goals. –In general, to make the trajectories taken by the agent satisfy some temporal (LTL, CTL*) formula. –Research issues: Are existing temporal languages enough? Do we need new constructs? –Knowledge temporal goals! –Issues: Complexity of planning with such goals.

12. Designing agents (cont) Agent architecture and execution languages –Pure reactive agent: a collection of rules of the form: ‘if observed x and context is z then do action y’. The agent continuously senses the environment and executes the rule whose condition matches the sensor values. Research issue: * Verifying that a collection of such rules when executed will satisfy the goal of the agent. * Automatic generation of such rules.

13. Designing agents (cont) Agent architecture and execution languages (cont) –Pure deliberative agent: Follows a cycle of Sense Assimilate observation (reasoning is needed here) Replan if necessary (reasoning and planning are needed here). Replanning becomes necessary if environmental actions make changes that invalidates the original plan. Execute part of the plan. (assimilation and planning are both hard; NP-complete even in simple cases.)

14. Designing agents (cont) Agent architecture and execution languages –Hybrid agents Reactive for some (important/frequent/critical) states and deliberative for others. The states for which reactions are stored are updated similar to the cache/main-memory hierarchy. Various levels of reactivity: simplest ones spell out the action; next: use a fast (logic) program to decide on the next action; use domain knowledge to do faster planning so as to come up with the action to be executed.

15. Designing agents -- summary 1.Actions, environment, and effect of actions. 2.Goal or directives. 3.Execution or architecture. 4.Design and analysis tasks: 1.Write 1 and 2 and then, 2.Given 1 and 2 come up with 3; or 3.Given 1, 2, and 3, verify 3 is correct with respect to 2; or 4.Given 1, 2, and 3, and some change in 2, update 3 to satisfy the new directives.

16. A data consistency maintenance agent. (An active database) Actions and their effect on the environment * Fluents: (technical term for objects in the world) purchase(purchaseid, client, amount). payment(paymentid, client, amount). account(client, credit, status). * Environmental actions: insertion of tuples to the purchase and payment relations for existing clients. *Agent's actions: Any other insert, delete and update.

17. Active database agent (cont.) Actions and their effects on the environment. (cont.) –insert(f) causes f – executable insert(f) if ~ f – delete(f) causes ~f – executable delete(f) if f – update(f,f') causes f', ~ f – executable update(f,f') if f

18. Active Dtabase agent (cont.) Goal: Make sure the stable state of the database satisfies the following. –For each client c which appears in a tuple a in the relation account: if a.credit = 3K then a.status = good. –For each client c which appears in a tuple a in the relation account: a.credit is 5K minus the sum of all the purchase amounts for c plus the sum of all the payment amounts for c.

19. Active Database agent (cont.) Reactive execution program (triggers) –Trigger 1: When a tuple p is added to the purchase relation, then update the tuple a in the relation account such that p.client = a.client so that the updated a.credit has the value obtained by subtracting p.amount from the old a.credit. –Trigger 2: When a tuple a in the relation account is updated such that a.credit is less than 3K then update a such that a.status has the value ``bad''. –Trigger 3: When a tuple p' is added to the payment relation, then update the tuple a in the relation account such that p'.client = a.client so that the updated a.credit has the value obtained by adding p'.amount to the old a.credit.

20. Active Database agent (cont.) Reactive execution program (cont.) –Trigger 4: When a tuple a in the relation account is updated such that a.credit is more than or equal to 3K then update a such that a.status has the value ``good''. Correctness: Suppose the database is in a stable state (I.e., it satisfies the goal conditions) and an allowable exogenous action occurs, then if the triggers are executed they will take the database to a stable state.

21. Paper ideas New language constructs for expressing the ability of agents and their impact on the environment New language constructs to express goals Agent architecture components Designing and implementing an agent in a particular domain