1. Workshop on High Performance, Fault- Adaptive Large Scale Real-Time Systems Vanderbilt University The SRTA Agent Architecture as a Basis for Building Soft Real-Time Multi-Agent Systems Victor R. Lesser Computer Science Department University of Massachusetts, Amherst November 15, 2002

2. Acknowledgements Byran Horling Dr. Regis Vincent (SRI) Dr. Tom Wagner (Honeywell Research) URL: http://mas.cs.umass.edu/~bhorl ing/papers/02-14.ps.gz

3. Outline Background/Motivation EW Challenge Problem Approach to Soft Real-Time Approach to Building MAS SRTA Agent Architecture Experimental Evaluation Summary

4. Long Term Motivation Development of domain-independent techniques (toolkits) for coordinating the soft real-time activities of teams of cooperative agents? Ease the construction of complex, multi-agent applications that operate in a coherent manner Avoid reproducing for each application the complex reasoning involved in soft real-time control and in coordinating the activities of agents

5. DARPA EW Challenge Problem: Distributed Sensor Network Small 2D Doppler radar units –Scan one of three 120  sectors at a time Commodity Processor associated with each radar Communicate short messages using radio Triangulate radars to do tracking

6. Approach to Soft Real-Time: Design-to-Time “Given a time bound, dynamically construct and execute a problem-solving procedure which will (probably) produce a reasonable answer with (approximately) the time available.” (D’Ambrosio) Involves elements of planning (deciding what to do) and scheduling (deciding when to perform particular actions).

7. TÆMS: A Domain Independent Framework for Modeling Agent Activities The top-level goals/objectives/abstract -tasks that an agent intends to achieve One or more of the possible ways that they could be achieved abstraction hierarchy (HTN) whose leaves are basic action instantiations, called methods A precise, quantitative definition of the performance (Qaf’s) solution quality, cost, time and resource usage.

8. Soft Real Time Control -- Different Paths for Achieving Task -- “BUILD PRODUCT OBJECTS” Schedule A - Client has no resource limitations; maximize quality Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process- ZDnet, Query-and-Process-Consumers-Report (Expected Q=55.3,C=2, D=11.5) Schedule B - Client is interested in a free solution Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process- Zdnet (Expected Q=33.2,C=0, D=8.4) Schedule C - Client request an even trade-off between quality, cost and duration Query-and-Extract-PC-Connection, Search-and-Process-Zdnet (Expected Q=22.4,C=0, D=5.6) Schedule D - Client wishes to maximize quality while meeting a hard deadline of 7 minutes Query-and-Extract-PC-Mall, Query-and-Process-Consumers-Report (Expected Q=25.9,C=2, D=6) Examples of Schedules Produced by the Design-To-Criteria (DTC) Scheduler

9. Representing Coordination Patterns Among Agents

10. Approach to Decomposing A Problem into Agents Sophisticated/Highly Competent Agents Concurrent goals, goals are time and resource sensitive, goals have varying utilities Goals have alternative ways of being solved that produce differing levels of utility and consume differing amounts of resources Not all goals necessarily need to be solved (Sub)Goals spread across agents are interdependent Contention for scarce resources Contributing towards the solution of a higher-level goal Hard and soft constraints Sufficient computational/communication resources to do “some” reasoning about coordination Medium granularity domain tasks

11. What about simpler agents? Activities of simple,single-threaded agents become the goals of sophisticated agents with dedicated processing resources Sophisticated agents do the selection, multiplexing, scheduling, coordination and distribution of goals Contrast with O.S. doing the scheduling without context Sector Manager Tracking Manager Tracking Agent Scanning Agent

12. Approach to Soft, Real-Time Distributed Coordination/Resource Allocation Structured as a distributed optimization problem with a range of “satisficing” solutions Adaptable to available time and communication bandwidth Responsive to dynamics of environment Organizationally constrained — range of agents and issues are limited Can be done at different levels of abstraction Does not require all issues to be resolved to be successful — resource manager agents able to resolve some issues locally

13. Layered Agent Architecture Domain analysis and goal formulation Organization-level resource allocation Agent-level resource allocation Constraint discovery and satisfaction Intra-agent organization and communication Environmental access points Problem Solver / Negotiation Soft Real Time Architecture Java Agent Framework

14. JAF: Java Agent Architecture Component-based agent design Attempt to maximize code reuse. ExecuteControlCommunicatePulse ActionsScan Scheduler Problem Solver ExecuteControlCommunicate Resource ModelerDirectory ServiceLogSchedulerStateSensorObserve Interfaces are hidden by JAF.Interfaces are hidden by JAF. Radsim/ RF communication/ sensorRadsim/ RF communication/ sensor

15. SRTA: Soft Real-Time Agent Architecture Facilitates creation of multi-resource management agents Basis of building complex “virtual” agent organizations Allows for abstract negotiation — maps abstract assignment into detailed resource allocations Ability to resolve conflicts locally that are not resolved through negotiation These are key to building soft real-time distributed allocation policies

16. Soft Real-Time Control Architecture Resource Modeler Conflict Resolution Module Task Merging Problem solver Periodic Task Controller TÆMS Library Cache Check DTC-Planner Partial Order Scheduler Parallel Execution Module Learning Update Cache Hit Linear Plan TAEMS-Plan Network/Objective Goal Description/Objective Parallel Schedule Schedule Failure Results Update Expectations Schedule Failure Other Agents Schedule Resource Uses Multiple Structures Negotiation (e.g. SPAM) Commitments/ Decommitments Schedule failure/ Abstract view

17. Operates at 50 to 100ms cycle time Written in JAVA except for Planner in C++ Uses domain-independent, quantitative representation of agent activities -- TÆMS Scheduling of multiple activities that have deadlines and are resource sensitive Can choose among alternative ways of achieving activities that trade off decreased utility for lower resource consumption Responds to uncertain conditions without the need for complete re-planning/scheduling of activities Characteristics of Soft Real-Time Control Architecture

18. Addressing Real Time – Direct Direct technologies - making it possible DTC (Design-To-Criteria) planner TÆMS HTN for representing alternative plan options using quantitative information Create appropriate plan given time, resource costs and quality constraints Partial order scheduling creates “loose” schedules which can be quickly shifted to real-time constraints. Avoids constant re-planning Allows parallel execution and resource usage. Modeling of some meta-level activities (e.g. negotiation) permit more direct reasoning of time allocation. We do not model scheduling costs yet. Learning component discerns actual execution characteristics so future actions can be better modeled.

19. Addressing Real Time – Indirect Indirect technologies - making it easier Periodic commitments reduce the need for re-negotiation. Scheduled caching reduces the need to call DTC Piecemeal addition and removal of tasks eliminates the need for constant dramatic rescheduling and re-planning.

20. Scheduling Partial-order Scheduler uses a “sliding” mechanism, coupled with a resource modeler, to quickly shift scheduled tasks. Action start time uncertainty - real time. Duration uncertainty. Commitments have a “window” of time in which the agent can perform them. Precise action scheduling is left to the discretion of the performing agent.

21. Set-parameters First task to achieve enables2 enables3 lock Track Low Track Medium Track High Send Results Sensor Task 1 Q_min Task Q_max Deadline: 3000 RF

22. 2 Other tasks to achieve Enables1 InitCalibrateSend-Message 1 lock1 Q_min Enables4 Task2 Q_min Task3 Negotiate-TrackingSend-tracking-Info Sensor Enables4 RF

23. Reacting to Unexpected Changes 500 1000150020002500300035004000 Init Calibrate Send Msg Set-ParametersTrack-MediumSend-Results Send-Info-TrackingNegotiate-Tracking 500 1000150020002500300035004000 Init Calibrate Send Msg Set-ParametersTrack-MediumSend-Results Send-Info-TrackingNegotiate-Tracking 500 1000150020002500300035004000 Init Calibrate Send Msg Set-Parameters Negotiate-Tracking time Send-Info-Tracking Send-Results Analogous reactions also take place within the periodic task controller Slot-based scheduler used to facilitate repetitive actions Track-Medium

24. Meta-Level Costing Typical scheduling reasons about primitive actions. This only accounts for some percentage of the agent’s time. So called meta-level activities (e.g. negotiation, scheduling, planning) use significant resources but are usually not accounted for directly. Without accountability, these activities can interfere with the actions and commitments currently scheduled over.

25. Meta-Level Costing (cont’d) To completely reason about all the agent’s actions, we must: Directly and Indirectly incorporate the activities in plans. Derive expected costs for these activities. Use this information when generating schedules. We are currently using a representation of negotiation activities in some our task structures Future goal is to more directly account for activities like planning and scheduling.

26. Reducing Scheduling Overhead Activity parallelism learning Anticipation of converting linear plan to parallel plan if resources are available Schedule caching Adjustable time granularity Responsiveness vs. meta-level overhead

27. Activity Parallelism Learning The plan is first scheduled… The schedule is analyzed for parallel actions These are used to form sets of “hints,” associated with the current resource context A hint is actually just a mutual facilitates relationship These hints are later applied to the task structure before planning The facilitates tells DTC that if A is run first, B will have a duration of zero (or vice versa) This technique required no changes to DTC This can result in better plan selection.

28. Schedule Caching Agents in repetitive environments must frequently address goals which have previously been seen. Ex: In sensor environment, “Scan-Sector” or “Perform-Track- Measurement” Results from prior planning can be reused A key is generated for each task structure Incorporates method names and expectations, interrelationships, normalized deadlines, etc. Works correctly with activity parallelism hints If results exist from a task structure with the same key, that plan is used instead of calling DTC DTC is an external C ++ binary, requiring file reads and writes, so savings are significant

29. Adjustable Time Granularity The millisecond time line is divided into coarser increments. Does not significantly degrade the agent’s ability to meet “wall clock” time deadlines (assuming success within a reasonable grain size) It does decrease the number of rescheduling events which are needed Since the agent was already operating effectively at a coarse timeline, this technique has no new drawbacks. 1234567891011121314151612345678910111213141516 12345678910111213141516 12345 Real Time Perceived Time Real Time Perceived Time Deadlines Actions Deadlines Actions   

30. Effects of Schedule Caching Periodic tasks and methods with deadlines are more achievable As a result, track updates increase

31. Effects of Changing Granularity Rescheduling attempts decrease, reducing overhead Methods with deadlines and periodic tasks are satisfied more often 30-40 seems ideal; above that, the coarse granularity decreases agent responsiveness

32. Summary SRTA architecture is a powerful tool for building soft real time agent organizations Sophisticated soft real time agent control is practical by exploiting a variety of mechanisms to speed up the planning, scheduling and rescheduling cycles

33. Future SRTA Work Continue to speed up architecture Simple Planner for time-critical situations Additional work on conflict-resolution strategies Meta-level control component to balance control and coordination costs and domain problem solving