SRTA: The Soft-Real Time Agent Control Architecture Bryan Horling, Victor Lesser, Regis Vincent, Thomas Wagner presented by Anita Raja

Agent Control Most multi-agent research addresses inter- agent activities Most multi-agent research addresses inter- agent activities The intra-agent mechanics are just as important… The intra-agent mechanics are just as important… Control affects the potential level of flexibility and sophistication for entire agent Control affects the potential level of flexibility and sophistication for entire agent Generality, efficiency, reliability… Generality, efficiency, reliability… Solid general control architecture provides foundation for further research Solid general control architecture provides foundation for further research

Motivation Several existing research artifacts… Several existing research artifacts… Task modeling Task modeling Planning and scheduling Planning and scheduling Coordination Coordination Previous work done in simulation Previous work done in simulation New more demanding domain (ANTs) New more demanding domain (ANTs) Real-time Real-time Uncertain Uncertain Resource-bound Resource-bound More realistic conditions More realistic conditions Desire to merge these technologies into a cohesive, functional, reusable entity Desire to merge these technologies into a cohesive, functional, reusable entity

Soft Real-Time Architecture Plan and schedule to solve goals Plan and schedule to solve goals Resource constraints Resource constraints Task interaction constraints Task interaction constraints Deadlines and earliest start times Deadlines and earliest start times Merge new goals with existing ones Merge new goals with existing ones Adjust interleaved schedules as necessary Adjust interleaved schedules as necessary Handle unexpected deviations in execution Handle unexpected deviations in execution Address time-related failures Address time-related failures Resolve conflicts from failed actions Resolve conflicts from failed actions

Soft Real-Time Hard real-time: formally bound and quantitatively describe performance Hard real-time: formally bound and quantitatively describe performance Soft real-time is a looser metric Soft real-time is a looser metric Tasks may still have value if time bounds are exceeded by small amount Tasks may still have value if time bounds are exceeded by small amount Our interest is to be statistically “fast enough” Our interest is to be statistically “fast enough” Can target more uncertain domains Can target more uncertain domains Better handle unexpected events Better handle unexpected events For motivating domain, tasks should be performed within ±500ms of scheduled time For motivating domain, tasks should be performed within ±500ms of scheduled time

SRTA Context Operates at the middle agent layer Operates at the middle agent layer API formed of two parts… API formed of two parts… Function accessors Function accessors TÆMS modeling language TÆMS modeling language Comprised of several components Comprised of several components Co-exists in JAF framework with other components Co-exists in JAF framework with other components Domain Problem Solver Soft Real Time Architecture JAF Controller

TÆMS Task Structures TÆMS is a goal decomposition planning language TÆMS is a goal decomposition planning language Tasks represent goals or sub-goals Tasks represent goals or sub-goals Methods are primitive actions that can be performed Methods are primitive actions that can be performed QAFs dictate how tasks accrue quality QAFs dictate how tasks accrue quality Interrelationships specify interactions between nodes Interrelationships specify interactions between nodes

Java Agent Framework Component-based design, similar to JavaBeans Component-based design, similar to JavaBeans Individual components are well-encapsulated and potentially ‘autonomous’ Individual components are well-encapsulated and potentially ‘autonomous’ Components organized much like a miniature multi-agent system Components organized much like a miniature multi-agent system Intra-agent interactions in the form of Intra-agent interactions in the form of Direct method invocation Direct method invocation Indirect common data handling Indirect common data handling Event delivery and receipt Event delivery and receipt

Soft Real-Time Architecture Resource Modeler Conflict Resolution Module Task Merging Problem solver TÆMS Library DTC-Planner Partial Order Scheduler Parallel Execution Module Learning Linear Plan TÆMS Structure Goal Description Parallel Schedule Schedule Failure Results Update Expectations Other Agents Schedule Failure Resource Uses Multiple Structures Negotiation SRTA Reasoning

Goal Instantiation Goals are represented using TÆMS Goals are represented using TÆMS May be dynamically created, or read from static files May be dynamically created, or read from static files pTÆMS allows for parameterized, template-like structure definitions pTÆMS allows for parameterized, template-like structure definitions (spec_method (label Track-Medium) (label Track-Medium) (agent Agent_A) (agent Agent_A) (supertasks Track) (supertasks Track) (earliest_start_time 500) (earliest_start_time 500) (deadline 2000) (deadline 2000) (outcomes (outcomes (Outcome (Outcome (density 1.0) (density 1.0) (quality_distribution ) (quality_distribution ) (duration_distribution ) (duration_distribution ) (cost_distribution ) (cost_distribution ) ) )) (spec_commitment (label commitment-1) (label commitment-1) (type deadline) (type deadline) (from_agent Agent_A) (from_agent Agent_A) (to_agent Agent_B) (to_agent Agent_B) (task Track) (task Track) (earliest_start_time 500) (earliest_start_time 500) (deadline 2000) (deadline 2000))

Planning (developed by Tom Wagner) Goal planning by Design-to-Criteria scheduler Goal planning by Design-to-Criteria scheduler Select the most appropriate set of end-to-end actions from a structure Select the most appropriate set of end-to-end actions from a structure Considers action and plan duration, quality, cost, interrelationships, constraints Considers action and plan duration, quality, cost, interrelationships, constraints Reasons about mandatory and optional requirements, with respect to desired plan criteria Reasons about mandatory and optional requirements, with respect to desired plan criteria Differentiated by reasoning over ‘soft’ conditions Differentiated by reasoning over ‘soft’ conditions SliderCriteria/ImportanceModel

Scheduling DTC was designed as a single-structure scheduler DTC was designed as a single-structure scheduler Multiple goal structures must be merged, or assumed independent Multiple goal structures must be merged, or assumed independent Merged structures are larger, slower to schedule Merged structures are larger, slower to schedule Goal independence is an impractical condition Goal independence is an impractical condition A more flexible approach is needed A more flexible approach is needed

Partially Ordered Scheduling (developed by Regis Vincent) Partial ordered scheduler analyses DTC plans Partial ordered scheduler analyses DTC plans Determines task-based precedence constraints Determines task-based precedence constraints Resource modeler detects resource constraints Resource modeler detects resource constraints Builds a precedence graph, used for scheduling and rescheduling Builds a precedence graph, used for scheduling and rescheduling Key: Leverage DTC’s existing expertise Key: Leverage DTC’s existing expertise

Resource Modeler Creates and maintains timeline of expected uses of resources Creates and maintains timeline of expected uses of resources Distribution based: probabilistic start time, duration and quantity consumed or produced Distribution based: probabilistic start time, duration and quantity consumed or produced Used by scheduler to find and bind appropriate times for methods Used by scheduler to find and bind appropriate times for methods Used by execution component to monitor resource level expectations Used by execution component to monitor resource level expectations

Schedule Merging STRA natively supports multiple concurrent, interdependent goals STRA natively supports multiple concurrent, interdependent goals PO Scheduler considers prior precedence graphs when scheduling new tasks PO Scheduler considers prior precedence graphs when scheduling new tasks Conflicts avoided by “shifting” methods based on graph information Conflicts avoided by “shifting” methods based on graph information Avoids monolithic rescheduling Avoids monolithic rescheduling …but retains the flexibility to modify prior scheduling results as needed …but retains the flexibility to modify prior scheduling results as needed

Execution Method execution is assumed to be in parallel Method execution is assumed to be in parallel Constraints (resource, interrelationships, etc.) are validated before method is started Constraints (resource, interrelationships, etc.) are validated before method is started Failed constraints require rescheduling Failed constraints require rescheduling PO Scheduler precedence graphs are again used for quick shifting where possible PO Scheduler precedence graphs are again used for quick shifting where possible Results are reported to other components and checked for failures Results are reported to other components and checked for failures

Future Work Provide an end-to-end model of performance bounds Provide an end-to-end model of performance bounds Add anytime character to techniques Add anytime character to techniques Meta-level reasoning system to control level of effort and resource expenditure Meta-level reasoning system to control level of effort and resource expenditure