1. Multi-Agent Based Search vs. Local Search and Backtrack Search for Solving Tight CSPs: A Practical Case Study Hui Zou and Berthe Y. Choueiry Constraint Systems Laboratory Department of Computer Science and Engineering University of Nebraska-Lincoln {hzou|choueiry}@cse.unl.edu

2. Introduction Search algorithms : systematic or local-repair C omplex, real-world optimization problems –Systematic search thrashes –Local search gets stuck in ‘local optima’ –Remedial: random walk, breakout, restart strategies, etc. Multi-agent-based search [Liu & al. AIJ 02] –provides us with a new way –Advantages & shortcomings via a practical application

3. Graduate Teaching Assistants (GTA) problem: In a semester, given –a set of courses –a set of graduate teaching assistants –a set of constraints that specify allowable assignments Find a consistent and satisfactory assignment of GTAs to courses Background - GTA Detailed modeling in [Glaubius & Choueiry ECAI 02 WS on Modeling] Types of constraints : unary, binary, non-binary – Each course has a load, indicates weight of the course – Each GTA has a (hiring) capacity, limits max. load

4. Background - GTA (cont’) Problem size : Date setMark # variablesDomain size Problem size Spring2001b B69353.5×10 106 O69264.3×10 97 Fall2001b B65352.3×10 100 O65343.5×10 99 Fall2002 B31331.2×10 47 O31287.7×10 44 Spring2003 B54361.1×10 84 O54345.0×10 82 B – boosted to make problem solvable O – original, not necessary solvable In practice, this problem is tight, even over-constrained Our goal: ensure GTA support to as many courses as possible

5. Background - GTA (cont’) Optimization criteria : 1.Maximize the number of courses covered 2.Maximize the geometric average of the assignments wrt the GTAs’ preference values (between 0 and 5). Problem : –Constraints are hard, must be met –Maximal consistent partial-assignment problem (MPA-CSP?) –Not a MAX-CSP (which maximizes #constraints satisfied)

6. Background - MAS for CSPs Multi-Agent System: agents interact & cooperate in order to achieve a set of goals – Agents: autonomous (perceive & act), goal-directed, can communicate – Interaction protocols: governing communications among agents – Environment: where agents live & act ERA [Liu & al. AIJ 2002] – Environment, Reactive rules, and Agents – A multi-agent approach to solving a general CSP – Transitions between states when agents move

7. Background - ERA’s components Environment : a n×m two-dimensional array –n: the number of variables (agents) –m: the maximum domain size, |D max | –e(i, j).value: domain value of agent i at position j –e(i, j).violation: violation value of agent i at position j – Zero position : where e(i, j).violation=0 When all agents are in zero position, we have a complete solution ERA=Environment + Reactive rules + Agents Example:

8. Background - ERA’s components Reactive rules : –Least-move: choose a position with the min. violation value –Better-move: choose a position with a smaller violation value –Random-move: randomly choose a position Combinations of these basic rules form different behaviors. ERA=Environment + Reactive rules + Agents

9. Background - ERA’s components Agents : a variable is represented by an agent ERA=Environment + Reactive rules + Agents At each state, an agent chooses a position to move to, following the reactive rules. The agents keep moving until all have reached zero position, or a certain time period has elapsed. All agents in zero position Some agents in zero position Assignments are made only for agents in zero position

10. Background - ERA vs local search ERA operates by local repairs, how different is it from local search? ERA –Each agent has an evaluation function –At each state, any agent moves wherever it desires to move Control is localized: Each agent is in pursuit of its own happiness Local search with min-conflict –One evaluation function for the whole state (cost), summarizes the quality of the state –At each state, few agents are allowed to move (most unhappy ones) Control is centralized: towards one common good

11. Background - Example ( ERA ) 4-queen problem Init 2 022 Eval (agent Q1) 123 2 Eval (agent Q2) 21 21 Eval (agent Q3) 1 Move (agent Q3) 0 Move(agent4)

12. ERA – any agent can kick any other agent from its position Local search with min-conflict – cannot repair a variable without violating a previously repaired variable Background - Example (ERA vs. Local search)

13. Empirical study - In general Apply ERA on GTA assignment problem : 0. (Test & understand the behavior of ERA) 1.Compare performance of: –ERA: FrBLR –LS: hill-climbing, min-conflict & random walk –BT: B&B-like, many orderings (heuristic, random) 2.Observe behavior of ERA on solvable vs. unsolvable problems 3.Observe behavior of individual agents in ERA 4.Identify a limitation of ERA: deadlock phenomenon 8 instances of the GTA assignment problem

14. Empirical study 1- Performance comparison Date setSystematic Search (BT)Local Search (LS)Multi-agent Search (ERA) Spring2001b B√35693529.61.1864.0526.53.7753.6906.40.8703.2005.30.18 O×26692629.60.88163.7902.54.09133.5400.90.39242.5588.37.39 Fall2001b B√35653129.31.0623.1202.51.7143.0103.80.3303.1811.92.68 O√34653029.31.0223.1201.52.4643.0413.70.1003.2700.81.15 Fall2002 B√333116.5131.2713.9303.52.3923.4005.00.8503.6223.00.02 O×283111.5130.8843.5801.82.5643.6102.00.1683.2212.00.51 Spring2003 B√365429.527.41.0834.4924.21.1733.6203.90.3203.0312.80.49 O√345427.527.41.0034.4502.21.5343.6303.31.4203.2600.80.14 Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio=Unassigned Courses Solution Quality Unused GTAs CC ( ×10 8 ) Unassigned CoursesSolution QualityUnused GTAsAvailable Resource CC ( ×10 8 ) Original/BoostedSolvable?# GTAs# CoursesTotal CapacityTotal LoadRatio= Original/BoostedSolvable?# GTAs# CoursesTotal capacity ( C )Total load ( L )Ratio= C \ L Unassigned CoursesSolution Quality Unused GTAsAvailable Resource CC (×10 8 )Unassigned CoursesSolution Quality Unused GTAsAvailable Resource CC (×10 8 )Unassigned CoursesSolution Quality Unused GTAsAvailable Resource CC (×10 8 ) Observations : - Only ERA finds complete solutions to all solvable instances - On unsolvable problems, ERA leaves too many unused GTAs - LS and BT exhibit similar behaviors

15. Empirical study 2- Solvable vs unsolvable ERA performance on solvable problems ERA performance on unsolvable problems Observation : - Number of agents in zero- position per iteration - ERA behavior differs on solvable vs. unsolvable instances

16. Empirical study 3- Behavior of individual agents Instances solvable unsolvable Motion of agents variable stable constant Observations: SolvableUnsolvable Variable NoneMost Stable A few Constant MostNone

17. Empirical study 4- Deadlock – Each circle corresponds to a given GTA – Each square represents an agent – A blank squares indicate that an agent is on a zero-position – The squares with same color indicate agents involved in a deadlock Observation: ERA is not able to avoid deadlocks and yields a degradation of the solution on unsolvable CSPs.

18. Discussion GoalActions Control SchemaUndoing assignmentsConflict resolution ERA Local + Escape local optima – May yield instability √ + Flexible + Solves tight CSPs Non-committal – Deadlock – Shorter solutions LS Global + Stable behavior – Liable to local optima × + Quickly stabilizes – Fails to solve tight CSPs even with randomness & restart strategies Heuristic + Longer solutions – Problem-dependent BT Systematic + Stable behavior – Thrashes ~ + Quickly stabilizes – Fails to solve tight CSPS even with backtracking & restart strategies + advantages – shortcomings

19. Dealing with the deadlock Possible approaches: — Direct communications, negotiation mechanisms — Hybrids of search Global control Conflict resolution Experiments: — Enhancing ERA with global control – Don’t accept a move that deteriorates the global goal – Lead to local-search-like behavior (i.e., local optima) — ERA with conflict resolution – add dummy resources – find a complete solution when LS and BT fail – remove dummy assignments, solutions are still better

20. Future research directions – Test approach using other search techniques – BT search: Randomized, credit-based – Other local repair: squeaky-wheel method – Market-based techniques, etc. – Validate conclusions on other CSPs – random instances, real-world problems – Try search-hybridization techniques References: R. Glaubius and B.Y. Choueiry, Constraint Modeling and Reformulation in the Context of Academic Task Assignment. In Workshop Modeling and Solving Problems with Constraints, ECAI 2002. J. Liu, H. Jing, and Y.Y. Tang. Multi-Agent Oriented Constraint Satisfaction. Artificial Intelligence, 136:101-144, 2002.

21. Questions