Improving Market-Based Task Allocation with Optimal Seed Schedules IAS-11, Ottawa. September 1, 2010 G. Ayorkor Korsah 1 Balajee Kannan 1, Imran Fanaswala 2, Bernardine Dias 1,2 1 Robotics Institute, Carnegie Mellon University 2 CS Department, Carnegie Mellon Qatar

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 2 Task Allocation  Key component of planning for team coordination  Example: disaster preparedness and response

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 3 Tradeoff: Optimality vs. Adaptivity  Optimality guarantees  Slow to compute  not suitable for dynamic problems  No optimality guarantees  Fast to compute  suitable for dynamic problems Optimal & Centralized Approaches e.g. Mathematical Programming Heuristic & Decentralized Approaches e.g. Market-Based Approaches A task at (4, 2) I can do it for $73 It will cost me $80

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 4 Real-World Problems  Many real-world problems have both static and dynamic components  Some tasks known ahead of time, or some likely scenarios known ahead of time  New tasks arrive in real time and changed information discovered in real time

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 5 Proposed Approach  Optimally pre-allocate static tasks then adapt plan (heuristically) as needed to handle dynamic situations  Can pre-compute several initial plans for various likely scenarios

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 6 Approach Overview Mathematical Programming Approach Used to compute optimal solution to the static component of the problem Use a branch-and-price approach Market-Based Approach for Dynamism Used to modify the initial optimal seed schedule to handle dynamic component of the problem Use TraderBots Problem Decomposition Identify static and dynamic components of problem

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 7 Mathematical Model: Set-Partioning Integer Linear Program (ILP) Formulation Objective Function (e.g. Total Team Distance) One route per agent One agent per task Minimize: Subject to constraints: “Route” = candidate time extended plan/task allocation for an agent

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 8 Branch-and-Price Approach Summary  Based on Branch-and-Bound  Useful when variables cannot be exhaustively enumerated (in our case, route variables)  Allows progressive generation and inclusion of profitable variables (in our case, routes)  Enables computation of the optimal ILP solution

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 9 Market-Based Approach Summary  Tasks are assigned via auctions  Agents bid the marginal cost to perform the new task  Task is awarded to the lowest bidder  Centralized or decentralized  Tasks auctioned by central operator or by individual agents My bid: $280 My bid: $101 My bid: $73 Task at (3.5) Winner!

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 10 Proposed Seeded Market-Based Approach  Start out with the initial optimal plan  Use market-based approach to modify the optimal plan as changes occur  Hold auctions for new tasks as they arrive  Hold auctions for previously assigned tasks if needed (environmental changes/ execution failure) Task at (3.5) My bid: $280 My bid: $101 My bid: $73

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 11 Experiments  In simulation & on robots  Tasks:  Visit specified location  Objective function:  Minimize total distance travelled by team

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 12 Experiments  Compare:  Post-execution evaluation:  “Hindsight optimal” plan (Optimal branch-and-price for static & dynamic tasks)  “Pure” Market-Based Plan (Auctions for static & dynamic tasks)  Seeded Market-Based Plan (Branch-and-price for static & auctions for dynamic tasks) Team distance for (Seeded) Market-based plan Suboptimality factor = Team distance for “Hindsight Optimal” plan

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 13 Experimental Procedure Use branch-and-price to compute initial optimal plan for static tasks Begin execution of computed plans Continue execution, handling dynamism with market-based approach Compute “hindsight” optimal plan for static & dynamic tasks Compute “Sub-optimality factor” Task at (4, 2) $73 (Seeded) Market-based = “Hindsight Optimal” Complete Execution

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 14 Results: Simulation 2 agents, 12 tasks 2 agents, 16 tasks 5 agents, 20 tasks (averaged over 5 random instances for each problem configuration) Observation: With high % static tasks we see benefit of seeded market based approach

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 15 Median Planning Times for Branch-and- Price Planner (Simulation Experiments) Terminated (timed-out) prior to proving optimality of solution Observation: Combinatorial nature of the optimal planning problem

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 16 Results: Robots 2 robots, 11 tasks (6 static) (averaged over 5 runs for each approach) Observation: more significant improvement of seeded market-based approach over pure market-based approach than in simulation.

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 17 Conclusion  Contributions:  A seeded market-based approach for task allocation  Current & future directions:  Finer-grained characterization of seeded market- based approach  Handling inter-task order constraints (precedence, simultaneity, etc)

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 18 Acknowledgments  Sponsors: Qatar National Research Fund (QNRF) under contract NPRP  Collaborators:  Anthony Stentz  M. Freddie Dias  Ameer Abdulsalam  Wael Ghazzawi  Victor Marmol  Jaime Bourne

Thank you! Questions?

Extra Slides

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 21 Branch-and-Price A B E C D Start out with a subset of feasible routes

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 22 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 23 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 24 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 25 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 26 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes If constraints violated, branch AB & together

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 27 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes If constraints violated, branch AB & together

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 28 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes AB & together If constraints violated, branch

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 29 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes AB & together If constraints violated, branch

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 30 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes Prune nodes if possible If constraints violated, branch AB & together

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 31 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes AB & together AD & not together Prune nodes if possible If constraints violated, branch

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 32 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes AB & together Repeat till no more violated constraints and no more nodes to process AD & not together Prune nodes if possible If constraints violated, branch

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 33 Branch-and-Price A B E C D Start out with a subset of feasible routes Solve a relaxed version of the problem Generate additional profitable routes Repeat till no more profitable routes AB & together Repeat till no more violated constraints and no more nodes to process AD & not together Prune nodes if possible If constraints violated, branch Finds optimal solution!

Korsah, Kannan, Fanaswala, Dias. “Improving Market-Based Task Allocation…” 34 Branch-and-price summary Master Problem: Tries to assign known routes to agents by solving a mixed integer linear programming problem using branch-and-bound Sub problem: At each node, generates additional useful routes to consider by solving a constrained shortest-route problem based on dual variables of master problem (column generation)  Start out with a subset of known routes r 0, r 1, r 2, r 3, r 4, r 5 … Solve by searching a multi- dimensional space: DD* Lite Depth-1 st search