1. Weighted Voting Game Based Multi-robot Team Formation for Distributed Area Coverage Ke Cheng and Prithviraj (Raj) Dasgupta Computer Science Department University of Nebraska, Omaha

2. Research Objective: Multi-robot Coverage Use a set of robots to perform complete coverage of an initially unknown environment in an efficient manner Efficiency is measured in time and space –Time: reduce the time required to cover the environment –Space: avoid repeated coverage of regions that have already been covered Tradeoff in achieving both simultaneously

3. Major Challenges Distributed – no shared memory or map of the environment that the robots can use to know which portion of the environment is covered Each robot has limited storage and computation capabilities –Can’t store map of the entire environment Other challenges: Sensor and encoder noise, communication overhead, localizing robots

4. How does a robot do area coverage? Using an actuator (e.g., vacuum) or a sensor (e.g., camera or sonar) Source: Manuel Mazo Jr. and Karl Henrik Johansson, “Robust area coverage using hybrid control,”, TELEC'04, Santiago de Cuba, Cuba, 2004 Robot’s coverage tool The region of the environment that passes under the swathe of the robot’s coverage tool is considered as covered

5. E-puck Mini Robot IR sensors (8); range ~ 4 cm Camera; 640 X 480 VGA Bluetooth wireless communication LEDs Mic + speaker 7 cm 4.1 cm 144 KB RAM dsPIC processor@14MIPS Photo courtesy: Mobots group@EPFL http://mobots.epfl.ch

6. Multi-robot coverage: Individually coordinated robots using swarming Global Objective: Complete coverage of environment

7. Multi-robot coverage: Individually coordinated robots using swarming Global Objective: Complete coverage of environment Local coverage rule of robot... Local coverage rule of robot

8. Multi-robot coverage: Individually coordinated robots using swarming Global Objective: Complete coverage of environment Local coverage rule of robot... Local coverage rule of robot Local interactions between robots

9. Multi-robot coverage: Individually coordinated robots using swarming Global Objective: Complete coverage of environment Local coverage rule of robot... Local coverage rule of robot Local interactions between robots How well do the results of the local interactions translate to achieving the global objective? Done empirically References: 1. K. Cheng and P. Dasgupta, "Dynamic Area Coverage using Faulty Multi-agent Swarms" Proc. IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT 2007), Fremont, CA, 2007, pp. 17-24. 2. P. Dasgupta, K. Cheng, "Distributed Coverage of Unknown Environments using Multi-robot Swarms with Memory and Communication Constraints," UNO CS Technical Report (cst-2009-1).

10. Multi-robot coverage: Team-based robots using swarming Global Objective: Complete coverage of environment Local coverage rule of robot-team... Local coverage rule of robot-team Flocking technique to maintain team formation

11. Multi-robot coverage: Team-based robots using swarming Global Objective: Complete coverage of environment Local coverage rule of robot-team... Local coverage rule of robot-team Flocking technique to maintain team formation Local interactions between robot teams How well do the results of the local interactions translate to achieving the global objective? Done empirically Relevant publications: 1.K. Cheng, P. Dasgupta, Yi Wang ”Distributed Area Coverage Using Robot Flocks”, Nature and Biologically Inspired Computing (NaBIC’09), 2009. 2.P. Dasgupta, K. Cheng, and L. Fan, ”Flocking-based Distributed Terrain Coverage with Mobile Mini-robots,” Swarm Intelligence Symposium 2009.

12. Multi-robot teams for area coverage Theoretical analysis: Forming teams gives a significant speed-up in terms of coverage efficiency Simulation Results: The speed-up decreases from the theoretical case but still there is some speed-up as compared to not forming teams Based on Reynolds’ flocking model Leader referenced Follower robots designated specific positions within team

13. Coverage with Multi-robot Teams Square Corridor Office

14. Dynamic Reconfigurations of Robot Teams Having teams of robots is efficient for coverage Having large teams of robots doing frequent reformations is inefficient for coverage Can we make the modules change their configurations dynamically – Based on their recent performance: If a team of robots is doing frequent reformations (and getting bad coverage efficiency), split the team into smaller teams and see if coverage improves

15. Robot Team Formation for Coverage: Agent Utility-based Approach Each robot/agent tries to get into a configuration that maximizes its utility Utility-function of each robot in a team Flocking-based Controller Mediator A team needs to reconfigure Calculate the configuration that gives highest utility Check inconsistencies Large team…inefficient coverage: low individual utility Reference: P. Dasgupta and K. Cheng, “Coalition game-based distributed coverage of unknown environments using robot swarms, “ AAMAS 2008.

16. Coalition game-based team formation We used coalition games to solve the multi- robot team formation problem – Coalition games provide a theory to divide a set of players into smaller subsets or teams – We used a form of coalition games called weighted voting games (WVG)

17. Robot Team Formation for Coverage: Weighted Voting Game Coalition Game Layer Flocking-based Controller Mediator A team needs to reconfigure Calculate the best partition of a team Maintain consistency between coalition game result and team formations 17

18. Coalitional Games: Weighted Voting Game (WVG) Definitions N: set of players v: characteristic function, assigns a real-valued utility to each subset of players Each player i is assigned a weight w i – W max  w i q: quota, fixed positive real number <= W max If there is a subset of players C whose weights taken together equal or exceed the quota, C is called a winning coalition and v(C) = 1 – Players not part of winning coalition get v = 0

19. Weighted Voting Game: Definitions Minimal winning coalition: smallest subset of players whose weights reach the quota Veto player: player that appears in all winning coalitions, without him other players can’t reach quota – A game may not have a veto player

20. WVG Example N = {A, B, C, D} w A = 45, w B = 25, w C = 15, w D = 15; quota = 51 – Winning coalitions are {A, B} {A, C} {A, D} {A, B, C} {A, B, D} {A, C, D} {B, C, D} {A, B, C, D} no veto player Same weights, quota = 56 – Winning coalitions are {A, B} {A, C} {A, D} {A, B, C} {A, B, D} {A, C, D} {A, B, C, D} A is a veto player

21. Robot Coverage as WVG Determining weights of players (robots) – Modeled as coverage capability Environment considered as a 2-D grid Coverage map: Region covered by robot in last T timesteps Coverage efficiency: – Time: What fraction of the coverage map has been covered at least once? – Space: What fraction of the coverage map has been covered more than once? C i = a X  i – b X  i + C 0 a=2, b=1, C 0 = -0.04 C i = 1.96C i = 0.96

22. Breaking Ties Between Multiple Minimal Winning Coalitions Tie breaking using heuristic

23. Stability of Coalitions Is the partition of players imposed by the MWC going to be stable? – Yes, if it’s in the core of the game – Core: Sum of the payoffs of all the players in a team is at least as great as the payoff of the whole team Theorem 1: The core of a WVG is non-empty iff it has a veto player Theorem 2: The best minimal winning coalition (BMWC) is in the core Theorem 3: The best minimal winning coalition is unique

24. Outline of Algorithm for Team Reformation When a team needs to reconfigure – For all robots that are within communication range of a leader robot Find the veto players, set MWC = veto players – If no veto players, don’t form team and move individually If the veto players weights are enough to reach the quota then stop * Else add players from non-veto set to MWC, one at a time, until sum of players’ weights reaches quota *: If there are multiple MWCs apply heuristic to find BMWC

25. Experimental Results on Webots Experimental Settings Percentage of environment covered after 2 hours of clock-time simulations Repeated Coverage after 2 hours of clock-time simulations E-puck robots Wheel speed: 2.8 cm/sec On-board GPS Arena size: 4 m X 4m Robot size = Grid cell size = 7 cm X 7 cm Results averaged over 10 runs

26. Effect of Environment (Obstacles) 20 robots, quota = 0.7 X W max

27. Effect of Communication Range 20 robots, 10% of environment occupied by obstacles

28. Video Demo 1

29. Conclusions, Ongoing and Future Work Coalition games (WVGs) provide a suitable, structured mechanism to dynamically reconfigure multi-robot teams Ongoing work: Reduce the computation complexity of generating winning coalitions in a WVG Future work: Dynamically changing quota value based on performance, learning from long-term coverage histories Tests with physical robots

30. Acknowledgements We are grateful to the sponsors of our projects: – COMRADES project, Office of Naval Research – NASA Nebraska EPSCoR Mini-grant Thank You! For more information C-MANTIC Lab: http://cmantic.unomaha.edu

31. Video Demo 2

32. Video Demo 3