A Game-Theoretic Approach to Determining Efficient Patrolling Strategies for Mobile Robots Francesco Amigoni, Nicola Gatti, Antonio Ippedico

F. Amigoni, N. Gatti, A, Ippedico Scenario 2

F. Amigoni, N. Gatti, A, Ippedico Summary of Contributions Problem: determination of an efficient patrolling strategy for a mobile robot Idea: 1. model the scenario as an extensive-form game played by the patroller and the intruder 2. solve the game to find the strategy for the patroller 3

F. Amigoni, N. Gatti, A, Ippedico The Proposed Model: Assumptions Time is discrete, players play in turns Environment with n places Patroller detects the presence of the intruder (captures the intruder) when it is in the patroller’s current place Intruder knows the strategy of the patroller Patroller’s actions: move from one place to another one (incurring in different costs), movements can be between any pairs of places Intruder’s actions: wait or attempt to enter a place Entering a place takes d turns The game ends either when the intruder is captured or has entered a place Players payoffs are defined according to values attributed to places, to costs for moving between places, and to rewards for capturing the intruder Intruder can be of different types, each one with different values for places 4

F. Amigoni, N. Gatti, A, Ippedico The Proposed Model: Extensive-form Game The intruder knows the patroller’s strategy and the patroller knows it  commitment-based strategy for the patroller Finding an optimal solution is not easy, basically because the environment can dynamically change and because the game is infinite-horizon  approximate solution 5 intruder’s action patroller’s action ……………………

F. Amigoni, N. Gatti, A, Ippedico Solving the Game: Finding a Patrolling Strategy Greedy approach: we consider a slice of the extensive-form game as an independent strategic-form game Solving each slice means finding the next optimal action for the patrolling robot A slice can be solved by resorting to a multi-LP [Conitzer and Sandholm, EC 2006] or to a MILP [Paruchuri et al., AAMAS 2008] mathematical programming formulation Solution: mixed strategy for the patrolling robot: {γ 1,γ 2,…,γ n } 6 intruder’s action patroller’s action ……………………

F. Amigoni, N. Gatti, A, Ippedico Experimental Results The approach scales reasonably well with the number n of places (using the multi-LP formulation) and with the number of intruder’s types (using the MILP formulation) The approach can be applied to different environments Linear environment Ring and star environments The approach adapts to dynamic changes in the environment 7 methodutility our approach (multi-LP)79.0 uniform52.1 proportional69.5 random nodes, 10 runs, 500 steps each

F. Amigoni, N. Gatti, A, Ippedico Conclusions We proposed a game-theoretic approach to determining strategies for patrolling robots Modeling a patrolling situation as an extensive-form game Finding an (approximate) solution of the game Patrolling strategies found with our approach are efficient Ongoing work Optimal solutions for the extensive-form game More realistic scenarios Implementation on real robots 8

F. Amigoni, N. Gatti, A, Ippedico State of the Art Patrolling strategies for mobile robots are based on random movements to be unpredictable for an observing intruder Some approaches do not consider any model for the intruder [Paruchuri et al., AAMAS 2006] [Agmon et al., AAMAS 2008] Few approaches do consider a model for the intruder, in a strategic- form game formulation [Paruchuri et al., AAMAS 2007] Some game theoretical inconsistencies, as shown in [Gatti, ECAI 2008] 9