1. 1 Pursuit-Evasion (Robotic) Games The Marco Polo Case Army Research Lab Rafael Fierro MARHES Laboratory School of Electrical and Computer Engineering Oklahoma State University Stillwater OK 74078 Systems Workshop on Adaptive & Networks SWAN 06 December 8, 2006

2. 2 Outline Motivation and interests Vision-based formation control Dynamic boundary tracking Robotic Games  The Marco Polo Case Hierarchical Optimization  Shape changes in robot formations Conclusions

3. 3 Boeing/Insitu ScanEagles® in a cooperative target tracking mission Surveillance Reconnaissance Coordination Goals: Use multiple low-cost vehicles to cooperatively track evading targets in urban environments. Deployment of Ad Hoc wireless sensor networks WSN with nodes supporting limited mobility. Extending the mission life of Ad Hoc wireless sensor nets. To develop a mathematically sound framework for the analysis and design of feedback control of UV teams in the presence of intermittent, asynchronous communication and estimation. High-confidence Distributed Vehicles Networks for Urban Navigation

4. 4 Target Acquisition, Deployment, and Surveillance Developing algorithms to deal with the heterogeneity in the system (information and vehicles) is a key challenge in multi- vehicle coordination. To the best of our knowledge, this is a scientific gap that still requires closing.

5. 5 Vision-based Control of Formations O.A.A. Orqueda and R. Fierro, “Robust vision-based nonlinear formation control,” Proceedings of the American Control Conference, Minneapolis, MN, June 2006, pp. 1422-1427. Algorithm is based on a high- gain observer (HGO) and second order sliding mode, Robust to communication failures, Closed-loop system is Lyapunov stable.

6. 6 Marker and ID Detection Image capture Grayscale conversion Thresholding Contour extraction Contour selection ID recognition

7. 7 Noise Effect - Fog Distances from 0.6 m to 3.065 m

8. 8 Information flow/sensing levels Partial state feedback controller (FSFB) Robust partial state feedback controller (RSFB) Output feedback controller (OFB)

9. 9 Control for Sensing: Dynamic Boundary Tracking Environment Dynamic Boundary Abstraction and Estimation Remote Sensing Satellite or UAVs Dynamic Sensor Net 4 1 2 N 3 5 G  { V, E } } http://www.bairdsoftware.com/ Y. Cao and R. Fierro, CDC 2006.

10. 10 Pursuit-Evasion (Robotic) Games Multi-objective optimization for detecting and intercepting evading targets. References S. Ferrari, C. Cai, R. Fierro, and B. Perteet, “A multi-objective optimization approach to detecting and intercepting targets in pursuit-evasion games,” American Control Conference, September 2006. (Submitted) B. Perteet, J. McClintock, and R. Fierro, “A Multi-Vehicle Framework for the Development of Robotic Games: The Marco Polo Case,” IEEE Int. Conf. on Robotics and Automation, September, 2006. (Submitted) S. Ferrari, “Track coverage in sensor networks,” in Proceedings of the American Control Conference, Minneapolis, MN, June 14-16 2006, pp. 2053–2059. V. Isler, S. Kannan, and S. Khanna, “Randomized pursuit-evasion in a polygonal environment,” IEEE Trans. on Robotics, vol. 21, no. 5, pp. 875–884, 2005.

11. 11 The Marco Polo Case Game Rules: One player is labeled Marco. The others are called Polo. The player labeled Marco must keep their eyes shut while chasing the other players. If Marco announces “Marco”, all other players immediately announce “Polo”. This audio cue allows Marco to track the other players. When a player is tagged by Marco they become the new Marco. Marco Polo was a famous Croatian (Italian) trader and explorer who lived during the 13 th century.

12. 12 Why is this problem interesting? It combines  vSLAM  Motion coordination (i.e., control algorithms and communication protocols) within uncertain and dynamic environments, Intermittent communication and estimation, Occasional cooperation  Targets must (asynchronously) broadcast their positions,  but limited sensing, Human-robot interaction Learning Robust Intelligence -- new NSF cluster!!

13. 13 Mobile sensor agents and targets Mobile sensor agents (autonomous) Target model (may be tele-operated by a child) where v  is uniformly distributed in [0, V  ] and   is uniformly distributed in [0, 2  ).

14. 14 More formally… Problem Given a set P of N pursuers and a set T of M target agents within a specified game area S, find a set of policies for all pursuers in P which maximizes the of probability of intercepting partially- observed and unobserved tracks and minimizes the time t c required to capture fully-observed targets in T. denotes the set of detections of target positions  i.

15. 15 Example of workspace Partially-observed track, R i A D S Y r L2L2 L1L1 r CO j CR i X

16. 16 Track Coverage S. Ferrari, “Track coverage in sensor networks,” in Proceedings of the American Control Conference, Minneapolis, MN, June 14-16 2006, pp. 2053–2059. x 2-Coverage cones y  ξ2ξ2 y’ x’    L 1  L 2 Rectangular Aera of Interest (S) ξ1ξ1

17. 17 Optimal Sensor Placement Before optimization After optimization (k = 3) Goal: Place sensors such that we maximize the probability of detecting new targets entering the environment with at least k different sensors

18. 18 Control Strategies Hierarchical Hybrid System (Pursuer)

19. 19 Pursuit Strategy

20. 20 Track Coverage and Detection The map of the environment is known a priori and is cell decomposed into regions free of and occupied by obstacles. Two classes of sensors: static and mobile. Sensors are optimally placed in the environment offline such that the probability of detecting targets with k = 3 different sensors is maximized (using track coverage algorithm). Mobile sensors are deployed after at least two detections. Mobile sensors deploy to gain additional information about the target in order to capture it. A* algorithm used to find optimal path with a reward function.

21. 21 Track Coverage and Detection Sensor paths are planned by choosing a sequence of adjacent cells, or channel, in the environment such that obstacles are avoided while moving toward hypothesized target tracks maximizing the reward: where The reward function is is the Euclidean distance, is the change in sensor coverage, is the probability of detecting the target at the given cell (assumed to be 1 or 0), and are weighting parameters.

22. 22 Preliminary Simulation Initial location of mobile sensor Optimal path Detection points Static sensors Target Mobile sensor Hypothesized track Obstacles

23. 23 Preliminary Simulation

24. 24 Marco Polo: Experimental Results

25. 25 Hierarchical Optimization for Deployment of MAS We integrate model predictive control (MPC) and mixed integer linear programming (MILP) into a hierarchical framework. J(X)X,CJ(X)X,C JH(XH)XH,CHJH(XH)XH,CH J1L(X1L)X1L,C1LJ1L(X1L)X1L,C1L J2L(X2L)X2L,C2LJ2L(X2L)X2L,C2L JNL(XNL)XNL,CNLJNL(XNL)XNL,CNL |X| = O((N r 2 +N r N o )T) |X H | = O(N r 2 ) LP problem Easy to solve!!!! |X i L | = O((N r +N o )T) each problem can be solved in parallel from each agent

26. 26 Case Study 1: Surveillance

27. 27 Case Study 2: Shape Changes in Robot Formations The Problem: Consider a formation of robots currently in shape P, with an objective of transitioning to shape Q.  Minimizing the combined total distance that the robots must travel.  Minimizing the maximum distance that any robot must travel. P Q Spletzer & Fierro, ICRA 2005.

28. 28 Shape changes in robot formations (cont…) Definition: The shape of a formation is the geometrical information that remains when location, scale, and rotational effects are removed. I. L. Dryden and K. V. Mardia. Statistical Shape Analysis. John Wiley and Sons, 1998. Given a shape icon S defined by a k  m matrix where k is the number of robots, and m the dimension of the Euclidean Space, the shape [S] is defined by Equivalent shapes examples:

29. 29 Shape changes in robot formations Objective Shape Q Initial Shape P Spletzer & Fierro, ICRA 2005 Second-order cone programming Branca & Fierro, ACC 2006 Hierarchical optimization with collision avoidance

30. 30 Conclusions The difficulty in reliably operating robotic networks lies in the need to carefully integrate communication, sensing, processing, power management, and motion control. Goals of this research effort:  The development of formal models and analytical tools to characterize the properties of coordination algorithms for teams of mobile robots and networked embedded systems,  the development of software tools to assess the complexity and support the synthesis of high-confidence and efficient cooperative strategies of coordination algorithms,  Experiments with more robots in more realistic outdoor scenarios,  Evaluation of the performance of the team based on different communication schemes. Stability of networked hybrid systems.

31. 31 Questions Thanks for your time!! http://marhes.okstate.edu ?