Probabilistic Pursuit-Evasion Games with UGVs and UAVs René Vidal C. Sharp, D. Shim, O. Shakernia, J. Hespanha, J. Kim, S. Rashid, S. Sastry University of California at Berkeley 04/05/01

Outline Introduction Pursuit Evasion Games Map Building Pursuit Policies Hierarchical Control Architecture Strategic Planner, Tactical Planner, Regulation, Sensing, Control System, Agent and Communication Architectures Architecture Implementation Tactical Layer: UGVs, UAVs, Hardware, Software, Sensor Fusion Strategic Layer: Map Building, Pursuit Policies, Visual Interface Experimental Results Evaluation of Pursuit Policies Pursuit Evasion Games with UGV’s and UAV’s Conclusions and Current Research

Introduction: The Pursuit-Evasion Scenario Evade!

Introduction: Theoretical Issues Probabilistic map building Coordinated multi-agent operation Networking and intelligent data sharing Path planning Identification of vehicle dynamics and control Sensor integration Vision system

Pursuit-Evasion Games Consider approach in Hespanha, Kim and Sastry Multiple pursuers catching one single evader Pursuers can only move to adjacent empty cells Pursuers have perfect knowledge of current location Sensor model: false positives (p) and negatives (q) for evader detection Evader moves randomly to adjacent cells Extensions in Rashid and Kim Multiple evaders: each one is recognized individually Supervisory agents: can “fly” over obstacles and evaders, cannot capture Sensor model for obstacle detection as well

Map Building: Map of Obstacles Sensor model: p = prob of false positive q = prob of false negative For a map, M, If sensor makes positive reading: M (x,y,t) = (1-q)*M(x,y,t-1)/((1-q)*M(x,y,t-1)+p*(1-M(x,y,t)) If sensor make negative reading: M (x,y,t) = q*M(x,y,t-1)/(q*M(x,y,t-1)+(1-p)*(1-M(x,y,t))

Map Building: Map of Evaders At each t, 1. Measurement step + y(t) ={v(t),e(t),o(t)} model for sensor 2. Prediction step model for evader’s motion

Pursuit Policies Greedy Policy Global-Max Policy Pursuer moves to the adjacent cell with the highest probability of having an evader over all maps Strategic planner assigns more importance to local measurements Global-Max Policy Pursuer moves towards the place with the highest probability of having an evader in the map May not take advantage of multiple pursuers (may move to the same place)

Pursuit Policies Theorem 1 (Hespanha, Kim, Sastry): For a greedy policy, The probability of the capture time being finite is equal to one The expected value of the capture time is finite Theorem 2 (Hespanha, Kim, Sastry): For a stay-in-place policy, The expected capture time increases as the speed of the evader decreases If the speed of the evader is zero, then the probability of the capture time being finite is less than one.

Hierarchical System Architecture position of evader(s) position of obstacles strategy planner position of pursuers map builder communications network evaders detected obstacles pursuers positions Desired pursuers positions tactical planner trajectory regulation tactical planner & regulation actuator positions [4n] lin. accel. & ang. vel. [6n] inertial [3n] height over terrain [n] obstacles detected evaders detected vehicle-level sensor fusion state of helicopter & height over terrain obstacles detected control signals [4n] agent dynamics actuator encoders INS GPS ultrasonic altimeter vision Exogenous disturbances terrain evader

Agent Architecture Segments the control of each agent into different layers of abstraction The same high-level control strategies can be applied to all agents Strategic Planner Mission planning, high level control, communication Tactical Planner Trajectory planning, Obstacle Avoidance, Regulation Regulation Low level control and sensing

Communication Architecture Map building and Strategic Planner can be Centralized: one agent will receive sensor information, build and broadcast the map Decentralized: each agent build its own map and shares its readings with the rest of the team Communication network can be Perfect: no packet loss, no transmission time, no network delay. Here all pursuers have identical map Imperfect: each agent will update its map and make decisions with the information available to it

Architecture Implementation: Part I Common Platform for UGV and UGV On board computer: Tactical Planner and Sensor Fusion GPS: Positioning Vision system: Obstacle and Evader Detection Wavelan and Ayllu: Communication Specific UGV Platform Pioneer Robot: sonars, dead-reckoning, compass Micro-controller: regulation and low-level control Saphira or Ayllu: Tactical Planning Specific UAV Platform Yamaha R-50: INS, ultrasonic sensors, inertial sensors, compass Navigation Computer: regulation and low-level control David Shim Control System

Vision System: PTZ & ACTS Hardware Onboard Computer: Linux Sony pan/tilt/zoom camera PXC200 frame grabber Camera Control Software in Linux Send PTZ Commands Receive Camera State ACTS System Captures and processes video 32 color channels 10 blobs per channel Extract color information and sends it to a TCP socket Number of blobs, Size and position of each blob

Visual based position estimation Motion Model Image Model Camera position and orientation Helicopter orientation relative to ground Camera orientation relative to helicopter Camera calibration Width, height, zoom Robot position estimate

Communication Hardware Network is setup in ad-hoc mode Lucent Wavelan wireless card (11Mbs) Network is setup in ad-hoc mode TCP/IP sockets Ayllu TBTRF: SRI mobile routing scheme Set of behaviors for distributed control of multiple mobile robots Messages can be passed among behaviors The output of a behavior can be connected to a local or remote input of another behavior

Pioneer Ground Robots Hardware Sensors Communication Micro controller: motion control Onboard computer: communication, video processing, camera control Sensors Sonars: obstacle avoidance, map building GPS & compass: positioning Video camera: map building, navigation, tracking Communication Serial Wave-LAN: communication between robots and base station Radio modem: GPS communication

Yamaha Aerial Robots Yamaha R-50 helicopter Navigation Computer Pentium 233, running QNX Low Level Control - Sensing GPS INS UAV controller David Shim Controller Vehicle Control Language Vision Computer Serial communication to receive state of the helicopter We do not send commands yet

Architecture Implementation: Part II Strategic Planner Navigation Computer Serial Vision Computer Helicopter Control GPS: Position INS: Orientation Camera Control Color Tracking UGV Position Estimation Communication Map Building Pursuit Policies Communication Runs in Simulink Same for Simulation and Experiments UAV Pursuer TCP/IP Serial Robot Micro Controller Robot Computer UGV Pursuer UGV Evader Robot Control DeadReck: Position Compass: Heading Camera Control Color Tracking GPS: Position Communication

Pursuit-Evasion Game Experiment using Simulink PEG with four UGVs Global-Max pursuit policy Simulated camera view (radius 7.5m with 50degree conic view) Pursuer=0.3m/s Evader=0.1m/s MAX

Experimental Results: Evaluation of Policies

Experimental Results: Evaluation of Policies

Experimental Results: Pursuit Evasion Games with 1UAV and 2 UGVs (Summer’ 00)

Experimental Results: Pursuit Evasion Games with 4 UGVs and 1 UAV (Spring’ 01)

Experimental Results: Pursuit Evasion Games with 4UGVs and 1 UAV (Spring’ 01)

Conclusions and Current Research The proposed architecture has been successfully applied to the control of multiple agents for the pursuit evasion scenario Experimental results confirm theoretical results Global-max outperforms greedy in a real scenario and is robust to changes in evader motion What’s missing: Vision computer controlling the helicopter Current Research Collision Avoidance and UAV Path Planning Montecarlo based learning of Pursuit Policies Communication