1. ONR July 21, 1998 Exploration of Hybrid and Intelligent Control Architectures in Conjunction with Probabilistic Verification S. Shankar Sastry July 21, 1998 Electronics Research Laboratory University of California, Berkeley ONR UCAV Project Overview

2. ONR July 21, 1998 Problem: Design of Intelligent Control Architectures for Distributed Multi-Agent Systems  An architecture design problem for a distributed system begins with specified safety and efficiency objectives for each of the system missions (surveillance, reconnaissance, combat, transport) and aims to characterize control, observation and communication. – Mission decomposition among different agents – Task decomposition for each agent – Inter-agent and agent—mother ship coordination – Continuous control and mode switching logic for each agent – Fault management  This research attempts to develop fundamental techniques, theoretical understanding and software tools for distributed intelligent control architectures with UCAV as an example.

3. ONR July 21, 1998 Fundamental Issues for Multi-Agent Systems  Central control paradigm breaks down when dealing with distributed multi-agent systems – Complexity of communication, real-time performance – Risk of single point failure  Completely decentralized control – Has the potential to increase safety, reliability and speed of response – But lacks optimality and presents difficulty in mission and task decomposition  Real-world environments – Complex, spatially extended, dynamic, stochastic and largely unknown  We propose a hierarchical perception and control architecture – Fusion of the central control paradigm with autonomous intelligent systems – Hierarchical or modular design to manage complexity – Inter-agent and agent–ship coordination to achieve global performance – Robust, adaptive and fault tolerant hybrid control design and verification – Vision-based control and navigation

4. ONR July 21, 1998 Autonomous Control of Uninhabited Combat Air Vehicles  UCAV missions – Surveillance, reconnaissance, combat, transport  Problem characteristics – Each UCAV must switch between different modes of operation Take-off, landing, hover, terrain following, target tracking, etc. Normal and faulted operation – Individual UCAVs must coordinate with each other and with the mothership For safe and efficient execution of system-level tasks: surveillance, combat For fault identification and reconfiguration – Autonomous surveillance, navigation and target tracking requires feedback coupling between hierarchies of observation and control

5. ONR July 21, 1998 Research Objectives: Design and Evaluation of Intelligent Control Architectures for Multi-agent Systems such as UCAV’s Research Thrusts  Intelligent control architectures for coordinating multi-agent systems – Decentralization for safety, reliability and speed of response – Centralization for optimality – Minimal coordination design  Verification and design tools for intelligent control architectures – Hybrid system synthesis and verification (deterministic and probabilistic)  Perception and action hierarchies for vision-based control and navigation – Hierarchical aggregation, wide-area surveillance, low-level perception Experimental Testbed  Control of multiple coordinated semi-autonomous DV8 helicopters

6. ONR July 21, 1998 Methods  Formal Methods – Hybrid systems (continuous and discrete event systems) Modeling Verification Synthesis – Probabilistic verification – Vision-based control  Semi-Formal Methods – Architecture design for distributed autonomous multi-agent systems – Hybrid simulation – Structural and parametric learning – Real-time code generation – Modularity to manage: Complexity Scalability Expansion Methods

7. ONR July 21, 1998 Thrust 1: Intelligent Control Architectures  Coordinated multi-agent system – Missions for the overall system: surveillance, combat, transportation – Limited centralized control Individual agents implement individually optimal (linear, nonlinear, robust, adaptive) controllers and coordinate with others to obtain global information, execute global plan for surveillance/combat, and avoid conflicts – Mobile communication and coordination systems Time-driven for dynamic positioning and stability Event-driven for maneuverability and agility  Research issues – Intrinsic models – Supervisory control of discrete event systems – Hybrid system formalism Research Thrust 1: Intelligent Control Architectures

8. ONR July 21, 1998  Given a strategic objective and local observation – What are the required information protocols with Human- centered system and other autonomous agents to command tactical control? Given a distributed control problem and the local observation at each site, what is the inter-site communication (minimal) or coordination protocols required to solve this problem?  Given a cooperative mission – What is the strategic objective (possibly dynamic) of each autonomous agent? How to distribute among the available agents a specified centralized control problem? Decentralized Observation, Communication and Control for Multi- Agent Systems

9. ONR July 21, 1998 Agent Communication Channels A1 A2 A3 Plant (L p ) The agents have partial observation but can exchange messages. The plant has a set of unobservable distinguished events (failures). OBJECTIVE: Design the inter-agent communication scheme required to detect and isolate the distinguished events Decentralized Observation and Communication for Discrete Event Systems

10. ONR July 21, 1998 Theorem 1 (L p,  poi,  fi  i  ) is decentrally diagnosable if there exists n  N  such that for all  f   f, u  f v  L p,  v  n, implies (w  L p )  ( i, P  poi (w) = P  poi  (u  f v  (  f  w  If any two sufficiently long plant traces look the same to all the agents, then either they have no failures or have all the same failures. Synthesis: The communicate all plant observations solution works. General drawback: Redundant information is communicated. L (  ) may not be regular even though L p is regular. Current focus: Minimal communication, protocol synthesis, trace abstraction Documentation: Draft paper available and sent to WODES’98 Synthesis of Inter-agent Communication for Decentralized Observation

11. ONR July 21, 1998  Each agent has a set of controllable events  Controllable events are a subset of the set of observable events  The next event is either an uncontrollable event from the plant, a controllable event enabled by an agent, or a message event scheduled by an agent  A control objective is specified by a language  Investigate the existence and synthesis of coordination protocols Decentralized Control of Discrete Event Systems Problem Formulation

12. ONR July 21, 1998  Advantages: – Will synthesize symbolic, event-driven, inter-agent communication over a finite message set – Very simple models permitting logical or combinatorial analysis and insights – AHS example: Worked for most coordinating maneuvers other than stability properties for vehicle following  Limitation: No formal way to capture continuous dynamics – The semantics of an event is generally some alignment or safety conditions in velocity, position, and euler angles with respect to targets or other agents – SOLUTION: Distributed control of hybrid systems systems Communication and Control Synthesis for DES models

13. ONR July 21, 1998 UCAV Dynamics Intelligent Control Architecture Strategic Layer Mission Control Tactical Layer Regulation Layer Strategic Objective Inter-UCAV Coordination Trajectory Constraints Sensor Info on Targets, UCAV’s Environmental Sensors Trajectory Actuator Commands Replan Tracking errors Flight Mode Switching Trajectory Planning Trajectory Tracking Set Point Control Mission Planning Resource Allocation Generating Trajectory Constraints Fault Management UCAV Control Architecture

14. ONR July 21, 1998 Preliminary Control Architecture for Coordinating UCAVs  Regulation Layer (fully autonomous) – Control of UCAV actuators in different modes: stabilization and tracking  Tactical Layer (fully autonomous) – Safe and efficient trajectory generation, mode switching – Strategic Layer (semi-autonomous) – Generating trajectory constraints and influencing the tasks of other agents using UCAV-UCAV and UCAV-ship coordination for efficient Navigation, surveillance, conflict avoidance – Fault management – Weapons configuration  Mission Control Layer (centralized) – Mission planning, resource allocation, mission optimization, mission emergency response, pilot interface

15. ONR July 21, 1998 Thrust 2: Verification and Design Tools The conceptual underpinning for intelligent multi-agent systems is the ability to verify sensory-motor hierarchies perform as expected  Difficulties with existing approaches: – Model checking approaches (algorithms) grow rapidly in computational complexity – Deductive approaches are ad-hoc  We are developing hybrid control synthesis approaches that solve the problem of verification by deriving pre-verified hybrid system. – These algorithms are based on game-theory, hence worst-case safety criterion – We are in the process of relaxing them to probabilistic specifications. Research Thrust 2: Verification and Design Tools

16. ONR July 21, 1998 Thrust 2: Verification and Design Tools  Approach – The heart of the approach is not to verify that every run of the hybrid system satisfies certain safety or liveness parameters, rather to ensure critical properties are satisfied with a certain safety critical probability  Design Mode Verification (switching laws) – To avoid unstable or unsafe states caused by mode switching (takeoff, hover, land, etc.)  Faulted Mode Verification (detection and handling) – To maintain integrity and safety, and ensure gradual degraded performance  Probabilistic Verification (worst case vs. the mean behavior) – To soften the verification of hybrid systems by rapprochement between Markov and Bayesian decision networks Hybrid Control Synthesis and Verification

17. ONR July 21, 1998 Controller Synthesis for Hybrid Systems  The key problem in the design of multi-modal or multi-agent hybrid control systems is a synthesis procedure. controller synthesis. It is based on the notion of a game theoretic approach to hybrid control design.  Our approach to controller synthesis is in the spirit of controller synthesis for automata as well as continuous robust controller synthesis. It is based on the notion of a game theoretic approach to hybrid control design.  Synthesis procedure involves solution of Hamilton Jacobi equations for computation of safe sets.  The systems that we apply the procedure to may be proven to be at best semi-decidable, but approximation procedures apply.

18. ONR July 21, 1998 Thrust 3: Perception and Action Hierarchies Design a perception and action hierarchy centered around the vision sensor to support surveillance, observation, and control functions  Hierarchical vision for planning at different levels of control hierarchy – Strategic or situational 3D scene description, tactical target recognition, tracking, and assessment, and guiding motor actions  Control around the vision sensor – Visual servoing and tracking, landing on moving platforms Research Thrust 3: Perception and Action Hierarchies

19. ONR July 21, 1998 What Vision Can Do for Control  Global situation scene description and assessment – Estimating the 3D geometry of the scene, object and target locations, behavior of the objects Allows looking ahead in planning, anticipation of future events Provides additional information for multi-agent interaction  Tactical target recognition and tracking – Using model-based recognition to identify targets and objects, estimating the motion of these objects Allows greater flexibility and accuracy in tactical missions Provides the focus of attention in situation planning

20. ONR July 21, 1998 Relation between Control and Vision  Higher level visual processing: precise, global information, computational intensive  Lower level visual processing: local information, fast, higher ambiguity Task decomposition for each agent Inter-agent, agent—mother ship coordination Higher level Lower level The control architecture needs The vision system provides Situation, 3D scene description Target recognition Continuous control Object tracking Motion detection, and optical flow Guided motor action

21. ONR July 21, 1998 Key Issues in Vision and Control: Deliver the Right Information at the Right Time  How to coordinate the planning stage with sensing stage – The planner should adjust to the speed and uncertainty of the vision system – The vision system should optimize its information flow from the lower level to the higher level, given the need of the planner  How to adjust the focus of attention – Selecting attention of visual processing in terms of the object locations, as well as level of abstraction – Fine tuning lower-level vision-motor control loop  A well-designed lower-level vision-motor control alleviates computation requirements of higher-level visual processing

22. ONR July 21, 1998 Approach for Hierarchical Vision Processing  Use grouping to extract a compact description of the scene from lower processing – Reduces the computation complexity of higher-level reasoning, provides a basis for attention selection – Information estimated from “big picture” of the scene is less likely to be affected by noise in the sensor – Efficient computation algorithm which is able to capture the “big picture” of a scene has been developed General results reported in CVPR’97, results on motion reported in ICCV’98

23. ONR July 21, 1998 Approach for Hierarchical Vision Processing  Applying higher-level reasoning on the groups extracted – Model-based object recognition Matching image groups to object models – 3D scene geometry estimation Based on the motion correspondence found – Tracking and behavior analysis of objects – Applying Bayesian theory in selecting the right level of visual processing

24. ONR July 21, 1998 Approach for Lower-level Vision—Motor Control  Vision-guided motor control – Use low-level image, motion flow information in formulating motor control law Tracking in the 3D coordinates –Use optical flow equations to build a model of the scene in 3D space –Look-ahead control law to allow for visual processing time Tracking in the image plane (2D) –Track objects (such as the landing pad) in image frame –Relate image measurement (such as image location of the pad, curvature of the lane marker) to motor control law

25. ONR July 21, 1998 Research Contributions  Fundamental Research Contributions – Design of hybrid control synthesis and verification tools that can be used for a wide range of real-time embedded systems – Design of vision and control hierarchies for surveillance and navigation Hierarchical vision for planning at different levels of control hierarchy Control around the vision sensor  Our multi-agent control architecture can be used for many applications – ONR applications UCAVs, simulated battlefield environment, distributed command and control, automatic target recognition, decision support aids for human-centered systems, intelligent telemedical system – General engineering applications Distributed communication systems, distributed power systems, air traffic management systems, intelligent vehicle highway systems, automotive control

26. ONR July 21, 1998 Software Tools for Synthesis and Verification Research Schedule O N D J F M A M J J A S FY 98 FY 99 FY 00 Intelligent Control Architectures Public Tests Perception O N D J F M A M J J A S Smart Aerobots 3D Simulation Verification Tools O N D J F M A M J Label Recognition C++ QNX Real-Time Visual Situation Assessment Cal Day Demo April 17 Terrain Following Control Scheme Vision System for Autonomous Takeoff/Landing Integrated System for Target Recognition and Terrain Following for Multiple UCAVs Multi-Agent Decentralized Observation System Robotic Helicopter Competition Aug 12-13, Richland, WA Preliminary UCAV Architecture Hybrid Control Synthesis Methods Final UCAV Architecture Probabilistic Verification Theory Performance Evaluation of UCAV Architecture UCAV Architecture Demo Cal Day Demo Robotic Helicopter Competition

27. ONR July 21, 1998 Deliverables TaskDurationDeliverables Intelligent Control Architectures Specification Tools7/97 - 7/98software, technical reports Design Tools7/97 - 9/99software, technical reports Architecture Evaluation Environment7/97 - 7/00software, technical reports UCAV Application7/97 - 7/00experiments, technical reports Verification Tools Design Mode Verification7/97 -12/98software, technical reports Faulted Mode Verification7/97 - 9/99software, technical reports Probabilistic Verification9/97 - 9/99 technical reports Perception Surveillance7/97 - 9/99software, experiments Hierarchical Vision7/97 - 7/00software, technical reports Visual Servoing9/97 - 7/00experiments, technical reports UCAV Application7/97 - 7/00experiments, technical reports

28. ONR July 21, 1998 Measures of Program Success  FY97-98 – Design of preliminary UCAV architeture – Design of hybrid control synthesis methods – Design of multi-agent decentralized observation system  FY98-99 – Development of probabilistic verification theory – Final UCAV architecture design – Vision and control for terrain following, take-off and landing for single UCAV  FY99-00 – Performance evaluation of UCAV architecture – Integration of vision and control for multiple coordinating UCAVs – Final version of the software tools for Hybrid control synthesis and verification, and Decentralized observation and control – Demonstration of UCAV architecture using the helicopter testbed

29. ONR July 21, 1998 FY97-98 Accomplishments Controller synthesis for hybrid systems. Developed algorithms and computational procedures for designing verified hybrid controllers optimizing multiple objectives Multi-agent decentralized observation problem. Designed inter-agent communication scheme to detect and isolate distinguished events in system dynamics SmartAerobots. 3D virtual environment simulation. Visualization tool for control schemes and vision algorithms—built on top of a simulation based on mathematical models of helicopter dynamics Label recognition: prototype in Matlab, then in C++ (QNX real-time)

30. ONR July 21, 1998 Berkeley Team NameRoleTelE-mail Shankar SastryPrincipal(510) 642-7200sastry@robotics.eecs.berkeley.edu Investigator(510) 642-1857 (510) 643-2584 Jitendra MalikCo-Principal(510) 642-7597malik@cs.berkeley.edu Investigator Datta Godbole Research(510) 643-5806godbole@robotics.eecs.berkeley.edu Engineer(510) 231-9582 John LygerosPostdoc(510) 643-5795lygeros@robotics.eecs.berkeley.edu Jianbao ShiPostdoc (510) 642-9940jshi@cs.berkeley.edu Omid ShakerniaGraduate Student(510) 643-2383omids@robotics.eecs.berkeley.edu

31. ONR July 21, 1998 Teaming and Interdependency  Collaboration with Prof. Varaiya (Berekeley) in designing a hierarchical control architecture for coordinating UCAVs  Collaboration with Prof. Russell (Berkeley) in developing probabilistic design and analysis tools  Collaboration with Prof. Zadeh (Berkeley) on soft computing tools for control of UCAVs and mode transition methods for DV 8 developed using fuzzy control  Collaboration with Prof. Speyer (UCLA) on fault detection and handling methods  Collaboration with Prof. Morse (Yale) on vision-guided navigation  Informal conversations with Prof. Anderson (ANU), Prof. Hyland (Michigan) and visit to Naval Post Graduate School  Pending: more formal collaborations with Profs. Narendra, Morse