Decentralized Optimization, with application to Multiple Aircraft Coordination Decision Making Under Uncertainty MURI Review, July 2002 Gökhan Inalhan, Dusan Stipanovic, Claire Tomlin Hybrid Systems Laboratory Department of Aeronautics and Astronautics Stanford University

Motivating applications (Source: Boeing X45-A) (Source: Northrop Grumman-X47A) (Source: NASA Ames)

Hybrid systems Continuous systems controlled by a discrete logic: embedded systems (autopilot logic) Coordinating processes: multi-vehicle systems interfacing continuous control with coordination protocols Continuous systems with a phased operation: (biological cell growth and division) continuous systems (control) discrete systems (computer science)

Verification: a mathematical proof that the system satisfies a property Controller synthesis: the design of control laws to guarantee that the system satisfies the property Methods give definitive answers, unlike simulation Often give surprising answers, trajectories which one might not think to simulate Reduces development time, cost of certification Verification and Controller Synthesis unsafe initial

Verification: a mathematical proof that the system satisfies a property Controller synthesis: the design of control laws to guarantee that the system satisfies the property Methods give definitive answers, unlike simulation Often give surprising answers, trajectories which one might not think to simulate Reduces development time, cost of certification Verification and Controller Synthesis unsafe initial

Verification: a mathematical proof that the system satisfies a property Controller synthesis: the design of control laws to guarantee that the system satisfies the property Methods give definitive answers, unlike simulation Often give surprising answers, trajectories which one might not think to simulate Reduces development time, cost of certification Verification and Controller Synthesis unsafe initial

Verification: a mathematical proof that the system satisfies a property Controller synthesis: the design of control laws to guarantee that the system satisfies the property Safety Property can be encoded as a condition on the system’s reachable set of states Verification and Controller Synthesis unsafe unsafe initialization initial unsafe safe, under appropriate control

Example: Aircraft Collision Avoidance Two identical aircraft at fixed altitude & speed: ‘evader’ (control) ‘pursuer’ (disturbance) x y u v  d v

Continuous Reachable Set  x y Solve: Display:

Collision Avoidance Filter Simple demonstration –Pursuer: turn to head toward evader –Evader: turn to head right pursuer safety filter’s input modification pursuer’s inputevader’s desired input evader evader’s actual input unsafe set collision set Movies …

Blunder Zones for Closely Spaced Approaches evader EEM Maneuver 1: accelerate EEM Maneuver 2: turn 45 deg, accelerate EEM Maneuver 3: turn 60 deg

Blunder Zone is shown by the yellow contour Red Zone in the green tunnel is the intersection of the BZ with approach path. The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay Implementation: Display design courtesy of Chad Jennings, Andy Barrows, David Powell

Map View showing a blunder The BZ calculations are performed in real time (40Hz) so that the contour is updated with each video frame.

Verified Mode Switching in Autopilots

Use in Cockpit Interface Verification Controllable flight envelopes for landing and Take Off / Go Around (TOGA) maneuvers may not be the same Pilot’s cockpit display may not contain sufficient information to distinguish whether TOGA can be initiated flare flaps extended minimum thrust rollout flaps extended reverse thrust slow TOGA flaps extended maximum thrust TOGA flaps retracted maximum thrust flare flaps extended minimum thrust rollout flaps extended reverse thrust TOGA flaps retracted maximum thrust revised interface existing interface controllable flare envelope controllable TOGA envelope intersection

V1 V2 V3V4 Communication Zone Safety Assurance Zone V1 V2 V3 V4 A More General Problem Structure

Neighborhood of i th vehicle (Decomposed) Centralized Optimization

Fixed time horizon – complete global map Bargaining start fixed time horizon

Flight Plans published by aircraft 1

Another Example

Flight Plans published by aircraft 1

Receding horizon – incomplete global map moving time horizon Bargaining start

Constraints embed: local dynamics: coordinated turn and straight flight [h d i ] input constraints: limited turn rate and velocity [g e i ] global coordination constraints: minimum safety assurance [g s ij ] for all j within neighborhood of i Local Optimization with Constraints

Centralized Optimization Decomposed Centralized Optimization Decomposition I Pareto optimality Nash equilibrium

Define Hamiltonian for each subsystem: is a Nash equilibrium for the centralized optimization problem if: where Thus, none of the subsystems can improve its solution, with all other subsystems’ solutions remaining fixed. Nash Equilibrium for Centralized Problem

Decomposed Centralized Optimization Decentralized Optimization Decomposition II Nash equilibrium Local optimal solutions

Define Hamiltonian for each subsystem: is a Nash equilibrium for the decentralized optimization problem if: Optimal solutions by each of the subsystems Nash Equilibrium for Decentralized Problem Proposition: is a Nash equilibrium of the centralized problem if and only if it is a Nash equilibrium of the decentralized problem

Example: Nash Equilibrium at (0,0)

Global contraction function from the local optimization structures For a particular solution, local optimization of the i th vehicle only affects the portion of F tied to its own local optimization Using Penalty function methods

Eliminates cases in which a subsystem is artificially acting against a constraint dictated by another group Eliminates cases in which two subsystems act against each other with non-identical constraints Cooperation Assumptions

Multiple solutions, or “threads”, exist within the system: Vehicle #1Vehicle #2 Vehicle #3 Vehicle #4 Iteration #1 Iteration #2 Iteration #3 Iteration #4 Nash Bargaining with Multiple Threads

Convergence Results 1.Global convergence to a (not necessarily feasible) Nash ‘solution’ 2.If the gradients of the constraint functions are linearly independent (Linear Constraint Qualification Condition, LICQ), then global convergence to a feasible Nash solution 3.Pareto optimality for convex problems [Inalhan, Stipanovic, Tomlin. Decentralized Optimization, with Application to Multiple Aircraft Coordination. CDC 2002, Submitted to JOTA]

V1 V2 V3 V4 4-Vehicle Example

Flight Plans published by aircraft 1

Decentralized Initialization Procedure Heuristics –Multiple-Depots (Vehicles), Time-windows for access, Priority on objectives and the vehicles –Iterative selection process carried at each vehicle –Best solution in the fleet is then selected from each vehicle’s solution set Applied to other problems of interest…

Non-cooperative Full information Cooperative Full information Cooperative incomplete information Non-cooperative No information Lack of information  Bounded Irrationality Spectrum of Approaches

Research Goals Design of provably correct and safe decentralized control protocols –Adapt to coordination –Allow for dynamic reconfiguration Treatment of information –Multi-scale provisioning of data based on inputs from various sensing modalities –Urgency of the need for sensed data –Available bandwidth Verification algorithms –Used during design phase, to reduce the time spent during the validation phase

Stanford DragonFly UAV 10 ft wingspan 12 ft wingspan [Jang, Teo, Inalhan, and Tomlin, DASC 2001], [Jang and Tomlin, AIAA GNC 2002]