1. Hybrid Systems Modeling, Analysis, Control
Datta Godbole, John Lygeros, Claire Tomlin, Gerardo Lafferiere, George Pappas, John Koo
Jianghai Hu, Rene Vidal, Shawn Shaffert, Jun Zhang,
Slobodan Simic, Kalle Johansson, Maria Prandini
David Shim, Jin Kim, Omid Shakernia, Cedric Ma, Judy Liebmann and Ben Horowitz
(with the interference of) Shankar Sastry

2. What Are Hybrid Systems?
Dynamical systems with interacting continuous and discrete dynamics

3. Why Hybrid Systems? Modeling abstraction of Important in applications
Continuous systems with phased operation (e.g. walking robots, mechanical systems with collisions, circuits with diodes)
Continuous systems controlled by discrete inputs (e.g. switches, valves, digital computers)
Coordinating processes (multi-agent systems)
Important in applications
Hardware verification/CAD, real time software
Manufacturing, chemical process control,
communication networks, multimedia
Large scale, multi-agent systems
Automated Highway Systems (AHS)
Air Traffic Management Systems (ATM)
Uninhabited Aerial Vehicles (UAV), Power Networks

4. Control Challenges Large number of semiautonomous agents Coordinate to
Make efficient use of common resource
Achieve a common goal
Individual agents have various modes of operation
Agents optimize locally, coordinate to resolve conflicts
System architecture is hierarchical and distributed
Safety critical systems
Challenge: Develop models, analysis, and synthesis tools for designing and verifying the safety of multi-agent systems

5. Proposed Framework Control Theory Computer Science Hybrid Systems
Control of individual agents
Continuous models
Differential equations
Computer Science
Models of computation
Communication models
Discrete event systems
Hybrid Systems

6. Different Approaches

7. Automated Highway Systems
Goal
Increase highway throughput
Same highway infrastructure
Same level of safety
Same level of passenger comfort
Introduce automation
Partial: driver assistance, intelligent cruise control, warning system
Full: individual vehicles, mixed traffic, platooning
Complex problem
Technological issues (is it possible with current technology)
Social/Political issues (insurance and legal issues, equality)

8. Safety-Throughput Tradeoff
Contradictory demands
Safety: vehicles far and moving slowly
Throughput: vehicles close and moving fast
Proposed compromise
Allow low relative velocity collisions
In emergency situations
Two possible safe arrangements
Large spacing (leader mode)
Small spacing (follower mode)
Platooning concept

9. Control Hierarchy Implementation requires automatic control
Control hierarchy proposed in [Varaiya 93]
Regulation layer: braking, acceleration and steering
Coordination layer: maneuvers implemented by communication protocols
Link layer: flow control, lane assignment
Network layer: routing
Hybrid phenomena appear throughout
Switching controllers for regulation
Switching between maneuvers
Lane and maneuver assignment
Degraded modes of operation

10. Air Traffic Management Systems
Studied by NEXTOR and NASA
Increased demand for air travel
Higher aircraft density/operator workload
Severe degradation in adverse conditions
High business volume
Technological advances: Guidance, Navigation & Control
GPS, advanced avionics, on-board electronics
Communication capabilities
Air Traffic Controller (ATC) computation capabilities
Greater demand and possibilities for automation
Operator assistance
Decentralization
Free flight

11. Automated Platoons on I-15

12. Computable Hybrid Systems
Current ATM System
CENTER B
CENTER A
TRACON
VOR
SUA
20 Centers, 185 TRACONs, 400 Airport Towers
Size of TRACON: miles radius, 11,000ft
Centers/TRACONs are subdivided to sectors
Approximately 1200 fixed VOR nodes
Separation Standards
Inside TRACON : 3 miles, 1,000 ft
Below 29,000 ft : 5 miles, 1,000ft
Above 29,000 ft : 5 miles, 2,000ft
TRACON
GATES
Computable Hybrid Systems

13. Current ATM System Limitations
Inefficient Airspace Utilization
Nondirect, wind independent, nonoptimal routes
Centralized System Architecture
Increased controller workload resulting in holding patterns
Obsolete Technology and Communications
Frequent computer and display failures
Limitations amplified in oceanic airspace
Separation standards in oceanic airspace are very conservative
In the presence of the predicted soaring demand for air
travel, the above problems will be greatly amplified leading
to both safety and performance degradation in the future
Computable Hybrid Systems

14. Computable Hybrid Systems
A Future ATM Concept
CENTER B
CENTER A
TRACON
ALERT ZONE
PROTECTED ZONE
Free Flight from TRACON to TRACON
Increases airspace utilization
Tools for optimizing TRACON capacity
Increases terminal area capacity and throughput
Decentralized Conflict Prediction & Resolution
Reduces controller workload and increases safety
TRACON
Computable Hybrid Systems

15. Hybrid Systems in ATM Automation requires interaction between
Hardware (aircraft, communication devices, sensors, computers)
Software (communication protocols, autopilots)
Operators (pilots, air traffic controllers, airline dispatchers)
Interaction is hybrid
Mode switching at the autopilot level
Coordination for conflict resolution
Scheduling at the ATC level
Degraded operation
Requirement for formal design and analysis techniques
Safety critical system
Large scale system

16. Control Hierarchy Flight Management System (FMS) Strategic planning
Regulation & trajectory tracking
Trajectory planning
Tactical planning
Strategic planning
Decentralized conflict detection and resolution
Coordination, through communication protocols
Air Traffic Control
Scheduling
Global conflict detection and resolution

17. Hybrid Research Issues
Hierarchy design
FMS level
Mode switching
Aerodynamic envelope protection
Strategic level
Design of conflict resolution maneuvers
Implementation by communication protocols
ATC level
Scheduling algorithms (e.g. for take-offs and landings)
Global conflict resolution algorithms
Software verification
Probabilistic analysis and degraded modes of operation

18. Other Applications Uninhabited Aerial Vehicles (UAV)
Automated aerial vehicles (airplanes and/or helicopters)
Coordinate for search and rescue, or seek and destroy missions
Control hierarchy similar to ATM
Mode switching, discrete coordination, flight envelope protection
Power Electronic Building Blocks (PEBB)
Power electronics, with sensing, control, communication
Improve power network efficiency and reliability for utilities, hybrid electric vehicle, universal power ships
Control hierarchy: load balancing/shedding, network stabilization, pulse width modulation
Hybrid phenomena: modulation, input characteristic switching, scheduling

19. UAV Laboratory Configuration
TCP/IP
Wireless LAN
GROUNDSTATION
VIRTUAL COCKPIT
GRAPHICAL
EMMULATION
WIRELESS
HUB

20. Motivation
Goal
Design a multi-agent multi-modal control system for Unmanned Aerial Vehicles (UAVs)
Intelligent coordination among agents
Rapid adaptation to changing environments
Interaction of models of operation
Guarantee
Safety
Performance
Fault tolerance
Mission completion
Conflict Resolution
Collision Avoidance
Envelope Protection
Tracking Error
Fuel Consumption
Response Time
Sensor Failure
Actuator Failure
Path Following
Object Searching
Pursuit-Evasion

21. Hierarchical Hybrid Systems
Envelope Protecting Mode
Normal Flight Mode
Tactical Planner
Safety
Invariant 
Liveness
Reachability

22. The UAV Aerobot Club at Berkeley
Architecture for multi-level rotorcraft UAVs to date
Pursuit-evasion games to date
Landing autonomously using vision on pitching decks to date
Multi-target tracking to date
Formation flying and formation change 2002

23. Hybrid Automata Hybrid Automaton Remarks: State space Input space
Initial states
Vector field
Invariant set
Transition relation
Remarks:
countable,
State
Can add outputs, etc. (not needed here)

24. Executions Hybrid time trajectory, , finite or infinite with
Execution with and
Initial Condition:
Discrete Evolution:
Continuous Evolution: over , continuous, piecewise continuous, and
Remarks:
x, v not function, multiple transitions possible
q constant along continuous evolution
Can study existence uniqueness

25. Controller Synthesis: Example
2D conflict resolution
Ensure aircraft remain more than 5nmi from each other

26. Hybrid Automaton Specification
Discrete input variable determines maneuver initiation
Safety specification

27. More Abstractly ...
Consider plant hybrid automaton, inputs partitioned to:
Controls, U
Disturbances, D
Controls specified by “us”
Disturbances specified by the “environment”
Unmodeled dynamics
Noise, reference signals
Actions of other agents
Memoryless controller is a map
The closed loop executions are

28. Controller Synthesis Problem
Given H and find g such that
A set is controlled invariant if there exists a controller such that all executions starting in remain in
Proposition: The synthesis problem can be solved iff there exists a unique maximal controlled invariant set with
Seek maximal controlled invariant sets & (least restrictive) controllers that render them invariant
Proposed solution: treat the synthesis problem as a non-cooperative game between the control and the disturbance

29. Gaming Synthesis Procedure
Discrete Systems: games on graphs, Bellman equation
Continuous Systems: pursuit-evasion games, Isaacs PDE
Hybrid Systems: for define
states that can be forced to jump to for some
states that may jump out of for some
states that whatever does can be continuously driven to avoiding by
Initialization:
while do
end

30. Algorithm InterpretationX
Proposition: If the algorithm terminates, the fixed point is
the maximal controlled invariant subset of F

31. Computation One needs to compute , and
Computation of the Pre is straight forward (conceptually!): invert the transition relation
Computation of Reach through a pair of coupled Hamilton-Jacobi partial differential equations
Semi-decidable if Pre, Reach are computable
Decidable if hybrid automata are rectangular, initialized.

32. Application: Control of Automated Highway Systems
Design of vehicle controllers & performance estimation
Two concepts
platooning & individual vehicles
Network
Link
Coordination
Regulation
Lane keeping
Vehicle following
Maneuver selection
inter-vehicle comm
Dynamic routing
Flow optimization
Entry
Join
Speed,
vehicle
following
Lane
Change
Platoon
Following
Split
Exit

33. Vehicle Following & Lane Changing
Control actions: (vehicle i)
-- braking, lane change
Disturbances: (generated by neighboring vehicles)
-- deceleration of the preceding vehicle
-- preceding vehicle colliding with the vehicle ahead of it
-- lane change resulting in a different preceding vehicles
-- appearance of an obstacle in front
Operational conditions:
state of vehicle i with respect to traffic i
i-1
i-2 j
Computable Hybrid Systems

34. Game Theoretic Formulation
Requirements
Safety (no collision)
Passenger Comfort
Efficiency
trajectory tracking (depends on the maneuver)
Safe controller (J1): Solve a two-person zero-sum game
saddle solution (u1*,d1*) given by
Both vehicles i and i-1 applying maximum braking
Both collisions occur at T=0 and with maximum impact

35. Safe Vehicle Following Controller
Partition the state space into safe & unsafe sets
Design comfortable and
efficient controllers in
the interior
IEEE TVT 11/94
Safe set characterization
also provides sufficient
conditions for lane change
CDC 97, CDC98

36. Automated Highway System Safety
Theorem 1: (Individual vehicle based AHS)
An individual vehicle based AHS can be designed to produce no inter-vehicle collisions,
moreover disturbances attenuate along the vehicle string.
Theorem 2: (Platoon based AHS)
Assuming that platoon follower operation does not result in any collisions even with a possible inter-platoon collision during join/split, a platoon based AHS can be safe under low relative velocity collision criterion.
References
Lygeros, Godbole, Sastry, IEEE TAC, April 1998
Godbole, Lygeros, IEEE TVT, Nov. 1994

37. Example: Aircraft Collision Avoidance
Two identical aircraft at fixed altitude & speed: y v u y x v
I demonstrate our algorithm with the 3D example from last year’s paper: two aircraft at fixed altitude and speed. The evader or control aircraft is fixed at the origin, and the state variables are relative x-y position and relative heading. The inputs are that each aircraft can choose its turn direction and rate. d
‘evader’ (control)
‘pursuer’ (disturbance)

38. Continuous Reachable Sety x
Starting from the initial set, a cylinder parallel to the psi axis of radius 5, we grow the reachable set via the hamilton-jacobi equation.
[Mitchell, Bayen, Tomlin 2001]
[Tomlin, Lygeros, Sastry 2000]

39. Fast Wavefront Approximation Methods (Tomlin, Mitchell)

40. Visualization of Unsafe Set: Mitchell-Tomlin

41. Transition Systems Transition System Define for
Given equivalence relation define
A ~ block is a union of equivalence classes

42. Bisimulations of Transition Systems
A partition ~ is a bisimulation iff
are ~ blocks
For all and all ~ blocks is a ~ block
Alternatively, for
Why are bisimulations important?

43. Bisimulation Algorithm
initialize :
while such that
define
refine
If algorithm terminates, we obtain a finite bisimulation

44. Computability & Finitiness
Decidability requires the bisimulation algorithm to
Terminate in finite number of steps and
Be computable
For the bisimulation algorithm to be computable we need to
Represent sets symbollically,
Perform boolean combinations on sets
Check emptyness of a set,
Compute Pre(P) of a set P
Class of sets and vector fields must be topologically simple
Set operations must not produce pathological sets
Sets must have desirable finiteness properties

45. Mathematical Logic
Every theory of the reals has an associated language
Decidable theories
Every formula is equivalent to a quantifier free formula
Quantifier free formulas can be decided
Quanitifier elimination
Computational tools (REDLOG, QEPCAD)

46. O-Minimal Theories A definable set is
A theory of the reals is called o-minimal if every definable subset of the reals is a finite union of points and intervals
Example: for polynomial
Recent o-minimal theories
Semilinear sets
Semialgebraic sets
Exponential flows
Bounded Subanalytic sets
Spirals ?

47. O-Minimal Hybrid Systems
A hybrid system H is said to be o-minimal if
the continuous state lives in
For each discrete state, the flow of the vector field is complete
For each discrete state, all relevant sets and the flow of the vector field are definable in the same o-minimal theory
Main Theorem
Every o-minimal hybrid system admits a finite bisimulation.
Bisimulation alg. terminates for o-minimal hybrid systems
Various corollaries for each o-minimal theory

48. O-Minimal Hybrid Systems
Consider hybrid systems where
All relevant sets are polyhedral
All vector fields have linear flows
Then the bisimulation algorithm terminates
Consider hybrid systems where
All relevant sets are semialgebraic
All vector fields have polynomial flows
Then the bisimulation algorithm terminates

49. O-Minimal Hybrid Systems
Consider hybrid systems where
All relevant sets are subanalytic
Vector fields are linear with purely imaginary eigenvalues
Then the bisimulation algorithm terminates
Consider hybrid systems where
All relevant sets are semialgebraic
Vector fields are linear with real eigenvalues
Then the bisimulation algorithm terminates

50. O-Minimal Hybrid Systems
Consider hybrid systems where
All relevant sets are subanalytic
Vector fields are linear with real or purely imaginary eigenvalues
Then the bisimulation algorithm terminates
New o-minimal theories result in new finiteness results
Can we find constructive subclasses?
Must remain within decidable theory
Sets must be semialgebraic
Need to perfrom reachability computations
Reals with exp. does not have quantifier elimination

51. Linear Hybrid Systems A hybrid system H is said to be linear if
the continuous state lives in
For each discrete state, all relevant sets are semialgebraic
For each discrete state, the vector field is of the form
where matrix has rational entries
Let Then we can express
Focus on the subformula

52. Diagonalizable, Rational Eigenvalues
Let be a linear vector field, rational,
diagonalizable with rational eigenvalues. Then is
definable in the decidable theory of reals
Example: h

53. Diagonalizable, Imaginary Eigenvalues
Procedure is conceptually similar if is diagonalizable with purely imaginary, rational eigenvalues
Equivalence is obtained by
Suffices to compute over a period
Composing all the constructive results together gives in…
Let be a linear vector field, rational,
diagonalizable with purely imaginary rational eigenvalues.
Then is definable in the decidable theory of reals

54. Semidecidable Linear Hybrid Systems
Let H be a linear hybrid system H where for each discrete
location the vector field is of the form F(x)=Ax where
A is rational and nilpotent
A is rational, diagonalizable, with rational eigenvalues
A is rational, diagonalizable, with purely imaginary, rational eigenvalues
Then the reachability problem for H is semidecidable.
Above result also holds if discrete transitions are not necessarily initialized but computable

55. Decidable Linear Hybrid Systems
Let H be a linear hybrid system H where for each discrete
location the vector field is of the form F(x)=Ax where
A is rational and nilpotent
A is rational, diagonalizable, with rational eigenvalues
A is rational, diagonalizable, with purely imaginary, rational eigenvalues
Then the reachability problem for H is decidable.

56. Linear Hybrid Systems with Inputs
Let H be a linear hybrid system H where for each discrete
location, the dynamics are where A,B are
rational matrices and one of the following holds:
A is nilpotent, and
A is diagonalizable with rational eigenvalues, and
A is diagonalizable with purely imaginary eigenvalues and
Then the reachability problem for H is decidable.

57. Linear DTS (compare with Morari Bemporad)
X = n, U = {u|Eu}, D = {d|Gd}, f = {Ax+Bu+Cd},
F = {x|Mx}.
Pre(Wl) = {x | l(x)}
l(x) = u d | [Mlxl]c[Eu]
[(Gd>)(MlAx+MlBu+MlCd l)]
Implementation
Quantifier Elimination on d: Linear Programming
Quantifier Elimination on u: Linear Algebra
Emptiness: Linear Programming
Redundancy: Linear Programming

58. Decidability Results for Algorithm
The controlled invariant set calculation problem is
Semi-decidable in general.
Decidable when F is a rectangle, and A,b is in controllable canonical form for single input single disturbance.
Extensions:
Hybrid systems with continuous state evolving according to discrete time dynamics: difficulties arise because sets may not be convex or connected.
There are other classes of decidable systems which need to be identified.

60. Summary Methodology Applications Modeling Framework
Game theoretic approach to controller synthesis
Linear hybrid systems and computability
Applications
Synthesis of safe conflict resolution maneuvers
Safe controllers for automated highways
Verification of avionic software (CTAS, TCAS)
Flight Envelope Protection
Flight Mode Switching

61. Newer Research Modeling Control Analysis
Robustness, Zeno (Zhang, Simic, Johansson)
Simulation, on-line event detection (Johannson, Ames)
Control
Extension to more general properties (liveness, stability) (Koo)
Links to viability theory and viscosity solutions (Lygeros, Tomlin, Mitchell, Bayen)
Numerical solution of PDEs (Tomlin, Mitchell)
Analysis
Develop (exact/approximate) reachability tools (Vidal, Shaffert)
Complexity analysis (Pappas, Kumar)
Probabilistic Hybrid Systems (Hu)
Observability of Hybrid Systems (Vidal)