1. Lecture #15 Supervisory Control João P. Hespanha University of California at Santa Barbara Hybrid Control and Switched Systems

2. Summary Supervisory control overview Estimator-based linear supervisory control Examples

3. Supervisory control supervisor process  controller 1 controller n y u w  Motivation: in the control of complex and highly uncertain systems, traditional methodologies based on a single controller do not provide satisfactory performance. bank of candidate controllers measured output control signal exogenous disturbance/ noise switching signal Key ideas: 1.Build a bank of alternative controllers 2.Switch among them online based on measurements For simplicity we assume a stabilization problem, otherwise controllers should have a reference input r

4. Supervisory control Supervisor: places in the feedback loop the controller that seems more promising based on the available measurements typically logic-based/hybrid system supervisor process  controller 1 controller n y u w  Motivation: in the control of complex and highly uncertain systems, traditional methodologies based on a single controller do not provide satisfactory performance. measured output control signal exogenous disturbance/ noise switching signal bank of candidate controllers

5. Multi-controller  controller 1 controller n u  bank of candidate controllers measured output control signal switching signal y Conceptual diagram: not efficient for many controllers & not possible for unstable controllers

6. Multi-controller Given a family of (n-dimensional) candidate controllers u measured output control signal switching signal y   controller 1 controller n u  bank of candidate controllers measured output control signal switching signal y

7. Multi-controller switching signal t  t   = 1  = 3  = 2  = 1 Given a family of (n-dimensional) candidate controllers u  measured output control signal switching signal y switching times

8. Supervisor supervisor  switching signal u measured output control signal y Typically an hybrid system:  ´ continuous state  ´ discrete state continuous vector field discrete transition function output function u y

9. Supervisory vs. Adaptive control  u y uy  u y uy Rapid adaptation –  need not vary continuously Flexibility & modularity – can use off-the-shelf candidate controllers, estimators, and several alternative switching logics (allows reuse of existing, nonadaptive theory) Between switching times one recovers the nonadaptive behavior hybrid supervisorcontinuous adaptive tuner

10. Types of supervision Pre-routed supervision  = 1  = 2  = 3 try one controllers after another in a pre-defined sequence stop when the performance seems acceptable not effective when the number of controllers is large Estimator-based supervision (indirect) estimate process model from observed data select controller based on current estimate – Certainty Equivalence Performance-based supervision (direct) keep controller while observed performance is acceptable when performance of current controller becomes unacceptable, switch to controller that leads to best expected performance based on available data

11. Estimator-based supervision’s setup: Example #1 process parameter p * is equal to p › (a,b) 2  use controller C q with controller selection function p * =(a *, b * ) 2  › [–1,1] £ {–1,1} Process is assumed to be u control signal y measured output unknown parameters we consider three candidate controllers: controller C 1 : u = 0to be used when a * · –.1 controller C 2 : u = 1.1yto be used when a * > –.1 & b * = –1 controller C 3 : u = – 1.1yto be used when a * > –.1 & b * = +1 process

12. Estimator-based supervision’s setup Process is assumed to be in a family  p ´ small family of systems around a “nominal” process model N p parametric uncertainty unmodeled dynamics for each process in a family  p, at least one candidate controller C q, q 2  provides adequate performance. process in  p, p 2  controller C q with q =  (p) provides adequate performance controller selection function process u control signal y measured output w exogenous disturbance/ noise

13. Estimator-based supervisor: Example #1 multi- estimator u measured output control signal y y – + – + Since  has infinitely many elements, this multi-estimator would have to be infinite dimensional !?(not a very good multi-estimator) Process is assumed to be unknown parameters Multi-estimator p * =(a *, b * ) 2  › [–1,1] £ {–1,1} 8 p=(a, b) 2  › [–1,1] £ {–1,1} y p ´ estimate of the output y that would be correct if the parameter was p=(a,b) e p ´ output estimation error that would be small if the parameter was p=(a,b) consider the y p corresponding to p = p *

14. Estimator-based supervisor: Example #1 Process is assumed to be Multi-estimator y p ´ estimate of the output y that would be correct if the parameter was p=(a,b) e p ´ output estimation error that would be small if the parameter was p=(a,b) multi- estimator u measured output control signal y y decision logic  switching signal – + – + set  =  (p) e p small process likely in  p should use C q, q =  (p) Decision logic: Certainty equivalence inspired unknown parameters p * =(a *, b * ) 2  › [–1,1] £ {–1,1} 8 p=(a, b) 2  › [–1,1] £ {–1,1}

15. Estimator-based supervisor set  =  (p) e p small process likely in  p should use C q, q =  (p) Multi-estimator y p ´ estimate of the process output y that would be correct if the process was N p e p ´ output estimation error that would be small if the process was N p Process is assumed to be in family process in  p, p 2  controller C q, q =  (p) provides adequate performance Decision logic: multi- estimator u measured output control signal y y decision logic  switching signal – + – + Certainty equivalence inspired

16. Estimator-based supervisor set  =  (p) e p small process likely in  p should use C q, q =  (p) Multi-estimator y p ´ estimate of the process output y that would be correct if the process was N p e p ´ output estimation error that would be small if the process was N p Process is assumed to be in family process in  p, p 2  controller C q, q =  (p) provides adequate performance Decision logic: multi- estimator u measured output control signal y y decision logic  switching signal – + – + A stability argument cannot be based on this because typically process in  p ) e p small but not the converse Certainty equivalence inspired

17. Estimator-based supervisor Multi-estimator y p ´ estimate of the process output y that would be correct if the process was N p e p ´ output estimation error that would be small if the process was N p Process is assumed to be in family process in  p, p 2  controller C q, q =  (p) provides adequate performance Decision logic: overall state is small e p small overall system is detectable through e p multi- estimator u measured output control signal y y decision logic  switching signal – + – + set  =  (p) Certainty equivalence inspired, but formally justified by detectability detectable means “small e p ) small state”

18. Performance-based supervision Performance monitor:  q ´ measure of the expected performance of controller C q inferred from past data Candidate controllers: Decision logic: performance monitor u measured output control signal y decision logic  switching signal   is acceptable   is unacceptable keep current controller switch to controller C q corresponding to best  q

19. Abstract supervision Estimator and performance-based architectures share the same common architecture process multi- controller multi-est. or perf. monitor decision logic u control signal y  switching signal w measured output In this talk we will focus mostly on an estimator-based supervisor…

20. Abstract supervision process multi- controller multi- estimator decision logic u measured output control signal y  switching signal switched system w Estimator and performance-based architectures share the same common architecture

21. The four basic properties (1-2) : Example #1 decision logic  switching signal Matching property: At least one of the e p is “small” Why? e p* is “small” essentially a requirement on the multi-estimator Detectability property: For each p 2 , the switched system is detectable through e p when  =  (p) index of controller that stabilizes processes in  p Process is detectable means “small e p ) small state” Multi-estimator is

22. Detectability a system is detectable if for every pair eigenvalue/eigenvector ( i,v i ) of A < [ i ] ¸ 0 ) C v i  0 From solution to linear ODEs… (assuming A diagonalizable, otherwise terms in t k e i t appear) 00 y(t) bounded )  i C v i = 0 (for < [ i ] ¸ 0) )  i = 0 (for < [ i ] ¸ 0) ) y(t) bounded for short: pair (A,C) is detectable Lemma: For any detectable system:y(t) bounded ) x(t) bounded &y(t) ! 0 ) x(t) ! 0

23. Detectability system is detectable if for every pair eigenvalue/eigenvector ( i,v i ) of A < [ i ] ¸ 0 ) C v i  0 Lemma: For any detectable system:y(t) bounded ) x(t) bounded &y(t) ! 0 ) x(t) ! 0 Lemma: For any detectable system, there exists a matrix K such that is asymptotically stable (all eigenvalues of A – KC with negative real part) Can re-write system as asympt. stable confirms that: y is bounded / ! 0 ) x is bounded / ! 0 (output injection)

24. The four basic properties (1-2) : Example #1 Detectability property: For each p 2 , the switched system is detectable through e p when  =  (p) index of controller that stabilizes processes in  p detectable means “small e p ) small state” Why? consider, e.g., p=(a,b)= (.5,1) ) use controller C 3 : u=– 1.1y ( 3 =  =  (p) ).5 – 1.1 = –.6 < 0 Thus, e p small ) y p small ) y = y p – e p small ) u small etc. Questions: Where we just lucky in getting –.6 < 0 ? NO (why?) Does detectability hold if u=– 1.1y does not stabilize the process (e.g., a * =.5, b * =-1)? YES (why?)

25. The four basic properties (1-2) decision logic  switching signal Matching property: At least one of the e p is “small” Why? process in 9 p * 2  : process in  p* e p* is “small” essentially a requirement on the multi-estimator Detectability property: For each p 2 , the switched system is detectable through e p when  =  (p) index of controller that stabilizes processes in  p essentially a requirement on the candidate controllers This property justifies using the candidate controller that corresponds to a small estimation error. Why? Certainty equivalence stabilization theorem…

26. The four basic properties (3-4) decision logic  switching signal Small error property: There is a parameter “estimate”  1  !   for which e  is “small” compared to any fixed e p and that is consistent with , i.e.,  =  (  ) Non-destabilization property: Detectability is preserved for the time-varying switched system (not just for constant  ) Typically requires some form of “slow switching” Both are essentially (conflicting) properties of the decision logic  (t) can be viewed as current parameter “estimate” controller consistent with parameter “estimate”

27. Decision logic 1.For boundedness one wants e  small for some parameter estimate  consistent with  (i.e.,  =  (  )) 2.To recover the “static” detectability of the time-varying switched system one wants slow switching. “small error” “non-destabilization” These are conflicting requirements:  should follow smallest e p  =  (  ) should not vary decision logic  switching signal

28. Dwell-time switching start wait  D seconds monitoring signals p 2 p 2  measure of the size of e p over a “window” of length 1/ Non-destabilizing property: The minimum interval between consecutive discontinuities of  is  D > 0. (by construction) forgetting factor

29. Small error property Assume  finite and 9 p * 2  : (e.g.,  2 noise and no unmodeled dynamics) · C*· C*< 1   when we select  = p at time t we must have  Two possible cases: 1.Switching will stop in finite time T at some p 2  :

30. Small error property Assume  finite and 9 p * 2  : (e.g.,  2 noise and no unmodeled dynamics) · C*· C* < 1   when we select  = p at time t we must have  Two possible cases: 2.After some finite time T switching will occur only among elements of a subset  * of , each appearing in  infinitely many times:

31. Small error property Assume  finite and 9 p * 2  : (e.g.,  2 noise and no unmodeled dynamics)   when we select  = p at time t we must have  Small error property: (  2 case) Assume that  is a finite set. If 9 p * 2  for which then at least one error  2 “switched” error will be  2

32. Implementation issues start wait  D seconds monitoring signals How to efficiently compute a large number of monitoring signals? Example #1: Multi-estimator   by linearity we can generate as many errors as we want with a 2- dim. systems input is linear comb. of y and u state-sharing

33. Implementation issues start wait  D seconds monitoring signals How to efficiently compute a large number of monitoring signals? Example #1: Multi-estimator 1. we can generate as many monitoring signals as we want with a (2+6)-dim. system  2. finding  is really an optimization state-sharing

34. Implementation issues start wait  D seconds When  is a continuum (or very large), it may be issues with respect to the optimization for  Things are easy, e.g., 1.  has a small number of elements 2.model is linearly parameterized on p (leads to quadratic optimization) 3.there are closed form solutions (e.g.,  p polynomial on p) 4.  p is convex on p usual requirement in adaptive control results still hold if there exists a computational delay  C in performing the optimization, i.e. monitoring signals

35. The four basic properties decision logic  switching signal Matching property: At least one of the e p is “small” Detectability property: For each p 2 , the switched system is detectable through e p when  =  (p) index of controller that stabilizes processes in  p Small error property: There is a parameter “estimate”  1  !   for which e  is “small” compared to any fixed e p and that is consistent with , i.e.,  =  (  ) Non-destabilization property: Detectability is preserved for the time-varying switched system (not just for constant  )  (t) can be viewed as current parameter “estimate” controller consistent with parameter “estimate”

36. Analysis outline (linear case, w = 0) decision logic  switching signal 1stby the Matching property: 9 p * 2   such that e p* is “small” 2ndby the Small error property: 9  such that  =  (  ) and e  is “small” (when compared with e p* ) 3rdby the Detectability property: there exist matrices K p such that the matrices A q – K p C p, q =  (p) are asymptotically stable 4ththe switched system can be written as “small” by 2nd step asymptotically stable by non-destabilization property ) x is small (and converges to zero if, e.g., e  2  2 ) injected system

37. Example #2: One-link flexible manipulator y(x, t) x mtmt T IHIH  mass at the tip torque applied at the base axis’s inertia (.023) transversal slice’s inertia beam’s elasticity deviation with respect to rigid body beam’s mass density (.68Kg total mass) beam’s length (113 cm) PDE (small bending):Boundary conditions:

38. Example #2: One-link flexible manipulator y(x, t) x mtmt IHIH  mass at the tip deviation with respect to rigid body Series expansion and truncation: eigenfunctions of the beam T torque applied at the base axis’s inertia (.023)

39. Example #2: One-link flexible manipulator y(x, t) x mtmt IHIH  mass at the tip Control measurements: ´ base angle ´ base angular velocity ´ tip position ´ bending at position x sg (measured by a strain gauge attached to the beam at position x sg ) Assumed not known a priori: m t 2 [0,.1Kg] x sg 2 [40cm, 60cm] T torque applied at the base axis’s inertia (.023)

40. Example #2: One-link flexible manipulator u torque transfer functions as m t ranges over [0,.1Kg] and x sg ranges over [40cm, 60cm]

41. Example #2: One-link flexible manipulator Class of admissible processes: u torque unknown parameter p › (m t, x sg ) parameter set: grid of 18 points in [0,.1] £ [40, 60]  p ´ family around a nominal transfer function corresponding to parameters p › (m t, x sg ) For this problem it is not possible to write the coefficients of the nominal transfer functions as a function of the parameters because these coefficients are the solutions to transcendental equations that must be computed numerically. Family of candidate controllers: 18 controllers designed using LQR/LQE, one for each nominal model

42. Example #2: One-link flexible manipulator 0510152025 -3 -2 0 1 2 3 t tip position torque set point 0510152025 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 t 051015202530 0 2 4 6 8 10 12 14 16 18 t mtmt x sg 

43. Example #2: One-link flexible manipulator (open-loop)

44. Example #2: One-link flexible manipulator (closed-loop with fixed controller)

45. Example #2: One-link flexible manipulator (closed-loop with supervisory control)

46.   C Doyle, Francis, Tannenbaum, Feedback Control Theory, 1992 Example #3: Uncertain gain The maximum gain margin achievable by a single linear time-invariant controller is 4 1 · k < 4

47.   multi- controller Example #3: Uncertain gain 1 · k < 40

48. Example #3: Uncertain gain outputreferencetrue parameter valuemonitoring signals

49. Example #3: 2-dim SISO linear process Class of admissible processes: nominal transfer function nonlinear parameterized on p Any re-parameterization that makes the coefficients of the transfer function lie in a convex set will introduce an unstable zero-pole cancellation unstable zero-pole cancellations –1 +1 But the multi-estimator is still separable and state-sharing can be used …

50. Example #3: 2-dim SISO linear process (without noise) outputreferencetrue parameter value

51. Example #3: 2-dim SISO linear process (with noise) outputreferencetrue parameter value

52. Example #3: Disturbance Rejection (rejection of one sinusoid) outputfrequency estimatedisturbance (unknown frequency)

53. Example #3: Disturbance Rejection (rejection of two sinusoids) output (unknown frequency) frequency estimatedisturbance

54. Example #3: Disturbance Rejection (rejection of a square wave) output (unknown frequency) frequency estimatedisturbance

55. Next lecture… Nonlinear supervisory control Examples