1. AFOSR Workshop, Dayton, OH, August 4-6, 1999 1 M. G. Safonov Robust Control, Feedback and Learning Michael G. Safonov University of Southern California

2. AFOSR Workshop, Dayton, OH, August 4-6, 1999 2 M. G. Safonov The Problem Control theory relies too much on models: –Step 1: System ID to obtain model –Step 2: Design controller for model System ID relies too much on assumptions: –Assume noise probabilities & model structure –Optimize model fit to data & assumptions Models & assumptions can be wrong Need theory that responds appropriately when observed data falsifies prior belief

3. AFOSR Workshop, Dayton, OH, August 4-6, 1999 3 M. G. Safonov The Approach Let the data speak... Unfalsify (validate) models and controllers against hard criteria: –Choose criteria expressible directly in terms of observed data (sensor outputs, actuator inputs) –Avoid criteria that that rely on “noise model” and other prior beliefs

4. AFOSR Workshop, Dayton, OH, August 4-6, 1999 4 M. G. Safonov Unfalsification Algorithm Brugarolas & Safonov, CCA/CACSD ‘99 UNFALSIFIED CONTROL: The ability of each candidate controller to meet the performance goal is treated as a hypothesis to be tested directly against evolving real-time measurement data. The controller need not be in the loop to test the hypothesis.

5. AFOSR Workshop, Dayton, OH, August 4-6, 1999 5 M. G. Safonov Plato introspective learning Data approximates Unobserved TRUTH Reality is an ideal, observable only through noisy sensors. ‘Probabilistic Estimation’ Two views on how we learn: Introspection vs. Observation vs. Galileo open-eyed learning MODELS approximate Observed Data Reality is what we observe. ‘Curve-Fitting’

6. AFOSR Workshop, Dayton, OH, August 4-6, 1999 6 M. G. Safonov Traditional control theory (‘Platonic’): –contains many assumptions about the plant. –some assumptions are unrealistic. Unfalsified control theory (‘Galilean’): –eliminates hypotheses that are not consistent with evolving experiment data.

7. AFOSR Workshop, Dayton, OH, August 4-6, 1999 7 M. G. Safonov Impact on SYSID Statisticians treat prior probabilities as part of model to be validated/unfalsified –validation via one-  ‘confidence intervals’ –‘Platonic’ probability becomes ‘Galilean’ Ljung (e.g., CDC ’97) proposed ‘confidence interval’ reinterpretation of SYSID –model validation/unfalsification

8. AFOSR Workshop, Dayton, OH, August 4-6, 1999 8 M. G. Safonov Two Views of SYSID: Curve-Fitting vs. Estimation CURVE FIT: (Galilean) Given data (y i,u i ), i=1,2,... (y 1,u 1 ) (y 2,u 2 ) (y 3,u 3 ) (y 4,u 4 ) (y 5,u 5 ) (y 6,u 6 ) (y 7,u 7 ) height 2  { BAYESIAN ESTIMATE (Platonic): Given data (y i,u i ), i=1,2,... ‘noise’ v=N(0,  ) ),ba (y |x,maxprob subject to prior beliefs

9. AFOSR Workshop, Dayton, OH, August 4-6, 1999 9 M. G. Safonov CURVE FIT (Galilean): The Galilean may reject model if 2/3 of future data fails to fall in his prior 2  confidence bound If the fit is bad, do you reject the data or the model? (y 1,u 1 ) (y 2,u 2 ) (y 3,u 3 ) (y 4,u 4 ) (y 5,u 5 ) (y 6,u 6 ) (y 7,u 7 ) height 2  { BAYESIAN ESTIMATE (Platonic): The Platonist may reject data if 2/3 of future data fails to fall in his prior 2  confidence bound

10. AFOSR Workshop, Dayton, OH, August 4-6, 1999 10 M. G. Safonov candidate controllers FALSIFIED COMPUTER SIEVE LEARNING FEEDBACK LOOPS K M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996. Unfalsified Controllers K evolving I/O data given M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996. How about Validation-Based Direct Controller ID?

11. AFOSR Workshop, Dayton, OH, August 4-6, 1999 11 M. G. Safonov Canonical Representation Signals: manifest control signal measurement signal latent reference signal performance signal controller signals controller parameters Brugarolas & Safonov, CCA/CACSD ‘99

12. AFOSR Workshop, Dayton, OH, August 4-6, 1999 12 M. G. Safonov Formulation in Truncated Space Observations operator maps input-output signals to measurement signals. Truncated Space results from applying the observations operator to a signal space. Basic sets: goals hypothesis hypotheses data

13. AFOSR Workshop, Dayton, OH, August 4-6, 1999 13 M. G. Safonov Formulation in Truncated Space, cont. Problem 1 (Truncated Space Unfalsified Control Problem): Given a performance specification (goal set), a set of candidate controllers (hypotheses), and experimental data then determine the subset of candidate controllers (hypothesis) which is not falsified.

14. AFOSR Workshop, Dayton, OH, August 4-6, 1999 14 M. G. Safonov Main Results Definition (Data-hypothesis consistency): Given a truncated space unfalsified control problem, we say that a hypothesis is consistent with the data if Theorem 1 (Truncated space unfalsified control): Given truncated space unfalsified control problem, then a candidate control law (hypothesis) consistent with the experimental data is unfalsified by data if and only if

15. AFOSR Workshop, Dayton, OH, August 4-6, 1999 15 M. G. Safonov ————— Jun & Safonov, CCA/CACSD ‘99 Example: Adaptive PID Performance Spec:

16. AFOSR Workshop, Dayton, OH, August 4-6, 1999 16 M. G. Safonov Adaptation Algorithm Procedure (at each time t = k  t) : 1. Measure u(k  t) and y(k  t). 2. For each calculate and calculate if then delete the controller index element i from I ; else continue. 3. If the set I is empty, terminate; else set the current to and increment time.

17. AFOSR Workshop, Dayton, OH, August 4-6, 1999 17 M. G. Safonov Controller Parameter Adaptation The i-th candidate PID controller K i is unfalsified if and only if

18. AFOSR Workshop, Dayton, OH, August 4-6, 1999 18 M. G. Safonov Simulation Jun & Safonov, CCA/CACSD ‘99

19. AFOSR Workshop, Dayton, OH, August 4-6, 1999 19 M. G. Safonov Simulation Jun & Safonov, CCA/CACSD ‘99 control input u(t) proportional gain k P (t) integral gain k I (t) derivative gain k D (t), plant output y(t) Evolution of unfalsified setTime responses Tsao & Safonov, IEEE Trans, AC-42, 1997.

20. AFOSR Workshop, Dayton, OH, August 4-6, 1999 20 M. G. Safonov Example: Missile Autopilot Learns control gains Adapts quickly to compensate for damage & failures Superior performance Brugarolas, Fromion & Safonov, ACC ’98 Specified target response bound Actual response Commanded response Brugarolas, Fromion & Safonov, ACC ’98 Unfalsified adaptive missile autopilot: discovers stabilizing control gains as it flies, nearly instantaneously maintains precise sure-footed control

21. AFOSR Workshop, Dayton, OH, August 4-6, 1999 21 M. G. Safonov Robot Example: Tsao & Safonov, CCA/CACSC’99 Tsao & Safonov, CCA/CACSD ’99

22. AFOSR Workshop, Dayton, OH, August 4-6, 1999 22 M. G. Safonov Other People’s Successes with Unfalsified Control Emmanuel Collins et al. (Weigh Belt Feeder adaptive PID tuning, CDC99) Kosut (Semiconductor Mfg. Process run-to- run tuning, CDC98) Woodley, How & Kosut (ECP Torsional disk control, adaptive tuning, ACC99 … maybe others soon?

23. AFOSR Workshop, Dayton, OH, August 4-6, 1999 23 M. G. Safonov Unfalsified Control  Learning/Adaptation Adaptation = Direct Controller ID Unfalsified control focuses on the knowable –‘ Galilean’ emphasis on observation –precise information in each datum –real-time learning (the essence of feedback) Practical, reliable adaptive feedback control Greater tolerance of evolving uncertainties, failures & battle damage Conclusions

24. AFOSR Workshop, Dayton, OH, August 4-6, 1999 24 M. G. Safonov Acknowledgment Bob Kosut’s mid-1980’s work on time-domain model validation and identification for control played a key role in laying the foundations of this work, as did later contributions of Jim Krause, Pramod Khargonekar, Carl Nett, Kamashwar Poolla, Roy Smith and many others who have advanced the use of validation methods in control-oriented identification. Tom Mitchell’s early 1980’s “candidate elimination algorithm’’ for machine learning is closely related to the unfalsified control methods presented here. And of course, none of this would have been possible without the superb graduate education that I received at MIT so many years ago under the guidance of first Jan Willems and later Michael Athans.

25. AFOSR Workshop, Dayton, OH, August 4-6, 1999 25 M. G. Safonov References Also, see webpage http://routh.usc.edu