1. 1 Controllability of feedback systems Sigurd Skogestad Department of Chemical Engineering Norwegian University of Science and Tecnology (NTNU) Trondheim, Norway Workshop: Modelling of astrocyte function June 8, 2006, University of Oslo

2. 2 NTNU, Trondheim

3. 3 Motivation I have co-authored a book: ”Multivariable feedback control – analysis and design” (Wiley, 1996, 2005) –What parts could be useful for system biology? Control as a field is closely related to systems theory Here: Focus on the use of negative feedback Some other areas where control may contribute (Not covered): –Identification of dynamic models from data (not in my book anyway) –Model reduction –Nonlinear control (also not in my book) –Discrete event and hybrid systems (also not in my book)

4. 4 Outline 1.Overview of control theory and concepts 2.Feedback Positive and negative feedforward Simple example: Feedforward vs. feedback Problem feedback: (Effective) Time delay Cascade control 3.Control hierarchies Time scale separation Controlled variables Self-optimizing control 4.Summary and concluding remarks

5. 5 Overview of Control theory Classical feedback control (1930-1960) (Bode): –Single-loop (SISO) feedback control –Transfer functions, Frequency analysis (Bode-plot) –Fundamental feedback limitations (waterbed). Focus on robustness Optimal control (1960-1980) (Kalman): –Optimal design of Multivariable (MIMO) controllers –Model-based ”feedforward” thinking; no robustness guarantees (LQG) –State-space formulation (A,B,C,D); Advanced mathematical tools (LQG) Robust control (1980-1995) (Zames, J.C. Doyle) –Combine classical and optimal control –Optimal design of controllers with guaranteed robustness (H ∞ ) Nonlinear control (1950 - ) – ”Feedforward thinking”, mostly mechanical systems Adaptive control (1970-1985) (Åstrøm) Discrete event and hybrid systems –Automata theory –Computer science

6. 6 Important system theoretic / control concepts Cause-effect relationship Classification of variables: –”Causes” (external independent variables): Disturbances (d) and inputs (u) –”Effects” (dependent variables): Internal states (x) and outputs (y) –Parameters (p): Internal model variables – give model uncertainty Linear system: Parameter changes can generate instability but not disturbances Typical state-space model used in control: Plant (uncontrolled system) Disturbance (d) Input (u)Output (y)

7. 7 Linearized model descriptions (useful for analysis and controller design) State space realization Transfer function realization (input-output model) Frequency analysis

8. 8 Control Active adjustment of inputs (available degrees of freedom, u) to achieve the operational objectives of the system Most cases: Acceptable operation = ”Output (y) close to desired setpoint (y s )” Plant (uncontrolled system) Disturbance (d) Input (u)Output (y) time ysys

9. 9 Control theory Design

10. 10 Control theory concepts Steady-state = Equilibrium point = DC Poles/modes/eigenvalues: Characteristic of dynamic response of system associated with system states – Model (= our realization of the real system): (state) Controllability (state) Observability Zeros: Characteristic of dynamic response of system inverse Stability –System maintains equilibrium point (Lyapunov stability) –Linear system (local behaviour): Stable = eigenvalue/pole in left half plane (LHP) Unstable = eigenvalue/pole(s) in right half plane (RHP)

11. 11 Robustness Insensitivity of system behavior to uncertainty (parameter variations) Most real biological systems are robust Robustness generally requires negative feedback Measures and tools for robustness analysis –Uncertainty representation –Stability margins –Positivity, small gain theorem –Structured singular value (Doyle)

12. 12 (Input-Output) Controllability Input-output: Is the system inherently controllable? First requirement: Stabilization ‘‘ Natural” (open-loop = without control) unstable system: –Can not be stabilized with feedforward control –Can only be stabilized by feedback –Must require that any unstable modes are detectable and stabilizable –Detectable: Unstable modes observable in the outputs –Stabilizable: Unstable modes controllable by the inputs Next requirement: performance

13. 13 Controllability limitations (performance) Serious limitations: 1.(Effective) Time delay in direct path from input to output –Want input close to output 2. Unstable inverse (RHP-zeros) –Limits input-output controllability because system inverse is unstable Other limitations: 3.Large disturbances 4.Small process gain (inputs weakly affect outputs) 5. Unstable system (even if unstable modes are detectable and stabilizable) 6. Nonlinearity

14. 14 Fundamental limitations Waterbed effect (”no free lunch”) for feedback control: Does NOT apply to first-order system

15. 15 FEEDBACK

16. 16 Positive and negative feedback in biological systems (from Wipikedia) Positive feedback amplifies possibilities of divergences (evolution, change of goals); it is the condition to change, evolution, growth; it gives the system the ability to access new points of equilibrium. –For example, in an organism, most positive feedbacks provide for fast autoexcitation of elements of endocrine and nervous systems (in particular, in stress responses conditions) and play a key role in regulation of morphogenesis, growth, and development of organs, all processes which are in essence a rapid escape from the initial state. Homeostasis is especially visible in the nervous and endocrine systems when considered at organism level. Negative feedback helps to maintain stability in a system in spite of external changes. It is related to homeostasis. –Negative feedback combined with time delay can give instability: A well known example in ecology is the oscillation of the population of snowshoe hares due to predation from lynxes. –Le Chateliers principle for stable equilibrium (OK for main effect): Systems responds by counteracting effect of disturbance

17. 17 Typical chemical plant: Tennessee Eastman process Recycle and natural phenomena give positive feedback

18. 18 Control uses negative feedback XC x As xAxA FAFA

19. 19 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) Acceptable operation = ”Output (y) close to desired setpoint (y s )” Control: Use input (u) to counteract effect of disturbance (d) on y Two main principles: Feedforward control (measure d, predict and correct ahead) (Negative) Feedback control (measure y and correct afterwards) CONTROL

20. 20 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) No control: Output (y) drifts away from setpoint (y s )

21. 21 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) Feedforward control: Measure d, predict and correct (ahead) Main problem: Offset due to model error FF-Controller≈ Plant model -1 Setpoint (y s ) Predict Offset

22. 22 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) FB Controller ≈ High gain Setpoint (y s ) Feedback control: Measure y, compare and correct (afterwards) Main problem: Potential instability (if we increase gain to improve performance) e

23. 23 Example G GdGd u d y Plant (uncontrolled system) 1 k=10 time 25

24. 24 G GdGd u d y

25. 25 Model-based control = Feedforward (FF) control G GdGd u d y ”Perfect” feedforward control: u = - G -1 G d d Our case: G=G d → Use u = -d

26. 26 G GdGd u d y Feedforward control: Nominal (perfect model)

27. 27 G GdGd u d y Feedforward: sensitive to gain error

28. 28 G GdGd u d y Feedforward: sensitive to time constant error

29. 29 G GdGd u d y Feedforward: Moderate sensitive to delay (in G or G d )

30. 30 Feedback (FB) control G GdGd u d y Feedback controller ysys e=y s -y Negative feedback: u=f(e) ”Counteract error in y by change in u’’

31. 31 Feedback (FB) control Feedback controller e=y s -y u Simplest: On/off-controller u varies between u min (off) and u max (on) Problem: Continous cycling

32. 32 Feedback (FB) control Feedback controller e=y s -y u Most common in industrial systems: PI-controller

33. 33 G GdGd u d y Back to the example

34. 34 Feedback PI-control: Nominal case d G GdGd u y C ysys e Input u Output y Feedback generates inverse! Resulting output

35. 35 G GdGd u d y C ysys e Integral (I) action removes offset offset

36. 36 G GdGd u d y C ysys e Feedback PI control: insensitive to gain error

37. 37 Feedback: insenstive to time constant error G GdGd u d y C ysys e

38. 38 Feedback control: sensitive to time delay G GdGd u d y C ysys e

39. 39 Summary example Feedforward control is NOT ROBUST (it is sensitive to plant changes, e.g. in gain and time constant) Feedforward control: gradual performance degradation Feedback control is ROBUST (it is insensitive to plant changes, e.g. in gain and time constant) Feedback control: “sudden” performance degradation (instability) Instability occurs if we over-react (loop gain is too large compared to effective time delay). Feedback control: Changes system dynamics (eigenvalues) Example was for single input - single output (SISO) case Differences may be more striking in multivariable (MIMO) case

40. 40 Feedback is an amazingly powerful tool

41. 41 Stabilization requires feedback Input u Output y

42. 42 Conclusion: Why feedback? (and not feedforward control) Simple: High gain feedback! Counteract unmeasured disturbances Reduce effect of changes / uncertainty (robustness) Change system dynamics (including stabilization) Linearize the behavior No explicit model required MAIN PROBLEM Can not have fast/tight control of plant with (effective) time delay Potential instability (may occur “suddenly”) with time delay

43. 43 Problem feedback: Effective delay θ Effective delay PI-control = ”original delay” + ”inverse response” + ”half of second time constant” + ”all smaller time constants” Example: Series-cascade of five first-order systems u y y(t) t u

44. 44 PI-control with single measurement

45. 45 Improve control? Some improvement possible with more complex controller –For example, add derivative action (PID-controller) –May reduce θ eff from 3.5 s to 2.5 s –Problem: Sensitive to measurement noise –Does not remove the fundamental limitation (recall waterbed) Add extra measurement and introduce local control –May remove the fundamental waterbed limitation Waterbed limitation does not apply to first-order system –Cascade

46. 46 Add four local P-controllers (inner cascades)

47. 47 CONTROL HIEARCHIES

48. 48 Hierarchical structure in chemical industry

49. 49 Engineering systems Most (all?) large-scale engineering systems are controlled using hierarchies of quite simple single-loop controllers –Commercial aircraft –Large-scale chemical plant (refinery) 1000’s of loops Simple components: on-off + P-control + PI-control + nonlinear fixes + some feedforward Same in biological systems

50. 50 Hierarchical structure Brain Local control in cells Organs

51. 51 Alan Foss (“Critique of chemical process control theory”, AIChE Journal,1973): The central issue to be resolved... is the determination of control system structure. Which variables should be measured, which inputs should be manipulated and which links should be made between the two sets?

52. 52 Alternatives structures for optimizing control What should we control? Hierarchical Centralized Brain Cells

53. 53 Alternatives structures for optimizing control Hierarchical Centralized What should we control at constant setpoint? (Control theory has relatively little to offer) Control theory has a lot to offer

54. 54 Self-optimizing Control – Marathon Optimal operation of Marathon runner, J=T –Any self-optimizing variable c (to control at constant setpoint)?

55. 55 Self-optimizing Control – Marathon Optimal operation of Marathon runner, J=T –Any self-optimizing variable c (to control at constant setpoint)? c 1 = distance to leader of race c 2 = speed c 3 = heart rate c 4 = level of lactate in muscles

56. 56 Self-optimizing control (Skogestad, 2000) Self-optimizing control is achieved when a constant setpoint policy results in an acceptable loss L (without the need to reoptimize when disturbances occur) Loss L = J - J opt (d) J = cost (overall objective to be minimized)

57. 57 Good candidate controlled variables c (for self-optimizing control) Requirements: The optimal value of c should be insensitive to disturbances c should be easy to measure and control The value of c should be sensitive to changes in the degrees of freedom (Equivalently, J as a function of c should be flat) For cases with more than one unconstrained degrees of freedom, the selected controlled variables should be independent. Singular value rule (Skogestad and Postlethwaite, 1996): Look for variables that maximize the minimum singular value of the appropriately scaled steady-state gain matrix G from u to c

58. 58 Biological systems ”Self-optimizing” controlled variables have presumably been found by natural selection Need to do ”reverse engineering” : –Find the controlled variables used in nature –From this identify what overall objective the biological system has been attempting to optimize

59. 59 Conclusion Negative Feedback is an extremely powerful tool Complex systems can be controlled by hierarchies (cascades) of single- input-single-output (SISO) control loops Control extra local variables (secondary outputs) to avoid fundamental feedback control limitations Control the right variables (primary outputs) to achieve ”self- optimizing control”

60. 60 Relationship to system biology: What can the control field contribute? Advanced methods for model-based centralized controller design –Probably of minor interest in biological systems –Unlikely that nature has developed many multivariable control solutions Understanding of feedback systems –Analysis tools –Robustness –Cascade control –Same inherent limitations apply in biological systems Understanding and design of hierarchical control systems –Important both in engineering and biological systems –BUT: Underdeveloped area in control ”Large scale control systems community”: Out of touch with reality