1. 1 TUTORIAL on Networked Control Systems with Delay Cicsyn2010 2nd International Conference on Computational Intelligence, Communication Systems and Networks Liverpool, UK, July 29th, 2010 Vasilis Tsoulkas: Center for Security Studies, Athens, Greece & Dept. of Mathematics, University of Athens Research Group: - Pantelous Athanasios., University of Liverpool, - Dritsas Leonidas., Hellenic Airforce Academy - Halikias George, City University, London, UK.

2. 2 Contents 1. Introduction – General Features 2.NCS Modeling - the issue of network induced delays 3.Discretization of NCS dynamics 4.Decomposing the Uncertain Delay (nominal and uncertain parts) and the NCS dynamics 5.Robust Stability Analysis based on the augmented closed-loop vector (ξ) 6.Design of a Simple Output Tracking Controller 7.Investigation of Robust Tracking Performance via Simulation - Numerical Examples for Networked Stable and Unstable systems 8.Conclusions & Topics for further study

3. 3 Schematics of Networked Control Systems Networked control systems (NCSs) are spatially distributed systems for which the communication between sensors, actuators, and controllers is supported by a shared communication network. Networked control systems (NCSs) are spatially distributed systems for which the communication between sensors, actuators, and controllers is supported by a shared communication network. Hespanha et al.: Survey of Recent Results in Networked Control Systems (Proceedings of the IEEE, Vol. 95, No. 1, January 2007)

4. Motivation & Some Benefits Easy and low cost installation, wiring, maintenance, configuration Easy and low cost installation, wiring, maintenance, configuration Distributed Controllers and Plant with low cost distributed sensors and actuators are all coupled over the same Real Time communications network Distributed Controllers and Plant with low cost distributed sensors and actuators are all coupled over the same Real Time communications network The distributed nature of elements offers great flexibility of architectures. The distributed nature of elements offers great flexibility of architectures. Applicable in a wide variety of fields such as: Remote surgery, mobile sensor networks, UAVs, Space tele- operations and Robotics. Applicable in a wide variety of fields such as: Remote surgery, mobile sensor networks, UAVs, Space tele- operations and Robotics. 4

5. 5 Distributed Networked System

6. 6 Control networks are indicated by solid lines, and diagnostics networks are indicated by dashed lines.

7. 7 7 1. Introduction Feedback control systems wherein the control loops are closed through a real-time network are called Networked Control Systems (NCSs) Defining feature of NCS: Information (reference input, plant output, control input, etc.) is exchanged using a network among control system components (sensors, controllers, actuators, etc.).

8. 8 1. Introduction Νetwork Induced Delays Information flow in the control loop is delayed due to Information flow in the control loop is delayed due to –buffering, –access contention (the time a node waits until it gets access to the network), –computation delay (assume absorbed into {τ ca (k)} ) –propagation (transmission) delays. – Network-induced delays in NCS appear in the information flow between (k denotes the dependence on the k th sampling period). A).The sensor and the controller {τ sc (k)}, (controller receives outdated information about process behavior) A).The sensor and the controller {τ sc (k)}, (controller receives outdated information about process behavior) B).The controller and the actuator {τ ca (k)}, (control action cannot be applied on time and the controller does not know the exact instance the calculated control signal will be received by the actuator) B).The controller and the actuator {τ ca (k)}, (control action cannot be applied on time and the controller does not know the exact instance the calculated control signal will be received by the actuator)

9. 9 1.Introduction Νetwork Induced Delays When a static linear time invariant controller is employed, can lump the delays τ sc (k), τ ca (k), into τ k = τ sc (k)+τ ca (k). Network-induced delays in NCS between the sensor and the controller {τ sc (k)}, and between the controller and the actuator {τ ca (k)}, (k denotes the dependence on the k th sampling period).

10. 10 1.Introduction Tracking Control Design for NCS The Usual Approach for NCS Analysis & Design: – design a controller ignoring the network, then – analyze stability, performance and robustness with respect to the effects of network-delays and scheduling policy…(usually via the selection of an appropriate scheduling protocol). The issue of Tracking Control over Networks has not been adequately met – very limited published work on NCS Tracking !!! – the majority of NCS publications concerns regulation, (design a controller which brings the output/state to 0 ) – many results on tracking for Time Delayed Systems (TDS) but cannot be applied as is to NCS due to the Network- centric nature of NCS e.g. special nature of delays in NCS special nature of delays in NCS the fundamental issue of Packet Loss/Drops the fundamental issue of Packet Loss/Drops Scheduling, Quality of Service, Middleware Scheduling, Quality of Service, Middleware

11. 11 1.Introduction Tracking Control Design for NCS Concerning NCS Robust Tracking Performance… Concerning NCS Robust Tracking Performance… – only preliminary results - no strict mathematical proofs – yet…useful lessons learned through extensive simulations on S.I.S.O systems – we investigate both constant unknown or time-varying uncertain delays with known bounds – we do not take into account the network delays in the tracking controller design process… – a posteriori analysis of stability, performance and conservatism of results – we do not take into account packet drops – Analysis & Synthesis in the continuous time domain – No need to assume knowledge of the P.D.Fs (not a stochastic approach)

12. 12 2. NCS Modeling 2. NCS Modeling NCS with network-induced delays in the actuation and sensor path Assumptions made … the dynamics of the NCS under investigation is a combination of a continuous–time LTI plant with a discrete–time controller. Time Invariant controller can lump τ sc (k), τ ca (k), into τ k = τ sc (k)+τ ca (k). Single source of uncertainty and performance degradation the lumped transmission delay τ k. No plant uncertainties or nonlinearities - No packet drops

13. 13 2. NCS Modeling - Assumptions In Practice… the dynamics of the NCS under investigation is a combination of a continuous–time uncertain/nonlinear plant with a discrete–time (sampled-data) controller. the dynamics of the NCS under investigation is a combination of a continuous–time uncertain/nonlinear plant with a discrete–time (sampled-data) controller. The sampler is time-driven, whereas both controller and actuator are event-driven, (=they update their outputs as soon as they receive a new sample). The sampler is time-driven, whereas both controller and actuator are event-driven, (=they update their outputs as soon as they receive a new sample). Some packets are lost or intentionally dropped (contain obsolete/useless info) Some packets are lost or intentionally dropped (contain obsolete/useless info)

14. 14 The delays τ k sc, τ k ca, τ k < h τ k sc = τ sc (k) is the delay experienced by a state or output sample x(kh), y(kh), sampled at time instance kh and presented –after a delay τ k sc to the event–driven remote controller for control computation purposes. τ k sc = τ sc (k) is the delay experienced by a state or output sample x(kh), y(kh), sampled at time instance kh and presented –after a delay τ k sc to the event–driven remote controller for control computation purposes. τ k ca =τ ca (k) is the delay experienced by the control–action, computed immediately after its reception at time instance kh+ τ k sc until it is transmitted via the network to the Z.O.H (and finally presented to the event–driven actuator). τ k ca =τ ca (k) is the delay experienced by the control–action, computed immediately after its reception at time instance kh+ τ k sc until it is transmitted via the network to the Z.O.H (and finally presented to the event–driven actuator). The computation delay is absorbed into τ k ca The computation delay is absorbed into τ k ca

15. 15 τ k : Total delay within the k th sampling period, τ k : Total delay within the k th sampling period, i.e. the time from the instant when the sampling node samples sensor data from the plant to the instant when actuators exert a control action –whose computation was based on this sample– to the plant. τ k = τ k sc + τ k ca τ k = τ k sc + τ k ca (since a static time invariant control law is employed) Known Bounds: 0 τ min < τ k < τ max h The delays τ k sc, τ k ca, τ k < h

16. 16 NCS Timing Diagram (τ k < h) for short (τ k < h) + bounded delay 0 τ min < τ k < τ max h

17. 17 2. NCS Modeling Difficulties in case of Discrete Sampled Data Controller û(t) is the most recent control action presented to the event–driven actuator at the time instance t within a sampling period [kh, kh + h) & can take two values û k or û k-1 û(t) experiences a jump at the uncertain or unknown time instance kh+ τ k, changing from û k-1 into û k (uncertain actuation instance) Very Complicated Dynamics Impulse Delayed Systems, Asynchronous Dynamical Systems, Hybrid Systems, etc even for the regulation case (r=0)

18. 18 2. NCS Modeling - the issue of network-induced delays NCS Timing Diagram form Zhang & Branicky paper (IEEE Control Systems Magazine, Febr.2001). Hence (unless τ k is constant) it is not possible to treat the ensuing NCS in a standard sampled data or time-delayed setting. Instead a hybrid setup should rather be used, as for example the one presented in P. Naghshtabrizi and J. P. Hespanha, Stability of network control systems with variable sampling and delays in Proc. of the 44th Annual Allerton Conf. on Communication, Control, and Computing, 2006. Possible misconceptions if symbols are not adequately clarified… Authors clarify that the confusing symbol u(kh) denotes the actuation that takes place at kh+ τ k and its value is u(kh) = -Kx(kh)

19. 19 CONTENTS 1. Introduction – General Features 2.NCS Modeling - the issue of network-induced delays 3.Discretization of NCS dynamics 4.Decomposing the Uncertain Delay (nominal and uncertain parts) 5.Robust Stability Analysis based on the closed-loop augmented vector (ξ) 6.Design of A Simple Output Tracking Controller 7.Investigation of Robust Tracking Performance via Simulation - Numerical Examples for Networked Stable and Unstable systems 8.Conclusions & Topics for further study

20. 20 3. Descretization of NCS state equation with small delay τ k < h x k = x(kh) x k+1 = Φ x k + Γ 0 (τ k ) û k + Γ 1 (τ k ) û k-1 (Σ1) notation x k, x k-1,… denotes the values x(kh), x(kh-h), … of the periodically sampled discrete–time signal coming out of the sampler. The same notation for y k, y k-1,… notation x k, x k-1,… denotes the values x(kh), x(kh-h), … of the periodically sampled discrete–time signal coming out of the sampler. The same notation for y k, y k-1,… We keep the hat notation for û k, û k-1 as a reminder of the asynchronous, (jump) nature of these signals. We keep the hat notation for û k, û k-1 as a reminder of the asynchronous, (jump) nature of these signals. Õ n … is an n-column zero vector, I n is the n x n identity matrix, 0 n is the n x n zero matrix. Õ n … is an n-column zero vector, I n is the n x n identity matrix, 0 n is the n x n zero matrix. M T is the transpose of a matrix. M > 0 ( 0 (< 0) means that M is positive (negative) definite.

21. 21 3. Discretization of state equation dynamics of NCS (Comments) x k+1 = Φ x k + Γ 0 (τ k ) û k + Γ 1 (τ k ) û k-1 (Σ1) Exact Discretization between equidistant sampling instances finite dimensional dynamics… Exact Discretization between equidistant sampling instances finite dimensional dynamics… The uncertain time varying delay τ k can still take any (out of infinite) values within the allowable interval The uncertain time varying delay τ k can still take any (out of infinite) values within the allowable interval the uncertainty of τ k generates an uncertainty in the actuation instance the uncertainty of τ k generates an uncertainty in the actuation instance System matrices (Γ 0 (τ k ), Γ 1 (τ k )) are uncertain System matrices (Γ 0 (τ k ), Γ 1 (τ k )) are uncertain Presence of a delayed input term û k-1 Presence of a delayed input term û k-1

22. 22 Exact Discretization despite the jump nature of Exact Discretization despite the jump nature of û(t) x k = x(kh), Φ = exp(A c h) λ λ

23. 23 Exact Discretization despite the jump nature of Exact Discretization despite the jump nature of û(t) x k = x(kh), Φ = exp(A c h) Similarly from the definition of Γ1, using the same change of variables as previously

24. 24 Notice that : Exact Discretization despite the jump nature of Exact Discretization despite the jump nature of û(t) xk = x(kh), Φ = exp(Ach) (3.A). (3.B).

25. 25 Contents 1.Introduction – General Features 2.NCS Modeling - the issue of network-induced delays 3.Descretization of NCS dynamics equation 4.Decomposing the Uncertain Delay (nominal and uncertain parts) and the NCS dynamics 5.Robust Stability Analysis based on the closed-loop augmented vector (ξ) 6.Design of A Simple Output Tracking Controller 7.Investigation of Robust Tracking Performance via Simulation - Numerical Examples for Networked Stable and Unstable systems 8.Conclusions & Topics for further study

26. 26 4.Decomposing the uncertain delay of the system (into nominal & uncertain part) Examples: τ o = τ min τ o = τ max τ o = τ avg τ o is chosen as constant and known («semi-arbitrary») τ o is chosen as constant and known («semi-arbitrary») Use of Min Max techniques for selection of τ o Use of Min Max techniques for selection of τ o The nominally delayed system, Stability Analysis and Controller Synthesis depend on the (users) choice of τ o The nominally delayed system, Stability Analysis and Controller Synthesis depend on the (users) choice of τ o

27. 27 4.Decomposing the uncertain delay of the system (into nominal & uncertain part)- (4.C).

28. 28 4. Decomposing the uncertain delay of the system (into nominal & uncertain part) - 4.D

29. 29 4. Decomposing the uncertain delay of the system (into nominal & uncertain part)

30. 30Contents 1.Introduction 2.NCS Modeling - the issue of network-induced delays 3.Descretization of NCS dynamics equation 4.Decomposing the Uncertain Delay (nominal and uncertain parts) 5.Robust Stability Analysis based on the augmented closed-loop vector (ξ) 6.Design of Simple Output Tracking Controller 7.Investigate Robust Tracking Performance via Simulation 8.Conclusions & Topics for further study

31. 31 5. Robust Stability Analysis based on the closed-loop vector augmented (ξ) - Closing the loop THE AUGMENTED CLOSED–LOOP STATE VECTOR (ACLSV) ξ THE AUGMENTED CLOSED–LOOP STATE VECTOR (ACLSV) ξ x k+1 = Φ x k + Γ 0 (τ k ) û k + Γ 1 (τ k ) û k-1 x k+1 = Φ x k + Γ 0 (τ k ) û k + Γ 1 (τ k ) û k-1 Static State Feedback (SSF) û k = -K sf x k û k-1 = -K sf x k-1 û k = -K sf x k û k-1 = -K sf x k-1 Closed Loop Dynamics x k+1 = [Φ- Γ 0 (τ k ) K sf ] x k + [- Γ 1 (τ k ) K sf ] x k-1 x k+1 = [Φ- Γ 0 (τ k ) K sf ] x k + [- Γ 1 (τ k ) K sf ] x k-1 only periodically sampled state vector values {x k+1, x k, x k-1 } are present

32. 32 5. Robust Stability Analysis based on the closed-loop vector (ξ) Define the augmented sampled data sampled data closed-loop state vector

33. 33 5. Robust Stability Analysis based on the closed-loop vector (ξ) The above matrix relation is manageable and Robust Control Methods now can be used.

34. 34 Contents 1.Introduction 2.NCS Modeling - the issue of network-induced delays 3.Discretization of NCS dynamics 4.Decomposing the Uncertain Delay (nominal and uncertain parts) and NCS dynamics 5.Robust Stability Analysis based on the augmented closed-loop vector (ξ) 6.Design of Simple Output Tracking Controller 7.Investigate Robust Tracking Performance via Simulation 8.Conclusions & Topics for further study

35. 35 6. Design of Simple (Set Point) Tracking Controllers SPCT = Set Point Tracking Controller(s) SPCT = Set Point Tracking Controller(s) The reference signal to be tracked by the output is (piecewise) constant (a set point) The reference signal to be tracked by the output is (piecewise) constant (a set point) –Assumption:both the plant and the controller under investigation are continuous–time LTI systems –Since the controller is time invariant, can lump the delays τ sc (k), τ ca (k), into τ k = τ sc (k)+τ ca (k). – A naïve tracking controller consists of two parts: Feedback & Feedforward u(t) = -Kx(t)+Fr The feedback part (-Kx(t)) assures closed-loop stability The feedback part (-Kx(t)) assures closed-loop stability The feedforward part (Fr) assures that the static gain is 1 (Stable Transfer Function from r to y) The feedforward part (Fr) assures that the static gain is 1 (Stable Transfer Function from r to y)

36. 36 6. Design of Simple (Set Point) Tracking Controller Suffers from three drawbacks (naïve): the plant must not contain integrators (system matrix A is nonsingular) cannot handle disturbances and/or model uncertainties (it is NOT Robust) Number of inputs Number of outputs (overactuation)

37. 37 Contents 1.Introduction 2.NCS Modeling - the issue of network-induced delays 3.Descretization of NCS dynamics equation 4.Decomposing the Uncertain Delay (nominal and uncertain parts) 5.Robust Stability Analysis based on the closed-loop vector (ξ) 6.Design of Simple Output Tracking Controller 7.Investigate Robust Tracking Performance via Simulation 8.Conclusions & Topics for further study

38. 38 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system A benevolent stable & minimum phase (=zeros in LHP) system with infinite Gain Margin and… Lightly Damped = stable poles close to the Imaginary axis damping ratio is small damped oscillative open-loop behaviour (typical in aerospace and flexible space structure applications) SPTC was designed via LQR with R=1, Q=1000*I 2 u(t) = -30.63 x 1 (t) - 30.63 x 2 (t) + 31.63*r gives perfect tracking in the absence of delays

39. 39 7. Robustness of Tracking Performance Nmerical Example 1: a networked stable & minimum phase system with constant delay constant delay τ k The Networked Version with constant delay τ k τ sc =τ ca = 0.0131 s τ k = τ sc +τ ca =0.0262s Assuming that τ k h this delay corresponds (for the discrete time control case) to a sampling frequency of 38Hz a relatively slow sampling… slow sampling is typical for NCS (fast sampling increases # of packets increases network traffic increases chances for collisions packet loss/drops)

40. 40 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with constant delay τ k The Networked Version with constant delay τ k τ sc =τ ca = 0.0131 s τ k = τ sc +τ ca =0.0262s 7 th order Pade Approximation used in simulations for the constant time-delay Reference Signal(s) r are (piecewise) constant: combination of step functions or square pulse with period slower than the systems time constants Simulation needs time for Instability to occur (see next Figs)

41. 41 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with constant delay

42. 42 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with constant delay

43. 43 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with uncertain (time-varying) delay τ k The Networked Version with uncertain time-varying delay τ k varying between τ min = 0 τ max = 0.0312s < h τ min = 0 and τ max = 0.0312s < h corresponding to a sampling frequency of 32 Hz Implementation used in simulations: τ k = τ o + δ τ unc, |δ| < 1 τ o = τ avg = (τ max + τ min )/2 = 0.0156 s with τ o = τ avg = (τ max + τ min )/2 = 0.0156 s being the mean value (a constant nominal delay) and |δ|<1 being a random variable of uniform distribution.

44. 44 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with uncertain (time-varying) delay 0=τ min τ k τ max = 0.0312s τ k = τ o + δ τ unc, |δ| < 1 τ o = τ avg = (τ max + τ min )/2 An instance of the actual uncertainly varying delay used in simulations τ k = 0.0156 + 0.0156* δ |δ| < 1

45. 45 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with uncertain (time-varying) delay τ k = 0.0156 + 0.0156 * δ |δ| < 1

46. 46 7. Robustness of Tracking Performance Numerical Example 1: a networked stable & minimum phase system with uncertain delay

47. 47 7. Robustness of Tracking Performance Numerical Example 2: a networked unstable system SPTC was designed via LQR with R=1, Q=100*I 2 u(t) = -9.05 x 1 (t) -10.78 x 2 (t) + 10.05 *r gives perfect tracking in the absence of delays The Q matrix was selected small in order to avoid high feedback gains…and yet

48. 48 7. Robustness of Tracking Performance Numerical Example 2: a networked unstable system with constant delay τ k = τ sc +τ ca =0.0155s

49. 49 7. Robustness of tracking performance…….some comments Many more simulation results with different 2 nd order benchmark S.I.S.O systems from the literature (not shown) Many more simulation results with different 2 nd order benchmark S.I.S.O systems from the literature (not shown) But….we can deduce useful conclusions (despite the lack of a mathematically rigorous approach) But….we can deduce useful conclusions (despite the lack of a mathematically rigorous approach) Clearly a more sophisticated approach is needed for the design of tracking controllers for NCS Clearly a more sophisticated approach is needed for the design of tracking controllers for NCS We cannot pretend that the delays are not there - must take them into account in the design phase. We cannot pretend that the delays are not there - must take them into account in the design phase. We can not compromise stability (avoid large gains) - rule of thump for Time-Delayed -Systems (mid 50s result !!!) We can not compromise stability (avoid large gains) - rule of thump for Time-Delayed -Systems (mid 50s result !!!)

50. 50 CONCLUSIONS AND FUTURE WORK 1.The constant delay case (contrary to intuition) is as detrimental to tracking performance as the varying delay case. 1.The constant delay case (contrary to intuition) is as detrimental to tracking performance as the varying delay case. 2.The feedback gain must be kept small. 2.The feedback gain must be kept small. –If an LQR design is employed : extensive trial-and-error simulations with various Q matrices must be carried out for the entire delay range to ensure (at least) stability. –Tracking for the case of unstable plants and/or lightly damped plants is not trivial. 3.For Unstable plants it is always difficult to enforce tracking (with or without delays). 3.For Unstable plants it is always difficult to enforce tracking (with or without delays). 4. When implementing the tracking controllers in discrete- time special attention is needed due to (1) the interplay between sampling period and delay and (2) the asynchronous / jump nature of the control signal 4. When implementing the tracking controllers in discrete- time special attention is needed due to (1) the interplay between sampling period and delay and (2) the asynchronous / jump nature of the control signal

51. 51 Last Minute Thoughts – Dynamical Systems with Time Delays Consider the time delay systems: Consider the time delay systems:

52. 52

53. 53 CONCLUSIONS AND FUTURE WORK Generalize achieved results for Generalize achieved results for –MIMO NCS plants with multiple delays, Parametric Uncertainties & Actuator constraints The use of Robust Control Methodologies (H or Guaranteed Cost) for the design of Feedback Gain The use of Robust Control Methodologies (H or Guaranteed Cost) for the design of Feedback Gain The employment of Integral Action (apart from feedback and feedforward terms) in the tracking control Algorithm(s). The employment of Integral Action (apart from feedback and feedforward terms) in the tracking control Algorithm(s). Investigate Specific Applications: Aerospace & Robotics (Teleoperation) Investigate Specific Applications: Aerospace & Robotics (Teleoperation) NCSs indeed constitute a very interesting and rich field of control systems both in theoretical results as well as in future applications. NCSs indeed constitute a very interesting and rich field of control systems both in theoretical results as well as in future applications.

54. 54 THANK YOU