1. 27/10/2009 IVR Herrmann 1 IVR: Control Theory OVERVIEW Given desired behaviour, determine control signals Inverse models: – Inverting the forward model for simple linear dynamic system – Problems for more complex systems Open loop control: advantages and disadvantages Feed forward control to deal with disturbances

2. 27/10/2009 IVR Herrmann 2 Control 2 Keypoints: Given desired behaviour, determine control signals Inverse models: – Inverting the forward model for simple linear dynamic system – Problems for more complex systems Open loop control: advantages and disadvantages Feed forward control to deal with disturbances

3. 27/10/2009 IVR Herrmann 3 The control problem For given motor commands, what is the outcome? → Forward model For a desired outcome, what are the motor commands? → Inverse model From observing the outcome, how should we adjust the motor commands to achieve a goal? → Feedback control Motor command Robot in environment Outcome Goal Action

4. 27/10/2009 IVR Herrmann 4 x(t) is an unknown function f is a given analytical expression Differential Equations Using known relations between quantities and their rate of change in order to predict the behaviour of these quantities Equation: Initial condition: Examples: Pendulum equation simplified pendulum equation and an initial condition (see last lect.) Matrix form of the simplified equation Each of these: two initial conditions, e.g. x(0)=x 0 and v(0)=v 0

5. 27/10/2009 IVR Herrmann 5 Differential equation of a two-link planar arm r=2 Equations are classical physics. Here copied from the paper:

6. 27/10/2009 IVR Herrmann 6 Solving differential equations numerically* 1.Given and 2.Choose a small step size Δt 3.Start at t = 0 and x(0) = x 0 4.Iterate x(t+Δt) = x(t) + Δt dx/dt until t = T *This is the most simple way of integrating a differential equation. Even for small step sizes considerable errors may accumulate. x(t)x(t) x t ΔtΔt old value approximate new value tangent to x(t) t t+Δt

7. 27/10/2009 IVR Herrmann 7 Example from last lecture: Electric Motor Simple dynamic example – We have a process model: Solve to get forward model s(0)=s 0 : Derivation of this and more general cases using e.g. Laplace transformation Battery voltage V B Vehicle speed v ? VBVB IR e

8. 27/10/2009 IVR Herrmann 8 Solving differential equations numerically 1.Given and Rewrite: 2.Choose a small step size Δt 3.Start at t = 0 and s(0)= s 0 4.Iterate s(t+Δt) = s(t) + Δt ds/dt until t = T Example from last lecture:

9. 27/10/2009 IVR Herrmann 9 The control problem Forward: Given the control signals, can we predict the motion of the robot? Motor command Robot in environment Forward model Predicted output Inverse: Given the desired motion of the robot Can we determine the right control signals? Control theory aims at unique or easy exact solutions Difficult in the presence of non-linearities or noise Actual output ? Desired output

10. 27/10/2009 IVR Herrmann 10 Open loop control Examples: To execute a memorised trajectory, produce appropriate sequence of motor torques To obtain a goal, make a plan and execute it – means-ends reasoning could be seen as inverting a forward model of cause-effect ‘Ballistic’ movements such as saccades Motor command Robot in environment Inverse model

11. 27/10/2009 IVR Herrmann 11 Open loop control Potentially cheap and simple to implement e.g. if solution is already known. Fast, e.g. useful if feedback would come too late. Benefits from calibration e.g. tune parameters of approximate model. If model unknown, may be able to use statistical learning methods to find a good fit e.g. neural network.

12. 27/10/2009 IVR Herrmann 12 Open loop control Neglects possibility of disturbances, which might affect the outcome. For example: – Change in temperature may change the friction in all the robot joints. – Or unexpected obstacle may interrupt trajectory Motor command Robot in environment Inverse model Disturbances

13. 27/10/2009 IVR Herrmann 13 Feed-forward control One solution is to measure the (potential) disturbance and use this to adjust the control signals. For example – Thermometer signal alters friction parameter. – Obstacle detection produces alternative trajectory. Motor command Robot in environment Inverse model Disturbances Measurement

14. 27/10/2009 IVR Herrmann 14 Feed-forward control Can sometimes be effective and efficient. Requires anticipation, not just of the robot process characteristics, but of possible changes in the world. Does not provide or use knowledge of actual output (for this need to use feedback control)

15. 27/10/2009 IVR Herrmann 15 Open loop control: Motor command Robot in environment Inverse model Feed-forward control: Motor command Robot in environment Inverse model Disturbances Measurement Motor command Robot in environment Control Law Disturbances Measurement Feed-back control: Disturbances ?

16. 27/10/2009 IVR Herrmann 16 Combining control methods In practice most robot systems require a combination of control methods – Robot arm using servos on each joint to obtain angles required by geometric inverse model – Forward model predicts feedback, so can use in fast control loop to avoid problems of delay – Feed-forward measurements of disturbances can be used to adjust feedback control parameters – Training of an open-loop system is essentially a feedback process

17. 27/10/2009 IVR Herrmann 17 Feedback control Example: Greek water clock Requires constant flow to signal time Obtain by controlling water height Float controls inlet valve

18. 27/10/2009 IVR Herrmann 18 Feedback control Example: Watt governor Steam engine Drive shaft Steam valve

19. 27/10/2009 IVR Herrmann 19 Compare: room heating Open loop: for desired temperature, switch heater on, and after pre-set time, switch off. Feed forward: use thermometer outside room to compensate timer for external temperature. Feedback: use thermometer inside room to switch off when desired temperature reached. NO PREDICTION REQUIRED!

20. 27/10/2009 IVR Herrmann 20 Motor command Robot in environment Control Law Disturbances Measurement Control laws in feed-back control On-off: switch system if desired = actual Servo: control signal proportional to difference between desired and actual, e.g. spring: Desired output Actual output

21. 27/10/2009 IVR Herrmann 21 Servo control Simple dynamic example: Control law: So now have new process: Battery voltage V B Vehicle speed s ? ss goal gain=K With steady state: And half-life:

22. 27/10/2009 IVR Herrmann 22 ? ? Say we have s goal =10. What does K need to be for s ∞ = 10? If K=5, and s ∞ = 8, and the system takes 14 seconds to reach s=6 starting from s=0, what is the process equation?

23. 27/10/2009 IVR Herrmann 23

24. 27/10/2009 IVR Herrmann 24 Motor command Robot in environment Control Law Disturbances Measurement Complex control law May involve multiple inputs and outputs E.g. ‘homeostatic’ control of human body temperature – Multiple temperature sensors linked to hypothalamus in brain – Depending on difference from desired temperature, regulates sweating, vasodilation, piloerection, shivering, metabolic rate, behaviour… – Result is reliable core temperature control Desired output Actual output

25. 27/10/2009 IVR Herrmann 25 Negative vs. Positive Feedback Examples so far have been negative feedback: control law involves subtracting the measured output, acting to decrease difference. Positive feedback results in amplification: – e.g. microphone picking up its own output. - e.g. ‘runaway’ selection in evolution. Generally taken to be undesirable in robot control, but sometimes can be effective – e.g. in exploratory control, self-excitation – model of stick insect walking (Cruse et al 1995)‏ – (requires negative feedback from the environment)‏

26. 27/10/2009 IVR Herrmann 26 6-legged robot has 18 degrees of mobility. Difficult to derive co-ordinated control laws But have linked system: movement of one joint causes appropriate change to all other joints. Can use signal in a feed- forward loop to control 12 joints (remaining 6 use feedback to resist gravity).

27. 27/10/2009 IVR Herrmann 27 Advantages Major advantage is that (at least in theory) feedback control does not require a model of the process. – E.g. thermostats work without any knowledge of the dynamics of room heating (except for time scales and gains)‏ Thus controller can be very simple and direct – E.g. hardware governor does not even need to do explicit measurement, representation and comparison Can (potentially) produce robust output in the face of unknown and unpredictable disturbances – E.g. homeostatic body temperature control keeps working in unnatural environments

28. 27/10/2009 IVR Herrmann 28 Issues Requires sensors capable of measuring output – Not required by open-loop or feed-forward Low gain is slow, high gain is unstable Delays in feedback loop will produce oscillations (or worse)‏ In practice, need to understand process to obtain good control: – E.g. James Clerk Maxwell’s analysis of dynamics of the Watt governor …more next lecture

29. 27/10/2009 IVR Herrmann 29 Cybernetics Feedback control advanced in post WW2 years (particularly by Wiener) as general explanatory tool for all ‘systems’: mechanical, biological, social… E.g. William Powers – “Behaviour: the control of perception” Jay Wright Forrester – “World dynamics” – modelling the global economic-ecological network

30. 27/10/2009 IVR Herrmann 30 Further reading: Most standard robotics textbooks (e.g. McKerrow) discuss forward and inverse models in great detail. For the research on saccades see: Harris, C.M. & Wolpert, D.M. (1998) Signal dependent noise determines motor planning. Nature, 394, pp. 780-184. For an interesting discussion of forward and inverse models in relation to motor control in humans, see: Wolpert, D.M. & Ghahramani, Z. (2000) “Computational principles of movement neuroscience” Nature Neuroscience 3, pp. 1212- 1217. Wolpert D.M., Ghahramani Z, & Flanagan J. R. (2001) Perspectives and problems in motor learning. Trends in Cognitive Sciences 5:11, pp 487-494.

31. 27/10/2009 IVR Herrmann 31 Further reading on feedback control: Most standard robotics textbooks (e.g. McKerrow) discuss feedback control in great depth. For the research on stick insect walking see: Cruse, H., Bartling, Ch., Kindermann, T. (1995) High-pass filtered positive feedback for decentralized control of cooperation. In: F. Moran, A. Moreno, J. J. Merelo, P. Chacon (eds. ), Advances in Artificial Life, pp. 668-678. Springer 1995 Classic works on cybernetics: Wiener, Norbert: (1948) Cybernetics. or Control and Communication in the Animal and Machine, MIT Press, Cambridge Powers, WT: (1973) Behavior, the Control of Perception, Aldine, Chicago Forrester, JW (1971) World Dynamics, Wright and Allen, Cambridge