1. Modelling Hybrid Control Systems with Behaviour Networks Pierangelo Dell’Acqua Anna Lombardi Dept. of Science and Technology - ITN Linköping University, Sweden L. M. Pereira Centro de Inteligência Artificial - CENTRIA Universidade Nova de Lisboa, Portugal ICINCO 05, Barcelona Spain September, 2005 September ‘05

2. Problem We present an approach to model adaptive, dynamic hybrid control systems based on behaviour networks. We extend/modify behaviour networks to make them self- adaptive and dynamic. The approach is general, reconfigurable, and suitable for environments that are dynamic and too complex to be entirely predictable.

3. Outline 1. Hybrid control systems 2. Behaviour networks 3. Extended/modified behaviour networks 4. Modelling hybrid control systems 5. Discussion and future work

4. 1. Hybrid control systems Controller: discrete-time system Interface Plant: time-continuous system to be controlled (possibly traditional controllers) Generator Actuator Plant Controller x(t) x(t k )r(t k ) r(t) Closed-loop configuration

5. Plant: described by differential equations x(t) - state vector r(t) - input vector Generator Actuator Plant Controller x(t) x(t k )r(t k ) r(t)

6. Generator Actuator Plant Controller x(t) x(t k )r(t k ) r(t) Controller: described by difference equations s(t k+1 ) =  ( s(t k ), x(t k ) ) r(t k ) =  ( s(t k ) ) x(t k ) – plant events r(t k ) - actions

7. Interface Generator: converts continuous-time output of the plant into symbols x(t)  x(t k ) Actuator: receives symbols (actions) from the controller and converts them into an input to the plant r(t k )  r(t) Generator Actuator Plant Controller x(t) x(t k )r(t k ) r(t)

8. 2. Behaviour networks Introduced by P. Maes in ‘89 for action selection in dynamic and complex environments where the system has limited computational and time resources. A behaviour network is a network composed of specific competence modules (rules) which activate and inhibit one another along the links of the network. The activation/inhibition dynamics of the network is guided by global parameters. Competence modules cooperate in such a way that the network as a whole functions properly. This architecture is distributed, modular, robust and has an emergent global functionality.

9. 3. Extended/modified behaviour networks To model hybrid control systems, we extended behaviour networks to allow for: rules containing variables; internal actions; integrity constraints, and modules (sets of atoms and rules).

10. An extended behaviour network consists of 5 modules: R - a set of rules formalizing the behaviour of the network P - set of global parameters H - internal memory C - set of integrity constraints G - set of goals We call the state of the network the tuple S=(R,P,H,C,G). We assume given a module Math formalizing the axioms of elementary mathematics.

11. Atom takes the form: m:X meaning that X belongs to module M whose name is m m÷X meaning that X does not belong to M Sequence of atoms: Atom 1, Atom 2,..., Atom n (n  0) # denotes the empty sequence Goal is a sequence of atoms. It expresses a condition to be achieved. Integrity constraint is a sequence of atoms. It expresses a condition that must not hold. Language (simplified)

12. Rule is a tuple of the form: prec - preconditions del, add - represent the internal effect of the rule action - atom representing the external effect of the rule  - level of strength of the rule Variables in rules are universally quantified: E denotes the environment

13. A substitution  is a finite set of bindings: variable/expression A substitution  can be applied to an expression X, written as X , by simultaneously replacing any variable v in X with t, for every binding v/t in . if X = h:q(x,y,z,c) and  = {x/3, z/6}, then X  = h:q(3,y,6,c) Two expressions X and Y are unifiable (i.e. two-way pattern matched), written as X  Y, if there exists a substitution  such that X  =Y . Substitutions can be applied to rules: if r = and  = {x/12}, then r  =

14. At every state, a rule in R has to be selected for execution. A sequence L of atoms is true at a state S iff: for every m:X in L, X  M and for every m÷X in L, there exists no  for which X   M. A state S is safe if there exists no  that makes an integrity constraint in C true at S. Executable rules

15. Applying a rule r= S=(M 1,M 2,M 3,M 4,M 5 ) r(S)=(M’ 1,M’ 2,M’ 3,M’ 4,M’ 5 ) every M’ i is obtained from M i : by removing X for every m i :X in del, and by adding Y for every m i :Y in add. A rule r is executable at S iff: prec is true at S, r(S) is safe, and action is an atom with no variables. r

16. Rules in a network are linked by three types of links. Let x and y be rules. Successor linkxy for every m:X in the add list of x and m:Y in the prec list of y such that X  Y Predecessor link xy for every successor link from y to x Conflict linkxy for every m:X in the prec list of x and m:Y in the del list of y such that X  Y Rule selection

17. Rules use these links to activate/inhibit each other. Forward propagation There is input of activation energy coming from S and E towards rules whose preconditions partially match them. Backward propagation There is input of activation energy from G towards rules whose add lists partially match G. Inhibition There is inhibition by the goals that have been achieved. They remove some activation energy from the rules that would undo them.

18. Rules also inhibit and activate each other along the links in the network. The mathematical model is based on several global parameters in P that are used to tune the spreading of activation energy through the network:  - threshold of rules to become active,  - amount of energy that a true atom injects into the network,  - amount of energy that a goal injects into the network,  - amount of energy that achieved goals take away from the network,... etc.

19. Let r be a rule in R and  a substitution. The rule r  becomes active when: it is executable, its activation level overcomes , has the greatest activation level among all other executable rules. When an active rule has been executed, its activation level is set to 0.

20. 4. Modelling hybrid control systems An adaptive, dynamic hybrid controller can be described by an extended behaviour network. P R H Math C G EActuator CU

21. The basic engine of CU can be described via the following cycle: Cycle(n, R, P, H, C, G) 1.Load the rules of R into CU and calculate their activation level wrt. the global parameters in P. 2.If one rule becomes active, then execute its internal effect, and send its external effect (its action) to the actuator. Let R', P', H', C', G' be the modules after the execution of the rule. Cycle with (n+1, R', P', H', C', G'). 3.If no rule becomes active, then lower the level of  in P. Cycle with (n+1, R, P', H, C, G).

22. Consider a virtual marine world inhabited by a variety of fish. Fish are situated in the environment, and sense and act over it. The behaviour of a fish is reduced to search and eating food, escaping and sleeping. Actions of the fish: search, eat, escape and sleep. Example: artificial fish

23. Modelling an artificial fish Stimuli of the fish: hungry: fear: tired: Input vector to the controllerState of the controller hungry(t k ) food(t k ) x(t k ) = fear(t k ) s(t k ) = satiated(t k ) tired(t k ) safe(t k )

24. C = { } no constraints G = { h:safe, h:satiated } Behaviour of the fish: 0.5, h÷food; # ; h:food; search; 0.5 > 0.5, h:food; # ; h:satiated; eat; 0.5 > 0.5; # ; h:safe; escape; 0.7 > E - stimuli of the fish H - internal state of the fish R =

25. Implementation: XSB Prolog + Java Extension of L to allow variables in the strength level of rules: 20; # ; h:on; heating(on); 0.5*(20-x) > Engineering network of behaviour networks to model complex environments Techniques developed in Logic Programming can be employed: - preference reasoning to enhance action selection - genetic algorithms to tune the global parameters of the network - belief revision to resolve cases of conflicting rules if more rules can become active 5. Future work