1. Self-organization in Multi-Agent Systems Manoel Teixeira de Abreu mnetto@les.inf.puc-rio.br

2. Roadmap Motivation –Multi-agent System Self-Organization –Definitions –Properties –Characteristics –Measures Emergence –Definitions –What Does Emerge? –Properties Design Patterns for Decentralised Coordination

3. Motivation Operate in dynamic, heterogeneous environments and face the challenge of handling frequently changing requirements. Such ubiquitous computing, biological systems and autonomic computing. Must be flexible, robust and capable of adapting to the circumstances. The ability of a system to have a complex collective response arising from interactions among relatively simple individual components.

4. Multi-agent System Autonomous software entites are agents MAS is a group of cooperating agents Macroscopic behaviour using only locally agents interations

5. Roadmap Motivation –Multi-agent System Self-Organization –Definitions –Properties –Characteristics –Measures Emergence –Definitions –What Does Emerge? –Properties Design Patterns for Decentralised Coordination

6. Self-Organization - Definitions Defined as the mechanism or the process enabling a system to change its organization without explicit external command during its execution time. Strong self-organizing systems are those systems where there is no explicit central control either internal or external. Purely decentralized (access to global information is limited or impossible) Weak self-organizing system are those systems where, from an internal point of view, there is re-organization maybe under an internal (central) control or planning. Interactions occur locally (among neighbours) and based on local information. 6

7. Self-Organization – Properties (I/II) Absence of explicit external control. This is a mandatory property that states that the system is autonomous; it imposes and changes its organization based solely on internal decisions and without following any explicit external (re-) organization command. This property refers to the self- part of the self-organization definition. Decentralized control. A self-organizing system can work under decentralized control. In this case, there is no internal central authority or centralized information flow. As a result, access to global information is limited by the locality of interactions, which is governed by simple rules. This property is generally not mandatory, since we can observe internal central control in many natural self-organizing systems, such as the termite systems.

8. Self-Organization – Properties (II/II) Dynamic operation. This mandatory property is linked to the system evolution in time. Since the organization evolves independently of any external control, this property implies continuity in the self-organization process.

9. Self-Organization – Characteristics (I/III) Endogenous global order. The system reaches some (stable) global state that is produced from within the system. Emergence. This refers to having emergent phenomena arising from local interactions. Such phenomena can be observed at global level but they cannot be accurately deduced from examining individual behaviours. Arise from local interactions occurring among the individual components. Local rules. The overall complex system behaviour can be based on simple individual behavioural rules. Local information describes the mechanism for producing the global behavioural pattern and not the pattern itself.

10. Self-Organization – Characteristics (II/III) Dissipation. In the absence of external perturbation, the system is expected to stabilize in some states in which emergent properties can be observed. This implies a kind of dissipation of some energy, otherwise the system would be continuously changing. Instability. Systems showing instability are mainly characterized by nonlinear dynamics which have the effect of small fluctuations in environmental conditions that result in significant variations in overall system behaviour. Multiple equilibria. Multiple equilibria are observed when many possible attractors for stable states are present in the system.

11. Self-Organization – Characteristics (III/III) Redundancy. A system presents redundancy when it demonstrates insensitivity to damage due to replication of components. Self-maintenance. A system is self-maintained if it has the capacity to reproduce or repair itself, essentially by reproducing or repairing its components. Adaptivity. The re-organization capability of self-organizing systems implies adaptation to external (environmental) variations. Complexity. This characteristic usually arises from the irreducibility of the global properties to a combination of local behaviours. Hierarchies. Hierarchies are present in a system when multiple nested self-organized levels can be observed.

12. Self-Organization - Measures Typical self-organization measures includes: Capacity to reach an organization able to fulfil the goal of the system as a whole once the system is started (success/failure/time required, convergence); Capacity to reach a re-organization after a pertubing event (success/failure/time required); Degree of decentralized control (central/totally decentralized/hybrid); Capacity to withstand pertubations (stability/adaptability)

13. Roadmap Motivation –Multi-agent System Self-Organization –Definitions –Properties –Characteristics –Measures Emergence –Definitions –What Does Emerge? –Properties Design Patterns for Decentralised Coordination Some examples

14. Emergence - Definitions The whole is more than the sum of the parts. Witch captures the essence of the emergent phenomena. Tom de Wolf et al., 2005 [1] defines Emergence as A system exhibits emergence when there are coherent emergents at the macroscopic level that arise from the interactions between the parts at the microscopic level. Such emergents are novel w.r.t. the individual parts of the system. Di Marzo Serugendo el at., 2006: A structure (pattern, property or function), not explicitly represented at the level of the individual components (lower level), and which appears at the level of the system (higher level). 14

15. Pheromone Path of Ants What does emerge? It is considered emergent if and only if there exists an observer capable of observing and characterizing it as such. 15

16. Emergence – Properties (I/II) Novelty. Refers to the fact that although the resulting phenomenon is derived from a particular organization and interaction of the micro-level parts, it is radically different from their individual properties and it cannot be derived or predicted from them. Irreducibility. Structures and/or functions appear at a macroscopic level and the observation of the component properties cannot enable to predict them (Atlan, 1993). For example, in complex systems, the complexity often results in features, emergent properties, which are properties of the system that the individual parts do not have.

17. Emergence – Properties (II/II) Interdependency between levels. In these systems there are at least two levels, a micro-level corresponding to the substrate where the emergent phenomena comes from and a macro-level enabling the observation of the emergent phenomena. The micro-level causes the emergent phenomena and the macro-level constrains the behaviour of the entities at the micro-level. Therefore, there is a strong dependency between the dynamics observed at both macro- and micro-levels. Nonlinearity. An emergent phenomenon originates from nonlinear activities at the micro-level. Typical examples of nonlinear activities are loops of positive and negative feedback.

18. Why Decentralized? Computation and decisions to be distributed among the different components, thus preventing the need for a central powerful computer; The system is more robust since it does not rely on a single node that may fail and crash the whole system; Network and CPU resources are better used in the sense that communication does not occur among a dedicated central node and a large number of components, but locally among the whole set of components; In dynamic systems, where components join and leave the system permanently, decentralized control allows a flexible schema for communication, e.g. with a neighbor instead of with the central entity.

19. Roadmap Motivation –Multi-agent System Self-Organization –Definitions –Properties –Characteristics –Measures Emergence –Definitions –What Does Emerge? –Properties Design Patterns for Decentralised Coordination

20. Design Patterns for Decentralized Coordination Choice of one or more decentralised coordination mechanisms to achieve the desired macroscopic behaviour. Using such a set of structured patterns, self-organising emergent system can be designed more systematically.

21. Gradient Fields Context/Applicability –Need for decentralized coordination of multiple autonomous entities in a environment which is robust and flexible in face of changes. Problem –Routing of messages, agents. Solution –Come from physics: movement of masses and particles according to gravitational/electromagnetic fields

22. 22 Gradient Fields How does the mechanism work?

23. 23 Gradient Fields Solution (cont.) –Parameter Tuning: Propagation factor, propagation rate -> How quick are updates needed when changes occur –Infrastructure: What is need to support the mechanism? An environment supporting field storage and propagation –Characteristics: Handles dynamic situations – Robust Simple agents – Complex Environment. Environment transparently and completely handles coordination issues. Know Uses –Spatial shape formation –Urban Traffic Management

24. Problem(s)SolutionPattern Spatial Source to Destination Routing, Task Recruitment, Relation Identification, Integration of various information Sources Agents explicitly search for goals, tasks, or related items and drop pheromones to form paths for other agents to follow to the goal or task. Reinforcement of an existing path by other agents can be seen as a reinforcement of the relation between source and destination. Evaporation, Aggregation, and Propagation keep the pheromones up-to- date and support integration of various information sources. Digital Pheromone Paths Spatial Movement, Pattern Formation, Structure Formation, Routing, Integration of Contextual Information Spatial, contextual, and coordination-related information is automatically spread/propagated by the environment as multiple computational fields. Agents simply follow the waveform of these fields to achieve the coordination task, no explicit exploration is needed. The spatial information comes to the agents instead of agents explicitly searching. Gradient Fields Resource Allocation in general (resource=task, power, bandwidth, space, time, etc.), Integration of resource Usage/Need Information A virtual market where resource users sell and buy resource usage with virtual currency. The price evolves according to the market dynamics and indicates a high (high price) or low (low price) demand. This information is used by agents to decide on using the resource or not. Economic market theory states that the prices converge to a stable equilibrium. Market-based Coordination Team-formation, Trust and reputation Agents put and modify tags on other agents and a team is formed by only collaborating with agents with the same tag or some other condition. If tags indicate how well agents behaved in collaborations with others then trust and reputation information can be available. Tags Synchronisation, Resource Allocation A capability or a resource is represented by a token. Synchronisation happens because only the holder can use the resource or executes a certain capability. Tokens can be passed among agents to allocate the resource or capability to others. Tokens ………

25. Simple Patterns Replication –Allows for recovering minor mutations; –Lowers the time required to access a resource. Collective Sort –Grouping together related information helps to manage batch processing; –The goal of Collective Sort is to group together similar information in the same node, while separating different kinds of information. Evaporation –Mechanism to reduce information amount, based on a time relevance criterion. Aggregation –Mechanism of reinforcement (eg. pheromone). Diffusion –When pheromone is deposited into the environment it spontaneously tends to diffuse to neighboring locations.

26. References Gardelli, Luca and Viroli, Mirko and Omicini, Andrea. "Design Patterns for Self-Organizing Multiagent Systems, 2nd International Workshop on Engineering Emergence in Decentralised Autonomic Systems (EEDAS 2007), CMS Press, University of Greenwich, London, UK. 61-70. G. Di Marzo Serugendo, M.-P. Gleizes, A. Karageorgos. "Self-organisation and emergence in MAS: an overview", Informatica 30(1): 45-54, Slovene Society Informatika, Ljubljana, Slovenia, 2006. G. Di Marzo Serugendo, M.-P. Gleizes, A. Karageorgos. "Self-Organisation in MAS", Knowledge Engineering Review 20(2):165-189, Cambridge University Press, 2005. G. Di Marzo Serugendo. "Autonomous Systems with Emergent Behaviour". Chapter in Handbook of Research on Nature Inspired Computing for Economy and Management. Jean-Philippe Rennard (Ed), Idea Group, Inc., Hershey-PA, USA, pp. 429-443, September 2006. T. De Wolf, Analysing and engineering self-organising emergent applications, Ph.D. Thesis, Department of Computer Science, K.U.Leuven, Leuven, Belgium, May, 2007. T. De Wolf, and T. Holvoet, Emergence Versus Self-Organisation: Different Concepts but Promising When Combined, Engineering Self Organising Systems: Methodologies and Applications (Brueckner, S. and Di Marzo Serugendo, G. and Karageorgos, A. and Nagpal, R., eds.), Lecture Notes in Computer Science, 2005, Volume 3464, May 2005. T. De Wolf, and T. Holvoet, Design Patterns for Decentralised Coordination in Self-organising Emergent Systems, Editors: Sven Brueckner, Salima Hassas, Màrk Jelasity and Daniel Yamins, Engineering Self-Organising Systems: Fourth International Workshop, ESOA 2006, Future University-Hakodate, Japan, 2006, Revised Selected Papers, Lecture Notes in Computer Science, Volume 4335, 2007, pp. 28–49, Springer Verlag 26

27. References Jan Sudeikat and Wolfgang Renz: "Toward Requirements Engineering for Self-Organizing Multi-Agent Systems" in: Proceedings of the First International Conference on Self-Adaptive and Self-Organizing Systems, 2007. Holland, J. H. (1998). Emergence – from Chaos to Order. Oxford University Press.

28. The End. Questions?