1. Neural Networks and Machine Learning Applications CSC 563 Prof. Mohamed Batouche Computer Science Department CCIS – King Saud University Riyadh, Saudi Arabia mbatouche@ccis.ksu.edu.sa

2. Complex Systems Emergent Behaviors and Patterns from Local Interactions

3. 3 What is a complex system? A complex system displays some or all of the following characteristics: Agent-based Basic building blocks are the characteristics and activities of individual agents Heterogeneous The agents differ in important characteristics Dynamic Characteristics change over time, usually in a nonlinear way; adaptation Feedback Changes are often the result of feedback from the environment Organization Agents are organized into groups or hierarchies Emergence Macro-level behaviors that emerge from agent actions and interactions

4. 4 Complex systems The fundamental characteristic of a complex system is that it exhibits emergent properties: Local interaction rules between simple agents give rise to complex pattern and global behavior

5. 5 Complex vs. Simple Systems Have many parts Parts are interdependent in behaviour Difficult to understand because: – behaviour of whole understood from behaviour of parts – behaviour of parts depends on behaviour of whole

6. 6 Examples of Complex Systems Brain Government Family Wold Ecosystem Local Ecosystem (desert, ocean, rainforest) Weather University Ant colony …

7. 7 Anything to be Learnt from Ant Colonies? Fairly simple units generate complicated global behaviour. An ant colony expresses a complex collective behavior providing intelligent solutions to problems such as: carrying large items forming bridges finding the shortest routes from the nest to a food source, prioritizing food sources based on their distance and ease of access. “If we knew how an ant colony works, we might understand more about how all such systems work, from brains to ecosystems.” (Gordon, 1999)

8. 8 Shortest path discovery

9. 9 Adaptation to Environmental Changes

10. 10 Interactions among Social Insects Direct Interactions Food or liquid exchange Visual or tactile contact Indirect Interactions: Stigmergy Pheromones Individual behaviour modifies the environment (e.g., by putting up signs = stigma), which in turn modifies the behaviour of other individuals.

11. 11 Demo NetLogo Massively Parallel MicroWolds

12. 12 Universal properties shared by complex systems Emergence: The appearance of macroscopic patterns, properties, or behaviors that are not simply the “sum” of the microscopic properties or behaviors of the components Self-organization: In biological systems, the emergent order often has some adaptive purpose – e.g., efficient operation of ant colony

13. 13 Emergence: Attempt of a Definition From the book: Steven Johnson, Emergence—The Connected Lives of Ants, Brains, Cities, and Software – Emergence is what happens when an interconnected system of relatively simple elements self-organizes to form more intelligent, more adaptive higher-level behaviour. – It’s a bottom-up model; rather than being engineered by a general or a master planner, emergence begins at the ground level. – Systems that at first glance seem vastly different […] all turn out to follow the rules of emergence.

14. 14 Emergence in complex systems How do neurons respond to each other in a way that produces thoughts (minds)? How do cells respond to each other in a way that produces the distinct tissues of a growing embryo? How do species interact to produce predictable changes, over time, in ecological communities?...

15. 15 Emergence in complex systems Boids of Craig Reynolds

16. 16 Emergence in complex systems Boids of Craig Reynolds

17. 17 Emergence in complex systems Boids of Craig Reynolds

18. 18 Emergence in complex systems Boids of Craig Reynolds

19. 19 Why Are Complex Systems Important for CS? Fundamental to theory & implementation of massively parallel, distributed computation systems How can millions of independent computational (or robotic) agents cooperate to process information & achieve goals, in a way that is: – efficient – self-optimizing – adaptive – robust in the face of damage or attack

20. 20 Some of Natural Systems adaptive path minimization by ants fish schooling and bird flocking evolution by natural selection information processing in the brain wasp and termite nest building pattern formation in animal coats game theory and the evolution of cooperation computation at the edge of chaos

21. 21 Some of Artificial Systems artificial neural networks simulated annealing cellular automata ant colony optimization artificial immune systems particle swarm optimization genetic algorithms other evolutionary computation systems

22. 22 Conclusions We can learn from nature and take advantage of the problems that she has already solved. Many simple individuals interacting with each other can make a global behavior emerge. Techniques based on natural collective behavior (Swarm Intelligence) are interesting as they are cheap, robust, and simple. They have lots of different applications. Swarm intelligence is an active field in Artificial Intelligence, many studies are going on.