batouche@ccis.edu.sa Prof. Dr. Mohamed Batouche Complex Systems Engineering SwE 488 Introduction Prof. Dr. Mohamed Batouche Department of Software Engineering CCIS – King Saud University Riyadh, Kingdom of Saudi Arabia batouche@ccis.edu.sa

Outline Introduction Natural Complex Systems Complexity Science Artificial Complex Systems Applications Conclusion

Introduction New Software Engineering Paradigm based on Complexity Science

What is Wrong with Today’s Software Engineering Paradigm? The foundation of today’s software engineering paradigm is wrong (software is a nonlinear complex system) The process models are wrong (Waterfall, incremental …) The software development methodologies are outdated The software testing paradigm is outdated The quality assurance paradigm is outdated Jay Xiong

What Is the Root Cause for Those Critical Issues Existing with Today’s Software Engineering? The root cause for those critical issues comes from the wrong foundation of the software engineering paradigm that software and the software engineering paradigm are complex nonlinear systems, and should be handled with complexity science to comply with the essential principles of complexity science, particularly the Nonlinearity principle and the Holism principle to make all tasks and activities being performed holistically and globally rather than partially and locally. Jay Xiong

The Challenge Sources of complexity: Creation of common principles, unified theories, and methods to design, operate, and protect complex engineered systems. Sources of complexity: Too much information, too many components, too many constraints, too many parameters for consideration to accomplish a particular task Not enough information about essential elements or components of a system or about their interfaces Not enough information about how elements or components will behave under known or unknown conditions that may lead to unintended consequences

Natural Complex Systems What are Natural Complex Systems

Natural Complex Systems Bottom-up behavior: Simple agents following simple rules generate complex structures/behaviors. A termite "cathedral" mound produced by a termite colony: a classic example of emergence in nature. Birds Flocking Fish School

Natural Complex Systems

Nature Inspired Computing Draws on the principles of emergence, self-organization, and complex systems Aims to develop new techniques and algorithms by getting ideas from nature.

Complexity Science New Kind of Science

Systems System Definition System Elements System Function A combination of interrelated elements, parts, methods, or procedures forming a complex unitary whole working together toward a common objective (Blanchard & Fabrycky, 2006). Unity, Functional relationship, Useful purpose. System Elements Components - Input, Process, Output. Attributes - Properties of the components. Relationships - Link between the first two. System Function Purposeful action performed by the system. Alter material, energy, information.

What is A System? A System is a collection of entities, elements, or components, e.g., people or machines, that act and interact together toward the accomplishment of some logical end. A system performs one or more functions. A system is composed of interacting components. A system performs functions not performable by its components. A system has a boundary, separating it from everything else. -- These are systems -- This chart describes the essential characteristics of a system. You can declare virtually anything a system and sometimes it’s useful to do so. If it performs a function, it’s a system. Even a paperweight, whose function is to hold down a stack of paper, has components at the molecular level. Small-scale Hardware Large-scale Hardware People & Organizations

Systems ©New England Complex Systems Institute, Cambridge, Massachusetts

Engineered Systems A combination of interacting elements organized to achieve one or more stated purposes. Systems Engineering is an interdisciplinary approach and means to enable the realization of successful systems. - INCOSE System Inputs Materials Information Energy Organizational Structure Outputs Product Resources Waste Controls Technical Political Economic Environmental Mechanisms Human Resources Computer Resources Facilities/Utilities Maintenance/Support Environment

Traditional Systems Engineering SE design process is developed to produce efficient and reliable systems. Respect pre-specified constraints. Meet pre-specified standards of performance. Work in pre-specified situations. Distinction between the design/production phase and the operational phase. Does not allow any type of adaptation and self-organization in the operational phase. Adaptive controllers do follow this two-phase. Adaptation only in the superficial sense of parameter adjustment.

Complexity and Complex Systems “I think the next century will be the century of complexity.” Stephen Hawking for the San Jose Mercury News on January 23, 2000. “Complex systems science emerged from the interplay of physics, mathematics, biology, economy, engineering, and computer science, with the mission to overcome the simplifications and idealizations that lead to unrealistic models in these sciences” (Chu et al., 2003). “...(Complex systems) landscape is bubbling with activity, and now is the time to get involved. Engineering should be at the center of these developments...” (Ottino, J. M. 2004).

Complex Systems System with a large number of components and interconnections, interactions or interdependencies that are difficult to describe, understand, predict, manage, design, and/or change (C.L Magee, O.L de Weck, MIT) System whose behavior is not deducible, nor may it be inferred from the structure and behavior of its component parts (Bar-Yam, 2003). Characteristics: Property of self-organization or emergence of structure from the interaction between the constituent parts of the system. Self-organization: spontaneous appearance of large-scale organization through limited interactions among parts of the system. Intrinsically related to notions such as complex adaptive systems, emergence, and complexity.

Complicated vs. Complex Complicated systems: Systems whose behavior can be completely understood through functional decomposition. Ex: Microprocessor Carefully designed and tested: works as documented. Efficiency dependent on all components. Increase in performance needs engineering redesign. Complex systems: Systems that exhibit emergent behavior. Ex: U.S. economy No one designed U. S. economy; yet it works. Efficiency is robust to perturbations and failures. Grows (shrinks) on its own, without any explicit control.

Complex Systems Complicated vs. Complex Complicated Systems Complex Systems

Complex Systems ©New England Complex Systems Institute, Cambridge, Massachusetts

Complex Systems Complex systems are systems that exhibit emergent behavior: Anthills Human societies Financial Markets Climate Nervous systems Immune systems Cities Galaxies Modern telecommunication infrastructures

Complex Engineered Systems Man-made complex systems that exhibit emergent behavior while retaining important systems engineering characteristics such as robustness, reliability, and fault-tolerance. Examples: air and ground transportation networks, distributed manufacturing environments, distributed supply networks, financial markets, healthcare delivery. In the large-scale systems domain, complex systems are the rule not the exception. Complex systems can only be engineered by intervention, not by specification and development. →The system is no longer designed, it is engineered to evolve, adapt, and reconfigure as needed.

Characteristics of a complex system? A complex system displays some or all of the following characteristics: Agent-based Basic building blocks are the characteristics and activities of individual agents Heterogeneous The agents differ in important characteristics Dynamic Characteristics change over time, usually in a nonlinear way; adaptation Feedback Changes are often the result of feedback from the environment Organization Agents are organized into groups or hierarchies Emergence Macro-level behaviors that emerge from agent actions and interactions

Attributes for complex system? Interdependent Independent Distributed Cooperative Competitive Adaptive

Complex systems The fundamental characteristic of a complex system is that it exhibits emergent properties: Local interaction rules between simple agents give rise to complex pattern and global behavior

The concept of Emergent Behavior the notion of emergence appeared with the general systems theory and is linked to complexity. “The global behavior is greater than sum of the behaviors of the individual parts” The global emergence characterizes the properties of a system that are new ones.

Complex Systems perspectives Complex Systems as a Science to Understand Nature Complex Systems as a New Form of Engineering

Engineering Complex Systems – Key idea The idea of nature inspired computing arose from the observations and questions regarding natural emergent phenomena: How do ants build their colonies, without any central control?   How do termites build their nests, without an architect that draws a plan? How do species evolve without any direction or imposed objective function?

Complex Systems Engineering How can we understand and make use of these emergent phenomena to develop new ways of generating computer programs?   Can we build self-adapting, self-organizing, and evolving computer systems and programs?

Complex Systems A new form of Engineering Engineering which faces complex problems using simple dynamic systems (vast parallelism, robustness, locality). « The whole is too much greater than the sum of the parts »

What is A System of Systems? A System of Systems (SoS) is a “super-system” made up of elements – each of which is itself a complex, independent system -- that interact to achieve a common goal. SoS elements (i.e., the systems) can and do operate independently. An SoS evolves – functions are added/removed/changed with experience. An SoS exhibits emergent behavior not attributable to any element (system). An SoS is geographically distributed – elements exchange information only. -- Are These SoS? -- YES YES Systems-of-systems should be distinguished from large but monolithic systems by the independence of their components, their evolutionary nature, emergent behaviors, and a geographic extent that limits the interaction of their components to information exchange . ... ... Five principal characteristics are useful in distinguishing very large and complex but monolithic systems from true systems-of-systems. Operational Independence of the Elements: If the system-of-systems is disassembled into its component systems the component systems must be able to usefully operate independently. The system-of-systems is composed of systems which are independent and useful in their own right. Managerial Independence of the Elements: The component systems not only can operate independently, they do operate independently. The component systems are separately acquired and integrated but maintain a continuing operational existence independent of the system-of-systems. Evolutionary Development: The system-of-systems does not appear fully formed. Its development and existence is evolutionary with functions and purposes added, removed, and modified with experience. Emergent Behavior: The system performs functions and carries out purposes that do not reside in any component system. These behaviors are emergent properties of the entire system-of-systems and cannot be localized to any component system. The principal purposes of the systems-of-systems are fulfilled by these behaviors. Geographic Distribution: The geographic extent of the component systems is large. Large is a nebulous and relative concept as communication capabilities increase, but at a minimum it means that the components can readily exchange only information and not substantial quantities of mass or energy. Mark W. Maier, Architecting Principles for Systems-of-Systems, UA Huntsville, http://www.infoed.com/Open/PAPERS/systems.htm NO Major League Baseball Hardware Oughta Be International Air Travel Joint Theater Ops From INCOSE

Areas of Application Integrated Deepwater System – Optimize and rejuvenate the force structure of the U. S. Coast Guard Future Combat Systems – Define an optimal Brigade Force structure P8A – Integrating a new aircraft into the fleet SBINet- Determine optimal coverage and coordination between sensor, communications, C2. General Navy Applications – formation of a carrier task force Air Force Applications – formation of an air wing From INCOSE

Source: Monica Farah-Stapleton, IEEE SOS conference, 2006 Army SOS Perception

Artificial Complex Systems

Artificial Complex Systems Artificial Neural Networks Cellular Automata Swarm Intelligence Evolutionary Computing Quantum Computing DNA/Molecular Computing Artificial Life Artificial Immune System

Artificial Neural Networks

Developing Intelligent Program Systems Neural Nets Artificial Neural Networks: Artificial Neural Networks are crude attempts to model the highly massive parallel and distributed processing we believe takes place in the brain. Two main areas of activity: Biological: Try to model biological neural systems. Computational: develop powerful applications.

Developing Intelligent Program Systems Neural Nets Neural nets can be used to answer the following: Pattern recognition: Does that image contain a face? Classification problems: Is this cell defective? Prediction: Given these symptoms, the patient has disease X Forecasting: predicting behavior of stock market Handwriting: is character recognized? Optimization: Find the shortest path for the TSP.

Cellular Automata

Cellular Automata Cell State = empty/off/0 State = filled/on/1 The CA space is a lattice of cells (usually 1D, 2D, 3D) with a particular geometry. • Each cell contains a variable from a limited range of values (e.g., 0 and 1). • All cells update synchronously. • All cells use the same updating rule, depending only on local relations. • Time advances in discrete steps.

Game of Life: 2D Cellular Automata using simple rules neighboring values First Generation Conway’s Life: Rules A living cell with 0-1 8-neighbors dies of isolation A living cell with 4+ 8-neighbors dies from overcrowding A cell with 3 living neighbors becomes a living cell All other cells are unaffected Second Generation Emergent pattern: Blinker

Game of Life: emergent patterns Gosper’s glider gun : emits glider stream Conway’s Rules: Game of Life Survive with 2 – 3 living neighbors Generate with 3 living neighbors gliders: patterns that moves constantly across the grid

Emergent Patterns

Emergent Patterns: A Clock See Demo: Game of Life

Artificial Life

Artificial Life Artificial Life: An attempt to better understand “real” life by in-silico modeling of the entities we are aware of. Motivations: The Emergent properties in life motivate scientists to explore the possibility of artificially creating life and expecting the unexpected. An Emergent property is created when something becomes more than sum of its parts.

Boids of Craig Reynolds Bird Flocking “Boids” model was proposed by Reynolds Boids = Bird-oids (bird like) Only three simple rules

Flocking Model The model consists in three simple forces (steering behaviors): Separation: steer to avoid crowding local flockmates Alignment: steer towards the average heading of local flockmates Cohesion: steer to move toward the average position of local flockmates

Boids of Reynolds simple rules give rise to complex behavior

Collision Avoidance Rule 1: Avoid Collision with neighboring birds (separation)

Velocity Matching: Alignment Rule 2: Match the velocity of neighboring birds (Alignment)

Flock Centering: Cohesion Rule 3: Stay near neighboring birds (Cohesion)

Characteristics Simple rules for each individual No central control Decentralized and hence robust Emergence: a system made up of discrete birds yet the overall motion is coordinated.

Boids of Reynolds Boids of Craig Reynolds

Emergence in complex systems Boids of Craig Reynolds

Emergence in complex systems Boids of Craig Reynolds

Swarm Intelligence Ant colony optimization Particle Swarm Optimization

Ant Colony Optimization

Shortest path discovery Ants get to find the shortest path after few minutes …

Ant Colony Optimization Each artificial ant is a probabilistic mechanism that constructs a solution to the problem, using: Artificial pheromone deposition Heuristic information: pheromone trails, already visited cities memory …

TSP Solved using ACO

Particle Swarm Optimization

Particle Swarm Optimization Particle Swarm Optimization (PSO) mimics the collective intelligent behavior of “ unintelligent ” creatures. It was developed in 1995 by James Kennedy and Russell Eberhart Individuals interact with one another while learning from their own experience, and gradually move towards the goal. It is easily implemented and has proven both very effective and quick when applied to a diverse set of optimization problems.

Bird flocking is one of the best example of PSO in nature. One motive of the development of PSO was to model human social behavior.

Algorithm of PSO Each particle (or agent) evaluates the function to maximize at each point it visits in spaces. Each agent remembers the best value of the function found so far by it (pbest) and its co-ordinates. Secondly, each agent know the globally best position that one member of the flock had found, and its value (gbest).

Algorithm of PSO Using the co-ordinates of pbest and gbest, each agent calculates its new velocity as: vi = vi + c1 x rand() x (pbestxi – presentxi) + c2 x rand() x (gbestx – presentxi) where 0 < rand() <1 presentxi = presentxi + (vi x Δt)

Algorithm of PSO

Applications

New Sorting Algorithms based on Natural Computing

Bead-sort Bead-Sort is a method of ordering a set of positive integers by mimicking the natural process of objects falling to the ground, as beads on an abacus slide down vertical rods. The number of beads on each horizontal row represents one of the numbers of the set to be sorted, and it is clear that the final state will represent the sorted set. [10 8 6 2 4 12] [2 4 6 8 10 12]

Bead-sort Extended The "extended" (anti-gravity) mode allows the inclusion of all integers, with "negative beads" rising while "positive beads" fall.

Image processing using Quantum Computing and Reverse Emergence

The proposed method The objective is to solve image processing tasks by emergence using cellular automata. Cellular automata filtered image Noisy image

The proposed approach • The problem at hand is which simple rules provide the desired complex behavior? Solution: Reverse emergence

Reverse Emergence To find simple rules that give rise to the desired complex behavior: Searching simple rules by hand (trials). Taking inspiration from nature (bees, ants, termites, spiders …) Optimization and Learning (GA, PSO, QGA, QPSO …)

Conclusions

Conclusions We can learn from nature and take advantage of the problems that she has already solved. Many simple individuals interacting with each other can make a global behavior emerge. Techniques based on natural collective behavior are interesting as they are cheap, robust, and simple. They have lots of different applications. Simple individuals which interact locally give rise to very complex behavior.

References

References Jay Xiong, New Software Engineering Paradigm Based on Complexity Science, Springer 2011. Claudios Gros : Complex and Adaptive Dynamical Systems. Second Edition, Springer, 2011 . Blanchard, B. S., Fabrycky, W. J., Systems Engineering and Analysis, Fourth Edition, Pearson Education, Inc., 2006. Braha D., Minai A. A., Bar-Yam, Y. (Editors), Complex Engineered Systems, Springer, 2006 Gibson, J. E., Scherer, W. T., How to Do Systems Analysis, John Wiley & Sons, Inc., 2007. International Council on Systems Engineering (INCOSE) website (www.incose.org). New England Complex Systems Institute (NECSI) website (www.necsi.org). Rouse, W. B., Complex Engineered, Organizational and Natural Systems, Issues Underlying the Complexity of Systems and Fundamental Research Needed To Address These Issues, Systems Engineering, Vol. 10, No. 3, 2007.

References Wilner, M., Bio-inspired and nanoscale integrated computing, Wiley, 2009. Yoshida, Z., Nonlinear Science: the Challenge of Complex Systems, Springer 2010. Stephen Wolfram, A New Kind of Science, Art-Quality Printing, 2002 (available online: http://www.wolframscience.com) Gardner M., The Fantastic Combinations of John Conway’s New Solitaire Game “Life”, Scientific American 223 120–123 (1970).   Nielsen, M. A. & Chuang, I. L. ,Quantum Computation and Quantum Information, 3rd ed., Cambridge Press, UK, 2000.

Robots Playing Soccer

Q & A