1. batouche@ccis.edu.sa Prof. Dr. Mohamed Batouche
Complex Systems Engineering SwE 488 Introduction
Prof. Dr. Mohamed Batouche
Department of Software Engineering
CCIS – King Saud University
Riyadh, Kingdom of Saudi Arabia

2. Outline
Introduction
Natural Complex Systems
Complexity Science
Artificial Complex Systems
Applications
Conclusion

3. Introduction New Software Engineering Paradigm based on Complexity Science

4. What is Wrong with Today’s Software Engineering Paradigm?
The foundation of today’s software engineering paradigm is wrong (software is a nonlinear complex system)
The process models are wrong (Waterfall, incremental …)
The software development methodologies are outdated
The software testing paradigm is outdated
The quality assurance paradigm is outdated
Jay Xiong

5. What Is the Root Cause for Those Critical Issues Existing with Today’s Software Engineering?
The root cause for those critical issues comes from the wrong foundation of the software engineering paradigm that software and the software engineering paradigm are complex nonlinear systems, and should be handled with complexity science to comply with the essential principles of complexity science, particularly the Nonlinearity principle and the Holism principle to make all tasks and activities being performed holistically and globally rather than partially and locally.
Jay Xiong

6. The Challenge Sources of complexity:
Creation of common principles, unified theories, and methods to design, operate, and protect complex engineered systems.
Sources of complexity:
Too much information, too many components, too many constraints, too many parameters for consideration to accomplish a particular task
Not enough information about essential elements or components of a system or about their interfaces
Not enough information about how elements or components will behave under known or unknown conditions that may lead to unintended consequences

7. Natural Complex Systems
What are
Natural Complex Systems

8. Natural Complex Systems
Bottom-up behavior: Simple agents following simple rules generate complex structures/behaviors.
A termite "cathedral" mound produced by a termite colony: a classic example of emergence in nature.
Birds Flocking
Fish School

9. Natural Complex Systems

10. Nature Inspired Computing
Draws on the principles of emergence, self-organization, and complex systems
Aims to develop new techniques and algorithms by getting ideas from nature.

11. Complexity Science New Kind of Science

12. Systems System Definition System Elements System Function
A combination of interrelated elements, parts, methods, or procedures forming a complex unitary whole working together toward a common objective (Blanchard & Fabrycky, 2006).
Unity, Functional relationship, Useful purpose.
System Elements
Components - Input, Process, Output.
Attributes - Properties of the components.
Relationships - Link between the first two.
System Function
Purposeful action performed by the system.
Alter material, energy, information.

13. What is A System?
A System is a collection of entities, elements, or components, e.g., people or machines, that act and interact together toward the accomplishment of some logical end.
A system performs one or more functions.
A system is composed of interacting components.
A system performs functions not performable by its components.
A system has a boundary, separating it from everything else.
-- These are systems --
This chart describes the essential characteristics of a system.
You can declare virtually anything a system and sometimes it’s useful to do so.
If it performs a function, it’s a system.
Even a paperweight, whose function is to hold down a stack of paper, has components at the molecular level.
Small-scale Hardware
Large-scale Hardware
People & Organizations

14. Systems
©New England Complex Systems Institute, Cambridge, Massachusetts

15. Engineered Systems
A combination of interacting elements organized to achieve one or more stated purposes.
Systems Engineering is an interdisciplinary approach and means to enable the realization of successful systems. - INCOSE
System
Inputs
Materials
Information
Energy
Organizational Structure
Outputs
Product
Resources
Waste
Controls
Technical
Political
Economic
Environmental
Mechanisms Human Resources
Computer Resources
Facilities/Utilities
Maintenance/Support
Environment

16. Traditional Systems Engineering
SE design process is developed to produce efficient and reliable systems.
Respect pre-specified constraints.
Meet pre-specified standards of performance.
Work in pre-specified situations.
Distinction between the design/production phase and the operational phase.
Does not allow any type of adaptation and self-organization in the operational phase.
Adaptive controllers do follow this two-phase.
Adaptation only in the superficial sense of parameter adjustment.

17. Complexity and Complex Systems
“I think the next century will be the century of complexity.”
Stephen Hawking for the San Jose Mercury News on January 23, 2000.
“Complex systems science emerged from the interplay of physics, mathematics, biology, economy, engineering, and computer science, with the mission to overcome the simplifications and idealizations that lead to unrealistic models in these sciences” (Chu et al., 2003).
“...(Complex systems) landscape is bubbling with activity, and now is the time to get involved. Engineering should be at the center of these developments...” (Ottino, J. M. 2004).

18. Complex Systems
System with a large number of components and interconnections, interactions or interdependencies that are difficult to describe, understand, predict, manage, design, and/or change (C.L Magee, O.L de Weck, MIT)
System whose behavior is not deducible, nor may it be inferred from the structure and behavior of its component parts (Bar-Yam, 2003).
Characteristics:
Property of self-organization or emergence of structure from the interaction between the constituent parts of the system.
Self-organization: spontaneous appearance of large-scale organization through limited interactions among parts of the system.
Intrinsically related to notions such as complex adaptive systems, emergence, and complexity.

19. Complicated vs. Complex
Complicated systems:
Systems whose behavior can be completely understood through functional decomposition.
Ex: Microprocessor
Carefully designed and tested: works as documented.
Efficiency dependent on all components.
Increase in performance needs engineering redesign.
Complex systems:
Systems that exhibit emergent behavior.
Ex: U.S. economy
No one designed U. S. economy; yet it works.
Efficiency is robust to perturbations and failures.
Grows (shrinks) on its own, without any explicit control.

20. Complex Systems Complicated vs. Complex
Complicated Systems
Complex Systems

21. Complex Systems
©New England Complex Systems Institute, Cambridge, Massachusetts

22. Complex Systems
Complex systems are systems that exhibit emergent behavior:
Anthills
Human societies
Financial Markets
Climate
Nervous systems
Immune systems
Cities
Galaxies
Modern telecommunication infrastructures

23. Complex Engineered Systems
Man-made complex systems that exhibit emergent behavior while retaining important systems engineering characteristics such as robustness, reliability, and fault-tolerance.
Examples: air and ground transportation networks, distributed manufacturing environments, distributed supply networks, financial markets, healthcare delivery.
In the large-scale systems domain, complex systems are the rule not the exception.
Complex systems can only be engineered by intervention, not by specification and development.
→The system is no longer designed, it is engineered to evolve, adapt, and reconfigure as needed.

24. Characteristics of a complex system?
A complex system displays some or all of the following characteristics:
Agent-based
Basic building blocks are the characteristics and activities of individual agents
Heterogeneous
The agents differ in important characteristics
Dynamic
Characteristics change over time, usually in a nonlinear way; adaptation
Feedback
Changes are often the result of feedback from the environment
Organization
Agents are organized into groups or hierarchies
Emergence
Macro-level behaviors that emerge from agent actions and interactions

25. Attributes for complex system?
Interdependent
Independent
Distributed
Cooperative
Competitive
Adaptive

26. Complex systems
The fundamental characteristic of a complex system is that it exhibits emergent properties: Local interaction rules between simple agents give rise to complex pattern and global behavior

27. The concept of Emergent Behavior
the notion of emergence appeared with the general systems theory and is linked to complexity.
“The global behavior is greater than sum of the behaviors of the individual parts”
The global emergence characterizes the properties of a system that are new ones.

28. Complex Systems perspectives
Complex Systems as a Science to Understand Nature
Complex Systems as a New Form of Engineering

29. Engineering Complex Systems – Key idea
The idea of nature inspired computing arose from the observations and questions regarding natural emergent phenomena:
How do ants build their colonies, without any central control?
How do termites build their nests, without an architect that draws a plan?
How do species evolve without any direction or imposed objective function?

30. Complex Systems Engineering
How can we understand and make use of these emergent phenomena to develop new ways of generating computer programs?
Can we build self-adapting, self-organizing, and evolving computer systems and programs?

31. Complex Systems A new form of Engineering
Engineering which faces complex problems using simple dynamic systems (vast parallelism, robustness, locality).
« The whole is too much greater than the sum of the parts »

32. What is A System of Systems?
A System of Systems (SoS) is a “super-system” made up of elements – each of which is itself a complex, independent system -- that interact to achieve a common goal.
SoS elements (i.e., the systems) can and do operate independently.
An SoS evolves – functions are added/removed/changed with experience.
An SoS exhibits emergent behavior not attributable to any element (system).
An SoS is geographically distributed – elements exchange information only.
-- Are These SoS? --
YES
YES
Systems-of-systems should be distinguished from large but monolithic systems by the independence of their components, their evolutionary nature, emergent behaviors, and a geographic extent that limits the interaction of their components to information exchange . ...
... Five principal characteristics are useful in distinguishing very large and complex but monolithic systems from true systems-of-systems.
Operational Independence of the Elements: If the system-of-systems is disassembled into its component systems the component systems must be able to usefully operate independently. The system-of-systems is composed of systems which are independent and useful in their own right.
Managerial Independence of the Elements: The component systems not only can operate independently, they do operate independently. The component systems are separately acquired and integrated but maintain a continuing operational existence independent of the system-of-systems.
Evolutionary Development: The system-of-systems does not appear fully formed. Its development and existence is evolutionary with functions and purposes added, removed, and modified with experience.
Emergent Behavior: The system performs functions and carries out purposes that do not reside in any component system. These behaviors are emergent properties of the entire system-of-systems and cannot be localized to any component system. The principal purposes of the systems-of-systems are fulfilled by these behaviors.
Geographic Distribution: The geographic extent of the component systems is large. Large is a nebulous and relative concept as communication capabilities increase, but at a minimum it means that the components can readily exchange only information and not substantial quantities of mass or energy.
Mark W. Maier, Architecting Principles for Systems-of-Systems, UA Huntsville, NO
Major League Baseball
Hardware
Oughta Be
International Air Travel
Joint Theater Ops
From INCOSE

33. Areas of Application
Integrated Deepwater System – Optimize and rejuvenate the force structure of the U. S. Coast Guard
Future Combat Systems – Define an optimal Brigade Force structure
P8A – Integrating a new aircraft into the fleet
SBINet- Determine optimal coverage and coordination between sensor, communications, C2.
General Navy Applications – formation of a carrier task force
Air Force Applications – formation of an air wing
From INCOSE

34. Source: Monica Farah-Stapleton, IEEE SOS conference, 2006
Army SOS Perception

35. Artificial Complex Systems

36. Artificial Complex Systems
Artificial Neural Networks
Cellular Automata
Swarm Intelligence
Evolutionary Computing
Quantum Computing
DNA/Molecular Computing
Artificial Life
Artificial Immune System

37. Artificial Neural Networks

38. Developing Intelligent Program Systems
Neural Nets
Artificial Neural Networks: Artificial Neural Networks are crude attempts to model the highly massive parallel and distributed processing we believe takes place in the brain.
Two main areas of activity:
Biological: Try to model biological neural systems.
Computational: develop powerful applications.

39. Developing Intelligent Program Systems
Neural Nets
Neural nets can be used to answer the following:
Pattern recognition: Does that image contain a face?
Classification problems: Is this cell defective?
Prediction: Given these symptoms, the patient has disease X
Forecasting: predicting behavior of stock market
Handwriting: is character recognized?
Optimization: Find the shortest path for the TSP.

40. Cellular Automata

41. Cellular Automata Cell State = empty/off/0 State = filled/on/1
The CA space is a lattice of cells (usually 1D, 2D, 3D) with a particular geometry.
• Each cell contains a variable from a limited range of values (e.g., 0 and 1).
• All cells update synchronously.
• All cells use the same updating rule, depending only on local relations.
• Time advances in discrete steps.

42. Game of Life: 2D Cellular Automata using simple rules
neighboring values
First Generation
Conway’s Life: Rules
A living cell with neighbors dies of isolation
A living cell with 4+ 8-neighbors dies from overcrowding
A cell with 3 living neighbors becomes a living cell
All other cells are unaffected
Second Generation
Emergent pattern: Blinker

43. Game of Life: emergent patterns
Gosper’s glider gun : emits glider stream
Conway’s Rules: Game of Life
Survive with 2 – 3 living neighbors
Generate with 3 living neighbors
gliders: patterns that moves constantly across the grid

44. Emergent Patterns

45. Emergent Patterns: A Clock
See Demo: Game of Life

46. Artificial Life

47. Artificial Life
Artificial Life: An attempt to better understand “real” life by in-silico modeling of the entities we are aware of.
Motivations:
The Emergent properties in life motivate scientists to explore the possibility of artificially creating life and expecting the unexpected.
An Emergent property is created when something becomes more than sum of its parts.

48. Boids of Craig Reynolds
Bird Flocking
“Boids” model was proposed by Reynolds
Boids = Bird-oids (bird like)
Only three simple rules

49. Flocking Model
The model consists in three simple forces (steering behaviors):
Separation: steer to avoid crowding local flockmates
Alignment: steer towards the average heading of local flockmates
Cohesion: steer to move toward the average position of local flockmates

50. Boids of Reynolds
simple rules give rise to complex behavior

51. Collision Avoidance
Rule 1: Avoid Collision with neighboring birds (separation)

52. Velocity Matching: Alignment
Rule 2: Match the velocity of neighboring birds (Alignment)

53. Flock Centering: Cohesion
Rule 3: Stay near neighboring birds (Cohesion)

54. Characteristics Simple rules for each individual No central control
Decentralized and hence robust
Emergence: a system made up of discrete birds yet the overall motion is coordinated.

55. Boids of Reynolds
Boids of Craig Reynolds

56. Emergence in complex systems
Boids of Craig Reynolds

57. Emergence in complex systems
Boids of Craig Reynolds

58. Swarm Intelligence Ant colony optimization Particle Swarm Optimization

59. Ant Colony Optimization

60. Shortest path discovery
Ants get to find the shortest path after few minutes …

61. Ant Colony Optimization
Each artificial ant is a probabilistic mechanism that constructs a solution to the problem, using:
Artificial pheromone deposition
Heuristic information: pheromone trails, already visited cities memory …

62. TSP Solved using ACO

63. Particle Swarm Optimization

64. Particle Swarm Optimization
Particle Swarm Optimization (PSO) mimics the collective intelligent behavior of “ unintelligent ” creatures.
It was developed in 1995 by James Kennedy and Russell Eberhart
Individuals interact with one another while learning from their own experience, and gradually move towards the goal.
It is easily implemented and has proven both very effective and quick when applied to a diverse set of optimization problems.

65. Bird flocking is one of the best example of PSO in nature.
One motive of the development of PSO was to model human social behavior.

66. Algorithm of PSO
Each particle (or agent) evaluates the function to maximize at each point it visits in spaces.
Each agent remembers the best value of the function found so far by it (pbest) and its co-ordinates.
Secondly, each agent know the globally best position that one member of the flock had found, and its value (gbest).

67. Algorithm of PSO
Using the co-ordinates of pbest and gbest, each agent calculates its new velocity as:
vi = vi + c1 x rand() x (pbestxi – presentxi)
+ c2 x rand() x (gbestx – presentxi)
where 0 < rand() <1
presentxi = presentxi + (vi x Δt)

68. Algorithm of PSO

72. Applications

73. New Sorting Algorithms based on Natural Computing

74. Bead-sort
Bead-Sort is a method of ordering a set of positive integers by mimicking the natural process of objects falling to the ground, as beads on an abacus slide down vertical rods. The number of beads on each horizontal row represents one of the numbers of the set to be sorted, and it is clear that the final state will represent the sorted set.
[ ]
[ ]

75. Bead-sort Extended
The "extended" (anti-gravity) mode allows the inclusion of all integers, with "negative beads" rising while "positive beads" fall.

76. Image processing using Quantum Computing and Reverse Emergence

77. The proposed method
The objective is to solve image processing tasks by emergence using cellular automata.
Cellular automata
filtered image
Noisy image

78. The proposed approach
• The problem at hand is which simple rules provide the desired complex behavior?
Solution: Reverse emergence

79. Reverse Emergence
To find simple rules that give rise to the desired complex behavior:
Searching simple rules by hand (trials).
Taking inspiration from nature (bees, ants, termites, spiders …)
Optimization and Learning (GA, PSO, QGA, QPSO …)

80. Conclusions

81. Conclusions
We can learn from nature and take advantage of the problems that she has already solved.
Many simple individuals interacting with each other can make a global behavior emerge.
Techniques based on natural collective behavior are interesting as they are cheap, robust, and simple.
They have lots of different applications.
Simple individuals which interact locally give rise to very complex behavior.

82. References

83. References
Jay Xiong, New Software Engineering Paradigm Based on Complexity Science, Springer
Claudios Gros : Complex and Adaptive Dynamical Systems. Second Edition, Springer,
Blanchard, B. S., Fabrycky, W. J., Systems Engineering and Analysis, Fourth Edition, Pearson Education, Inc., 2006.
Braha D., Minai A. A., Bar-Yam, Y. (Editors), Complex Engineered Systems, Springer, 2006
Gibson, J. E., Scherer, W. T., How to Do Systems Analysis, John Wiley & Sons, Inc.,
International Council on Systems Engineering (INCOSE) website (
New England Complex Systems Institute (NECSI) website (
Rouse, W. B., Complex Engineered, Organizational and Natural Systems, Issues Underlying the Complexity of Systems and Fundamental Research Needed To Address These Issues, Systems Engineering, Vol. 10, No. 3, 2007.

84. References
Wilner, M., Bio-inspired and nanoscale integrated computing, Wiley, 2009.
Yoshida, Z., Nonlinear Science: the Challenge of Complex Systems, Springer 2010.
Stephen Wolfram, A New Kind of Science, Art-Quality Printing, 2002 (available online:
Gardner M., The Fantastic Combinations of John Conway’s New Solitaire Game “Life”, Scientific American –123 (1970).
Nielsen, M. A. & Chuang, I. L. ,Quantum Computation and Quantum Information, 3rd ed., Cambridge Press, UK, 2000.

85. Robots Playing Soccer

86. Q & A