Complex Systems Engineering SwE 488 Artificial Complex Systems Prof. Dr. Mohamed Batouche Department of Software Engineering CCIS – King Saud University Riyadh, Kingdom of Saudi Arabia

Cellular Automata 2

Purpose 3 In Theory: Computation of all computable functions Construction of (also non-homogenous) automata by other automata, the offspring being at least as powerful (in some well-defined sense) as the parent In Practice: Exploring how complex systems with emergent patterns seem to evolve from purely local interactions of agents. I.e. Without a “master plan!”

Cellular Automata A cellular automata is a family of simple, finite-state machines that exhibit interesting, emergent behaviors through their interactions in a population 4

The famous BOIDS model shows how flocking behavior can emerge from a collection of agents following a few simple rules. Emergent Behavior 5

Original concept of CA is most strongly associated with John von Neumann who was interested in the connections between biology and the new study of automata theory Stanislaw Ulam suggested to von Neumann the use a cellular automata as a framework for researching these connections. The original concept of CA can be credited to Ulam, while the early development of the concept is credited to von Neumann. Ironically, although von Neumann made many contributions and developments in CA, they are commonly referred to as “non-von Neumann style”, while the standard model of computation (CPU, globally addressable memory, serial processing) is know as “von Neumann style ”. 6

7 Cellular Automata (CAs) Have been used as: massively parallel computer architecture model of physical phenomena (Fredkin, Wolfram) VLSI Testing Data Encryption Error Correcting Code Correction Testable Synthesis Generation of hashing Function Currently being investigated as model of quantum computation (QCAs)

Grid Mesh of cells. Simplest mesh is one dimensional. Cell Basic element of a CA. Cells can be thought of as memory elements that store state information. All cells are updated synchronously according to the transition rules. Rules 8

Local interaction leads to global dynamics. One can think of the behavior of a cellular automata like that of a “wave” at a sports event. Each person reacts to the state of his neighbors (if they stand, he stands). 9

Rule Application Next state of the core cell is related to the states of the neighboring cells and its current state. An example rule for a one dimensional CA: 011->x0x All possible states must be described. Next state of the core cell is only dependent upon the sum of the states of the neighboring cells. For example, if the sum of the adjacent cells is 4 the state of the core cell is 1, in all other cases the state of the core cell is 0. 10

11 Structure Discrete space (lattice) of regular cells 1D, 2D, 3D, … rectangular, hexagonal, … At each unit of time a cell changes state in response to ( Time advances in discrete steps ): its own previous state states of neighbors (within some “radius”) All cells obey same state update rule, depending only on local relations an FSA Synchronous updating (parallel processing)

Structure: Neighborhoods 12

1-DIMENSIONAL AUTOMATA 13

One-Dimensional CA’s Game of Life is 2-D. Many simpler 1-D CAs have been studied For a given rule-set, and a given starting setup, the deterministic evolution of a CA with one state (on/off) can be pictured as successive lines of colored squares, successive lines under each other 14

Neighborhoods 15

3 Black = White 2 Black = Black 1 Black = Black 3 White = White Now make your own CA 16

“A New Kind of Science” Stephen Wolfram ISBN

1-D CA Example 18 Rules Rule# = = 54

Wolfram Model Rule #126 = = 126 Most of the rules are degenerate, meaning they create repetitive patterns of no interest. However there are a few rules which produce surprisingly complex patterns that do not repeat themselves Rule #124 = =

Wolfram Model we can view the state of the model at any time in the future as long as we step through all the previous states. 20

21 Hundred generations of Rule 30

CA Example: Rule Rule ( ) = 30 22

Conus Textile pattern 23

The pattern is neither regular nor completely random. It appears to have some order, but is never predictable. 24

See Demo - NetLogo Rule #45= = = Rule #30= = = This naming convention of the 256 distinct update rules is due to Stephen Wolfram. He is one of the pioneers of Cellular Automata and author of the book a New Kind of Science, which argues that discoveries about cellular automata are not isolated facts but have significance for all disciplines of science. Wolfram Model 25

Rule ( ) = 90 Wolfram Rule 90 26

Wolfram Rule 110 Proven to be Turing Complete - Rich enough for universal computation interesting result because Rule 110 is an extremely simple system, simple enough to suggest that naturally occurring physical systems may also be capable of universality 27

Wolfram Rule Rule# 99 =

29

Mollusc Pigmentation Patterns 30

Wolfram’s CA classes 1,2 From observation, initially of 1-D CA spacetime patterns, Wolfram noticed 4 different classes of rule-sets. Any particular rule-set falls into one of these:-: CLASS 1: From any starting setup, pattern converges to all blank -- fixed attractor CLASS 2: From any start, goes to a limit cycle, repeats same sequence of patterns for ever. -- cyclic attractors 31

Wolfram’s CA classes 3,4 CLASS 3: Turbulent mess, chaos, no patterns to be seen. CLASS 4: From any start, patterns emerge and continue without repetition for a very long time (could only be 'forever' in infinite grid) Classes 1 and 2 are boring, Class 3 is messy, Class 4 is 'At the Edge of Chaos' - at the transition between order and chaos -- where Game of Life is!. 32

2-DIMENSIONAL AUTOMATA 33

2-dimensional cellular automaton consists of an infinite (or finite) grid of cells, each in one of a finite number of states. Time is discrete and the state of a cell at time t is a function of the states of its neighbors at time t-1. 34

Neighborhoods 35 Von NeumannMoore margolus

Cryptography use, Rule 30 Simulations Gas behaviour. Forest fire propagation. Urban development. Traffic flow. Air flow. Crystallization process. Alternative to differential equations 36

37 Snowflakes

Bead-sort Bead-Sort is a method of ordering a set of positive integers by mimicking the natural process of objects falling to the ground, as beads on an abacus slide down vertical rods. The number of beads on each horizontal row represents one of the numbers of the set to be sorted, and it is clear that the final state will represent the sorted set. 38 [ ][ ]

Bead-sort 39

Bead-sort - CA implementation 40 [ ] [ ] [ ] [ ] See Demo - Netlogo the natural process of objects falling to the ground

Bead-sort Extended CA implementation The Bead-Sort can be modeled by the two-dimensional cellular automaton (CA) rule,. For the Antigravity Bead-Sort, we add a second rule:. 41

Bead-sort Extended 42 The "extended" (anti-gravity) mode allows the inclusion of all integers, with "negative beads" rising while "positive beads" fall.

43 Example: Conway’s Game of Life Invented by Conway in late 1960s A simple CA capable of universal computation Structure: 2D space rectangular lattice of cells binary states (alive/dead) neighborhood of 8 surrounding cells (& self) simple population-oriented rule

44 Example: Conway’s Game of Life Cell State = dead/off/0 State = alive/on/1

A cell dies or lives according to some transition rule The world is round (flips over edges) How many rules for Life? 20, 40, 100, 1000? T = 0T = 1 transition rules Example: Conway’s Game of Life 45

46 State Transition Rule Live cell has 2 or 3 live neighbors  stays as is (stasis) Live cell has < 2 live neighbors  dies (loneliness) Live cell has > 3 live neighbors  dies (overcrowding) Empty cell has 3 live neighbors  comes to life (reproduction)

47 State Transition Rule Survive with 2 or 3 live neighbors Live cell  stays as is (stasis) otherwise dies from loneliness or overcrowding Generate with 3 live neighbors Empty cell  comes to life (reproduction)

48 State Transition Rule Three simple rules: dies if number of alive neighbor cells =< 1(loneliness) dies if number of alive neighbor cells >= 4 (overcrowding) generate alive cell if number of alive neighbor cells = 3(procreation)

Examples of the rules loneliness(dies if #alive =< 1) overcrowding(dies if #alive >= 4) procreation(lives if #alive = 3) State Transition Rule 49

CA: Discrete Time, Discrete Space Initial Setup Number of Neighbors After Pass 1 After Pass 2 50

T = 0 neighboring values T = 1 Game of Life: 2D Cellular Automata using simple rules Emergent pattern: Blinker 51

Emergent patterns Conway automaton can simulate a number of different effects that can be found in the evolution of a living population. Equilibria Oscillation Movement square 2 steps diagonal beehive 2 steps horizontal boat 3 steps instability (all the space is filled up by horizontal lines) ship 15 steps instability toast chaos? 52

Game of Life: emergent patterns 53 gliders: patterns that moves constantly across the grid Conway’s Rules: Game of Life Survive with 2 – 3 living neighbors Generate with 3 living neighbors Gosper’s glider gun : emits glider stream

Emergent Patterns 54

Emergent Patterns Oscillators-objects that change from step to step, but eventually repeat themselves. These include, but are not limited to, period 2 oscillators, including the blinker and the toad. Blinker Toad 55

Emergent Patterns: A Clock 56 See Demo: Game of Life

Emergent Patterns: 57 See Demo: Game of Life Oscillator Barber’s Pole Pulsar SpaceShip Beacon Glider

Emergent Patterns: Gosper’s Glider Gun 58 See Demo: Game of Life

Emergent Patterns: Puffer train 59

Emergent Patterns: Double-Barreled Gun 60

Emergent Patterns: Edge Shooter 61

Emergent Patterns: Evolution of a breeder... 62

Game of Life - implications Typical Artificial Life, or Non-Symbolic AI, computational paradigm: bottom-up parallel locally-determined Complex behavior from (... emergent from...) simple rules. Gliders, blocks, traffic lights, blinkers, glider-guns, eaters, puffer-trains... 63

Game of Life as a Computer ? Higher-level units in Game of Life can in principle be assembled into complex 'machines' -- even into a full computer, or Universal Turing Machine. 'Computer memory' held as 'bits' denoted by 'blocks‘ laid out in a row stretching out as a potentially infinite 'tape'. Bits can be turned on/off by well-aimed gliders. 64

Game of Life as a Computer ? Bits can be turned on/off by well-aimed gliders: 65

This is a Turing Machine implemented in Conway's Game of Life. 66

This is a Universal Turing Machine (UTM) implemented in Conway's Game of Life. Designed by Paul Rendell 10/February/

68 Foundations Turing Machine -> Computer The theory of computation and the practical application it made possible — the computer — was developed by an Englishman called Alan Turing.

Turing machines Turing machines (TM ’ s) were introduced by Alan Turing in 1936 They are more powerful than both finite automata or pushdown automata. In fact, they are as powerful as any computer we have ever built. The main improvement from PDA ’ s is that they have infinite accessible memory (in the form of a tape) which can be read and written to.

Basic design of Turing machine Control Infinite tape Tape head Usual finite state control The basic improvement from the previous automata is the infinite tape, which can be read and written to. The input data begins on the tape, so no separate input is needed (e.g. DFA, PDA).

Turing Machine... Infinite tape: Γ* Tape head: read current square on tape, write into current square, move one square left or right FSM:like PDA (PushDown Automata), except: transitions also include direction (left/right) final accepting and rejecting states FSM

72 A Turing Machine Tape Read-Write head Control Unit (FSM: Finite Sate Machine)

73 The Tape Read-Write head No boundaries -- infinite length The head moves Left or Right

Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

75 Turing Machine EG Successor Program Sample Rules: If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT! Let’s see how they are carried out on a piece of paper that contains the reverse binary representation of 47:

76 Turing Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

77 Turing Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

78 Turing Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

79 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

80 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

81 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

82 Turing Machine EG Successor Program So the successor’s output on was which is the reverse binary representation of = = = = 48

83 Turing Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

84 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

85 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

86 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

87 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

88 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

89 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

90 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT!

91 A Thinking Machine EG Successor Program If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT! The successor of 127 is 128 !!!

03/03/2000FLAC: Reductions92 Turing machines as functions TM Input word Output word f  A function f is a computable function if a TM with word w as input halts with f(w) as output. w f(w)

03/03/2000FLAC: Reductions93 First computers: custom computing machines Eniac: the control is hardwired manually for each problem. Control Input tape (read only) Output tape (write only) Work tape (memory)

03/03/2000FLAC: Reductions94 TMs can be described using text Input tape (read only) Control Output tape (write only) Work tape (memory) Program n states s letters for alphabet transition function: d(q0,a) = (q1, b, L) d(q0,b) = (q1, a, R) …. Consequence: There is a countable number of Turing Machines

The Universal Turing Machine We can build a Turing Machine that can do the job of any other Turing Machine. That is, it can tell you what the output from the second TM would be for any input! This is known as a universal Turing machine (UTM). A UTM acts as a completely general- purpose computer, and basically it stores a program and data on the tape and then executes the program.

03/03/2000FLAC: Reductions96 There is a TM which can simulate any other TM (Alan Turing, 1930) Input tape Universal TM Output tape Work tape (memory) Interpreter Program

03/03/2000FLAC: Reductions97 The Digital Computer: a UTM Machine code Universal TM Output Memory + disk Machine-code Interpreter Input

Palindrome Example - UTM Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

Palindrome Example Figure 2 – Palindrome Example of a TM computation

109 Can Machines Think? In “Computing machinery and intelligence,” written in 1950, Turing asks whether machines can think. He claims that this question is too vague, and proposes, instead, to replace it with a different one. That question is: Can machines pass the “imitation game” (now called the Turing test)? If they can, they are intelligent. Turing is thus the first to have offered a rigorous test for the determination of intelligence quite generally.

110 Self-reproducing CAs von Neumann saw CAs as a good framework for studying the necessary and sufficient conditions for self-replication of structures. von N's approach: self-rep of abstract structures, in the sense that gliders are abstract structures. His CA had 29 possible states for each cell (compare with Game of Life 2, black and white) and his minimum self-rep structure had some 200,000 cells.

111 Self-rep and DNA This was early 1950s, pre-discovery of DNA, but von N's machine had clear analogue of DNA which is both: used to determine pattern of 'body' interpreted and itself copied directly copied without interpretation as a symbol string Simplest general logical form of reproduction (?) How simple can you get?

112 Langton’s Loops Chris Langton formulated a much simpler form of self- rep structure - Langton's loops - with only a few different states, and only small starting structures. [ Further developments -- eg 'Wireworld'. ]

3-Dimensional CA 113 One difficulty with three-dimensional cellular automata is the graphical representation (on two- dimensional paper or screen)

114 Conclusions

115 Conclusions This topic is very hot and has widespread implications Biology Chemistry Computer science Complexity We’ve seen the basic concepts … But we’ve only scratched the surface!  From now on, Think Biology, Emergence, Complex Systems …

References 116

References Jay Xiong, New Software Engineering Paradigm Based on Complexity Science, Springer Claudios Gros : Complex and Adaptive Dynamical Systems. Second Edition, Springer, Blanchard, B. S., Fabrycky, W. J., Systems Engineering and Analysis, Fourth Edition, Pearson Education, Inc., Braha D., Minai A. A., Bar-Yam, Y. (Editors), Complex Engineered Systems, Springer, 2006 Gibson, J. E., Scherer, W. T., How to Do Systems Analysis, John Wiley & Sons, Inc., International Council on Systems Engineering (INCOSE) website ( New England Complex Systems Institute (NECSI) website ( Rouse, W. B., Complex Engineered, Organizational and Natural Systems, Issues Underlying the Complexity of Systems and Fundamental Research Needed To Address These Issues, Systems Engineering, Vol. 10, No. 3,

References Wilner, M., Bio-inspired and nanoscale integrated computing, Wiley, Yoshida, Z., Nonlinear Science: the Challenge of Complex Systems, Springer Gardner M., The Fantastic Combinations of John Conway’s New Solitaire Game “Life”, Scientific American –123 (1970). Nielsen, M. A. & Chuang, I. L.,Quantum Computation and Quantum Information, 3rd ed., Cambridge Press, UK,

119