2. Chaos and Information Dr. Tom Longshaw SPSI Sector, DERA Malvern longshaw@signal.dera.gov.uk

3. 2 Some background information n DERA is an agency of the MoD n Employs over 8000 scientists n Over 30 sites around the country n Largest research organisation in Europe n SPSI Sector n Parallel and Distributed Simulation

4. 3 Introduction n A further example of chaos u When is a system stable? n Measuring chaos u Energy, entropy and information. n Avoiding chaos when not wanted u How to avoid chaotic programs! n Practical applications of chaos u What use can chaos be?

5. 4 Further reading n Chaos Chaos: Making a new Science, James Gleick, Cardinal(Penguin), London 1987. http://www.around.com n Chaos and Entropy The Quark and the Jaguar, Murray Gell-Mann, Little, Brown and Company, London 1994. n Complexity Complexity, M. Mitchell Waldrop, Penguin, London, 1992. http://www.santafe.edu http://www-chaos.umd.edu/intro.html

6. 5 When is a system stable? n A street has 16 houses in it, each house paints its front door red or green. n Each year each resident chooses a another house at random and paints their door the same colour as that door. n Initially there are 8 red and 8 green. n Is this system stable…?

7. 6 What controls the chaos? n If we increase the size of the population (number of houses) does the system become more stable? n If we increase the sample size (e.g. look at 3 of our neighbours) does the system become more stable?

8. 7 Sample results n Varying the population size

9. 8 n Varying the sample size

10. 9 n Varying both together

11. 10 Why is the system unstable?

12. 11 Potential Energy n The “potential to change” Phase change Initial state

13. 12 Landscapes of possibility n Watersheds...

14. 13 Chaos and Entropy n Chaos and entropy are synonymous. n Entropy was originally developed to describe the chaos in chemical and physical systems. n In recent years entropy has been used to describe the ratio between information and data size.

15. 14 Information n Measuring the ratio of information to bits. n 00000000000000000000000000 u Low (0) information n 01010101010101010101010101 u Little information n 01011011101111011111011111 u More information n 01101001110110000101101110 u Random (0 information!)

16. 15 Measuring information n Shannon entropy (1949) u The ability to predict based on an observed sample. n Algorithmic Information Content (Kolmogorov 1960) u The size of program required to generate the sample n Lempel-Ziv-Welch (1977,1984) u The zip it and see approach!

17. 16 When is a system stable? n When there is insufficient energy in the system for the system to change its current behaviour. n Paradoxically such systems are rarely interesting or useful. Complexity SimplicityTotal randomness

18. 17 Characteristics of a chaotic system n Unpredictability n Non-linear performance n Small changes in the initial settings give large changes in outcome u The butterfly effect n Elegant degradation n Increased control increases the variation

19. 18 What makes a chaotic system? n Non-Markovian behaviour. n Positive feedback: state(n) affects state(n+1). n Any evolving solution. n Simplicity of rules, complex systems are rarely chaotic, just unpredictable. n Complex systems often hide simple chaotic systems inside.

20. 19 Dealing with chaos n Avoid programming with integers! n Avoid “while” loops n Add damping factors u Observers and pre-conditions n Add randomness into your programs!

21. 20 Practical Applications n “Modern” economic theory [Brian Arthur 1990] n Interesting images and games u Fractals, SimCity, Creatures II n Genetic algorithms n Advanced Information Systems n “Immersive simulations”

22. 21 Information Systems n Conventional database store data in a orderly fashion. n Reducing the data to its information content increases the complexity of the structure… n … but it can be accessed much faster, and some queries can be greatly optimised.

23. 22 smallWorlds n Developed to model political and economic situations n Difficult to quantify n Uses fuzzy logic and tight feedback loops If demand is high then price increases. If price is high then retailers grow. If supply is high then price decreases. If price is low then retailers shrink.

24. 23 Conclusions n Computer scientists should recognise chaotic situations. n Chaos can be avoided or forestalled. n Chaos is not always “bad”. n Sometimes a chaotic system is better than the alternatives.