Presentation is loading. Please wait.

Presentation is loading. Please wait.

Norman Lee Johnson Chief Scientist Referentia System Inc. Honolulu Hawaii CollectiveScience.com Complexity: A Diverse Description.

Similar presentations


Presentation on theme: "Norman Lee Johnson Chief Scientist Referentia System Inc. Honolulu Hawaii CollectiveScience.com Complexity: A Diverse Description."— Presentation transcript:

1

2 Norman Lee Johnson Chief Scientist Referentia System Inc. Honolulu Hawaii CollectiveScience.com Complexity: A Diverse Description ONR Workshop on Human Interactions in Irregular Warfare as a Complex System Apr 2011

3 Diversity Star Wars Novel fusion device Novel diesel engine Hydrogen fuel program P&G – Diapers Large scale Epidemiology – Flu modeling Biological threat reduction Bio-Risk assessment Cyber security Future of the internet Self-organizing collectives Diversity and collective intelligence – Finance applications Effects of rapid change – Finance applications Group identity dynamics – Coexistence applications Leadership models Bio-cyber analogies My Background

4 Counterinsurgency in Iraq: Theory and Practice, 2007 David Kilcullen Available online at:

5 © David J. Kilcullen, 2007 Caveat: the logic of field observation in Iraq +Everyone sees Iraq differently, depending on when they served there, what they did, and where they worked. The environment is highly complex, ambiguous and fluid It is extremely hard to know what is happening – trying too hard to find out can get you killed…and so can not knowing “Observer effect” and data corruption create uncertainty, and invite bias Knowledge of Iraq is very time-specific and location-specific Prediction in complex systems (like insurgencies) is mathematically impossible…but we can’t help ourselves, we do it anyway +Hence, observations from one time/place may or may not be applicable elsewhere, even in the same campaign in the same year: we must first understand the essentials of the environment, then determine whether analogous situations exist, before attempting to apply “lessons”.

6 Diversity Levels of Social complexity slime molds “low” social insects “high” social insects social mammals “low” apes “high” apes humans Identity, diverse, decentralized, collective survival and problem solving Collectively adaptable, self-organizing, emergent properties Individual Self-awareness & Consciousness Collective: Memory, intelligence, deception, tools Individual: High intelligence, deception & emotions, tool making From a workshop on “The Evolution of Social Behavior” which covered a wide range of social organisms Example: All social organism when stressed are “programmed” to copy the behavior of others in the “organism”

7 Complex Human Dynamics: Three Challenges System Analysis How different analysis perspectives can simplify the “complexity” of the system attributes or dynamics? Behavioral-Social Models How abstract models can help simplify the “complexity” of individuals components OR provide relationships between components?

8 © David J. Kilcullen, 2007 “Getting it” is not enough “[This] is a political as well as a military war…the ultimate goal is to regain the loyalty and cooperation of the people.” “It is abundantly clear that all political, military, economic and security (police) programs must be integrated in order to attain any kind of success” Gen William C. Westmoreland, COMUSMACV, MACV Directive 525-4, 17 September 1965 Understanding by leaders is not enough: everyone needs to understand, and we need a framework, doctrine, a system, processes and structures to enact this understanding. Understanding by leaders is not enough: everyone needs to understand, and we need a framework, doctrine, a system, processes and structures to enact this understanding.

9 Complex Human Dynamics: Three Challenges System Analysis How difference perspectives can simplify the “complexity” of the system attributes or dynamics? Behavioral-Social Model How abstract models can help simplify the “complexity” of individuals components OR provide relationships between components? Tools for Decision Makers How the above understandings lead to actionable knowledge for decision makers?

10 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Behavioral-Social Model Features Individual behavior models Tools for decision makers

11  Habitual repetition:  Classical conditioning theory (Pavlov), Operant conditioning theory (Skinner)  Individual optimization of decision:  Theory of reasoned action (Fishbein & Ajzen), Theory of planned behavior (Ajzen)  Socially aware:  Social comparison theory (Festinger), Group comparisons (Faucheux & Mascovici)  Social imitation:  Social learning theory (Bandura), Social impact theory (Latané), Theory of normative conduct (Cialdini, Kalgren & Reno) CONSUMAT model -- Marco Janssen & Wander Jager – Netherlands Individual preference + Social drives + Options + Rationality = ?

12 What drives the changes? RepeaterDeliberator ImitatorComparer Satisfied Dissatisfied Uncertain Certain Historical comparison Increased stress

13 Individual Behavior + Network = Global Dynamics 1000 Consumers with the same behavioral tendency buying 10 products on a small-world network

14 Population of “Repeaters” - satisfied and certain Few products of equal distribution - highly stable Repeater Closest to “Rest state”

15 Population of “Imitators” - satisfied but uncertain Few products of unequal distribution - highly stable Imitator Transitional individual => social state

16 Population of “Deliberators” - dissatisfied but certain High volatility on all products Deliberator Closest to Homo Economicus High rationality, low social

17 Population of “Comparers” - dissatisfied & uncertain Volatility over long times on few products But difficult to maintain - high energy state Comparers Social and Rational

18 “habitual” agent Highly stable with sustained diversity Homo Economicus High volatility Social and Rational Longer time volatility - difficult to sustain Socially driven Highly stable - decreased diversity RepeaterDeliberator ImitatorComparer

19 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global simple models can lead to complex global behavior Study of system thresholds Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Importance of habitual behavior & individual threshold transitions Tools for decision makers Focus on system thresholds first rather than quantifying the details between threshold states

20 Rat Studies of Maximum Carrying Capacity Social order system can carry 8 times the optimal capacity before going over the threshold. NIMH psychologist John B. Calhoun, 1971 Control group - no “rules” => Your worst nightmare One “social” rule => Cooperative social structure Both systems loaded to 2 1/2 times the optimal capacity.

21 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Importance of habitual behavior & individual threshold transitions Tools for decision makers Focus on system threshold changes first

22 Diversity Simple Ant Foraging Model Key concepts: Emergence, Productivity, Diversity, Structure Key concepts: Emergence, Productivity, Diversity, Structure Using NetLogo Collective information Evaporation Diffusion Agent internal state: Current direction Have food? Three rules of action: Carry food Drop food Search Collective information Evaporation Diffusion Agent internal state: Current direction Have food? Three rules of action: Carry food Drop food Search n “Productive men” n “Salaried men” n Innovator n Collective structure n “Productive men” n “Salaried men” n Innovator n Collective structure Nest Food supply

23 Diversity Quantified Environmental Change Infinite source moves at a fixed radius and fixed angular velocity

24 Diversity Slowly changing environment Productivity is only slightly less than an unchanging source Herd effect allows for quick utilization of new resource location Innovators are important at all times by sustaining optimal performance of the collective

25 Food Production Rate Effect of Rate of Change on System Development

26 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Emergent behavior via multi-level analysis Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Tools for decision makers Focus on system threshold changes first

27 Diversity Structural Efficiency - Boom and Bust Lower average production  crash avoidance strategy Bust is proceeded by increased production Greater minimums and maximum when compared to extreme rates!

28 Diversity Total production (units of food) Time (time units) For the slowest rate of change transient structure sustained structure Combination of Sustained Structure and Change How does the retention of structure change the collective response? Suggests that fixed evolutionary adaptations lead to inefficiencies in the presence of even small rates of change What would be the effect of a faster worker? What would be the effect of mass communication?

29 Prigogine’s Laws of stasis, change and evolution: Original observations for chemical systems 1.Equilibrium states are an attractor for non-equilibrium states. 2.System near equilibrium cannot evolve spontaneously to generate spatial−temporal (dissipative) structures 3.As the system is driven far from equilibrium, it may become unstable and generate spatial−temporal structure from nonlinear kinetic processes associated with flows of matter and energy. 4.The possibility of new structures is determined by the system size. 5.Bifurcation introduces “history” into the model (trajectories are replaced by processes). Every description of a system which has bifurcations will imply both deterministic and probabilistic elements. Generalized observations 1.Perturbed systems will return to their normal state. 2.The mechanisms for permanent change are not accessible to systems near equilibrium. 3.Bang on a system hard enough, existing structures can be replaced by new structures. 4.The evolution of new structures are limited by system size. 5.Multiple outcomes change certainty into probabilities, and require a fundamentally different approach.

30 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Emergent behavior via multi-level analysis Optimization vs. robustness Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Individual and collective structure Tools for decision makers Focus on system threshold changes first

31 Structure in a system increases over time for decentralized, self-organizing collectives (nature, societies, technologies) Structure (e.g., the rules required to “run” the system) Time Structure declines because the number of new rules are limited by past rules. Structure increases first by components developing structure Structure increases rapidly as components build structure together

32 Diversity The Structure of Structures Structures direct the evolution of the system by creating and limiting potential options Their definition depends on the time constant of exogenous/endogenous change.

33 Options around Structure also change Structure (the rules required to “run” the system) Time Little initial structure means few Options Options are greatest when structure connects the components These ideas are captured by researchers studying “infodynamics” Options are the free choices both created and limited by the structure (example: the rules of chess create an “environment” where many options are possible - while also limiting what choices are available) Options Options are reduced as more structure restricts options

34 Difference between Options and Diversity Structure Time Diversity and Options are low Diversity and Options are high These ideas are captured by researchers studying “infodynamics” Diversity is the the unique variety in the system Options are when the diversity has multiple expressions of differences, often expressed as multiple connectivity in the network Options Diversity may be very high but Options are low

35 Adaptability and Robustness of System Structure Time Diversity Both adaptable and robust These ideas are captured by researchers studying “infodynamics” Robustness is a system-level ability to sustain performance in the presence of change Adaptability is a component level ability to accommodate change Options Not robust or adaptable, but existing components can be rearranged for new features Robustness achieved by component replacement

36 Diversity Collective Response to Environmental Change Unimpeded development Innovators are essential Collective actions lead to inefficiencies Potential system-wide failure Condensed (optimization of collective) Co-Operational (synergism from individuals) Formative (creation of individual features) Featureless Stable “no change” Change slower than collective response Change faster than collective response Change faster than individual response Rate of Environmental Change Stages in Development

37 Diversity Why Care about Structure-Options? Studies of thresholds in structure: – Prigogine’s Laws of Stasis, Change and Evolution – Joseph Schumpeter’s Creative Destruction – Foster and Kaplan "Creative Destruction: Why Companies that are Built to Last Underperform the Market - And how to Successfully Transform Them”, 2001 – John Padgett life’s work on innovation in the Florentine (and world) finance system Dynamic “structural” thresholds do the same

38 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Multi-level analysis & emergence Optimization vs. robustness Interplay of structure and options Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Individual and collective structure Tools for decision makers Focus on system threshold changes first Capture structure and options: What has to be removed before change can occur Diversity is not the same as options!

39 © David J. Kilcullen, 2007 “15. Do not try to do too much with your own hands. Better the Arabs do it tolerably than that you do it perfectly. It is their war, and you are to help them, not to win it for them. Actually, also, under the very odd conditions of Arabia, your practical work will not be as good as, perhaps, you think it is.” T.E. Lawrence, “Twenty-Seven Articles”, The Arab Bulletin, 20 August 1917 Remember article 15

40 Diversity Expert Performance in Finance Why can’t financial experts outperform the S&P 500 “collective” – good + bad – consistently? Professional money managers fail to beat the S&P 500 at an average rate of 70% per year. 90% trail the S&P over a 10-year period. Over decades are only a few – Soros, Miller, …. “These are the people who have more knowledge and more training than the vast majority of investors. And yet, neither the superior knowledge nor the superior experience helps them in the long run.” Bill Mann, TMFOtter

41 The ant colony (and individuals) finds the shortest path Nest Food Nest Food How does it work? Ants Solving “HARD” problems

42 Start End In “Learning” the maze, individuals create a diversity of experience. A Model for Solving Hard Problems How can groups > solve hard problems, > without coordination, > without cooperation, > without selection? The Maze has many solutions > non-optimal and optimal. Individuals > Solve a maze > Independently > Same capability When individuals solve the maze again, they eliminate “extra” loops But because a global perspective is missing, they cannot shorten their path. This is were diversity helps.

43 How collectives find the Shortest path Paths of three ants Collective path Unlike in natural selection, no one individual is the fittest!

44 Ensemble (Averaged) Behavior. Individuals in Collective Decision Normalized number of steps Average Individual Using novice information, with two different collections Using established information Performance correlates with high unique diversity

45 Diversity Expert Performance & Complexity Where Experts Have Value Simple Complex Domain Complexity Value of Experts Michael Mauboussin - Legg Mason Capital Management Value of Collectives Complexity Barrier

46 From a workshop on Complex Science for the Physician’s Alliance

47 Effect of Complexity in Stable Systems Structure (the rules required to “run” the system) time System goes to optimization via “expert” route “Complexity Barrier” requires Collective Solutions X System goes to optimization via “collective” route

48 Collective Error = Average Individual error minus Prediction Diversity “The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies”

49 Diversity Collective Performance Where Experts Have Value Simple Complex Domain Value of Experts Michael Mauboussin - Legg Mason Capital Management Value of Collectives Collective Error = Average Individual error minus Prediction Diversity

50 Diversity Options in infrastructure, societal structure, economies, etc. Collectives in complex environments begin  end In complex domains: People beginning points differ Their final goals may differ But local paths can overlay and find synergy

51 © David J. Kilcullen, 2007 Why counterinsurgency is population-centric +This is not about being “nice” to the population, it is a hard-headed recognition of certain basic facts, to wit: The enemy needs the people to act in certain ways (sympathy, acquiescence, silence, provocation) -- without this insurgents wither The enemy is fluid; the population is fixed – therefore controlling the population is do-able, destroying the enemy is not Being fluid, the enemy can control his loss rate and can never be eradicated by purely enemy-centric means (e.g. Vietnam VC losses) In any given area, there are multiple threat groups but only one local population – the enemy may not be identifiable but the population is. Terrain-centric and enemy-centric actions are still vital and crucial to success. Enemy and Terrain still matter, but Population is the key.

52 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Multi-level analysis & emergence Optimization vs. robustness Interplay of structure and options Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Individual and collective structure Conditions for synergy/conflict Group identity models Tools for decision makers Focus on system threshold changes first Capture structure and options New social consensus tools

53 Diversity - source of conflict or synergy? Diversity can lead to synergy when collectives have: Common goals Common group identity Common worldview (agreement on options), but with different preferences or goals Otherwise, diversity can lead to competition and conflict More restrictive

54 © David J. Kilcullen, 2007 Governance, development, democracy are not universal goods “There is no such thing as impartial governance or humanitarian assistance. In this environment, every time you help someone, you hurt someone else.” General Rupert Smith, Commander UNPROFOR 1995 The enemy will perceive actions by political staff, NGOs, economic and development staffs, PRTs and government officials as a direct challenge to grass-roots control over the population, and will react with violence

55 Diversity Why Care about Group Identity? Social organisms have a strong drive to form group identity: “... experiments show that competition is not necessary for group identification and even the most minimal group assignment can affect behavior. ‘Groups’ form by nothing more than random assignment of subjects to labels, such as even or odd.” Group Identity can be the dominant factor of behavior: “Subjects are more likely to give rewards to those with the same label than to those with other labels, even when choices are anonymous and have no impact on their own payoffs. Subjects also have higher opinions of members of their own group.” Akerlof, G. A. and R. E. Kranton (2000). “Economics and Identity.” Quarterly Journal of Economics 115(3):

56 Identity: Assertions and Definitions Identity Group Identity == Mechanism of Group Immunity Common definition: if someone does something to a person in your identity group, it is the same as if they did it to you. Working definition: Identity is the individual behavioral bond/process that creates a “group self” that has all of properties of an individual self. Assert: Group identity in higher social organisms can be an abstraction that detaches from the origin of the identity group. Despite the long-standing recognition of the importance of identity in social systems, most studies of identity are observations of identity's influence on individual and group behavior, rather than understanding the processes by which identity forms and modifies behaviors.

57 Questions from an Identity Perspective Identity What are the characteristics of identity groups? What are their dynamics? Formation, stability, coalescence, expression, polarization, influence, dissolution How does group identity affect the acceptance, adaptation, spread and exploitation of an idea? Does the rapid spread of an infectious idea (e.g., a fad) require a common group identity? What are the conditions that cause identity groups to become destructive to other groups or to factions within a group? How does a ”leaderless group” group use identity to self-organize? How does insurgent news/press of “violence on self” polarize the “self”? How does diversity affect identity formation? Performance? Stability? As diversity decreases, will tolerance decrease? What are the conditions necessary for coexistence to emerge between identity groups? How does one really build a democratic nation from fractionalized groups? Answers, or at least a beginning of an understanding, of questions like these will help to inform policy regarding intra and international negotiations and other actions designed to bring about the resolution of conflict. It will aid them in identifying unintended consequences of hasty actions, such as the use of extreme force. Or, how to prevent violent conflict and grow the conditions for peace.

58 Characteristics of Identity Groups Identity Groups (IDGs) express a common worldview – an understanding of how the world works: what are options and what is forbidden IDGs have a shared, unspoken knowledge that is typically unknowable outside the IDG IDGs often have symbols of association such as dress or language differences, often unobserved by others IDGs – when in larger and sustained groups – develop culture and civilizations While most of us are born or develop within existing IDGs, we also form many identity groups during our lives.

59 Identity Groups under Stress Stress – a heightened state of anxiety about one’s current state that might originate from outside the IDG (e.g., oppression) or from within (e.g., internal dissension) – can cause the IDG to act as a single organism (“circling up the wagons”) Stressed Identity Groups: Are more likely to reject ideas coming from outside the IDG Oppress, reduce, or prohibit expression of diverse ideas within the IDG Make irrational actions that are potentially self-destructive Can “dehumanize” individuals and groups that represent opposition IDGs Are in a state that can lead to polarization, particularly if “outside” IDGs are well defined, are in opposition and are creating the stress Can be strongly influenced by a leader (or idea!) that represents the IDG, particularly potential “martyrs” that have received the brunt of the oppression or violence from the opposing IDGs.

60 Diversity and Identity Self-Organizing Decentralized IDGs (SODIDs) can have sophisticated information gathering, complex problem solving, collectively organized action, and high performance, predictability and system-wide stability. SODIDs with high expressed diversity typically exhibit these desirable collective attributes – Identity is the mechanism that provides coordination of diversity in social organisms. SODIDs with low expressed diversity often exhibit negative attributes of oscillating performance (boom and bust cycles), failures due to an excessive dominance of one component of the system and unpredictability. The tendency of Identity Groups under stress to repress diversity can results in negative global outcomes. Identity can both enable the quick adaptation of a life-preserving idea or reinforcement of a destructive idea that leads to failure of the system as a whole – which path is taken is dependent on diversity of the group.

61 Application - Tipping Point Law of the Few  Identity largely determines the social network  Identity coherence determines the success of trendsetters to represent the the group and the sticky idea  Trendsetters often span multiple identity groups Stickiness Factor  The “stickiness” strongly correlates with the resonance with identity  Ability to jump across multiple identity groups determines widespread propagation  An idea that is sticky to an opposing identity group will be aborted and demonized.

62 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Multi-level analysis & emergence Optimization vs. robustness Interplay of structure and options Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Individual and collective structure Conditions for synergy/conflict Group identity models Descriptive vs. predictive models Tools for decision makers Focus on system threshold changes first Capture structure and options New social consensus tools Tools are changing - data rich vs. poor Validation requirements

63 Prediction of collective behavior is generally easier at extremes of diversity or variation Diversity and Collective Prediction Low Diversity Low Diversity High Diversity High Diversity Locally and Globally Predictable Globally Predictable Unpredictable

64 How does this translate to distribution functions? Diversity and Collective Prediction Low Diversity Low Diversity High Diversity High Diversity Locally and Globally Predictable Globally Predictable Unpredictable Problem distributions: Discrete distributions Multi-modal distributions Long-tailed distributions (e.g., power law, instead of Gaussian statistics) p(ø) 01

65 Analysis - Increasing levels of discovery: Ontology or qualitative characterization Statistical characterization Dimensionless functionality (correlations) Scaling - self-similarity – fractal structure Descriptive-predictive “Laws” Functional relationships Static Dynamic (governing equations of change) Higher moments (variation within) Error generation - uncertainty quantification Analysis - Increasing levels of discovery: Ontology or qualitative characterization Statistical characterization Dimensionless functionality (correlations) Scaling - self-similarity – fractal structure Descriptive-predictive “Laws” Functional relationships Static Dynamic (governing equations of change) Higher moments (variation within) Error generation - uncertainty quantification Origin of the model or “The Theory” Structure and options Averages and outliers Data generation Discovery

66 Complex Human Dynamics: Three Solutions System Analysis Connection between local  global Study of system thresholds Developmental perspective Multi-level analysis & emergence Optimization vs. robustness Interplay of structure and options Behavioral-Social Model Features Individual behavior models Drivers & threshold transitions Habitual behavior Emergent problem solving Individual and collective structure Conditions for synergy/conflict Group identity models Descriptive vs. predictive models Tools for decision makers Focus on system threshold changes first Capture structure and options New social consensus tools Tools are changing - data rich vs. poor Validation requirements Cost-benefit assessment with transparency and uncertainty quantification

67 Diversity Norman L. Johnson Norman L. Johnson Questions?

68 Some References on Prediction and Complexity Shalizi, Cosma R., “METHODS AND TECHNIQUES OF COMPLEX SYSTEMS SCIENCE: AN OVERVIEW”, Chapter 1 (pp ) in Thomas S. Deisboeck and J. Yasha Kresh (eds.), Complex Systems Science in Biomedicine (New York:Springer, 2006) Farmer, J. Doyne & John Geanakoplos, “Power laws in economics and elsewhere”, DRAFT April 4, 2005 (chapter from a preliminary draft of a book called “Beyond equilibrium and efficiency”) - Contact the authors for a copy. Farmer, J. Doyne, “Power laws”, Santa Fe Institute Summer School June 29, Contact the author for a copy. Holbrook, Morris B "Adventures in Complexity: An Essay on Dynamic Open Complex Adaptive Systems, Butterfly Effects, Self-Organizing Order, Co-evolution, the Ecological Perspective, Fitness Landscapes, Market Spaces, Emergent Beauty at the Edge of Chaos, and All That Jazz White, Douglas R., “Civilizations as dynamic networks: Cities, hinterlands, populations, industries, trade and conflict”, European Conference on Complex Systems Paris, November ppt/CivilizationsasDynamicNetworksParis.pptEuropean Conference on Complex Systems For exceptional talks on Complexity in financial systems, see the Thought Leaders Forums: for for


Download ppt "Norman Lee Johnson Chief Scientist Referentia System Inc. Honolulu Hawaii CollectiveScience.com Complexity: A Diverse Description."

Similar presentations


Ads by Google