1. WWW.OPENABM.ORG1 Changing the rules of the game: experiments with humans and virtual agents Marco Janssen School of Human Evolution and Social Change, School of Computing and Informatics, Center for the Study of Institutional Diversity In cooperation with: ASU: Allen Lee, Deepali Bhagvat, Marty Anderies, Sanket Joshi, Daniel Merritt, Clint Bushman, Marcel Hurtado, Takao Sasaki, Priyanka Vanjari, Christine Hendricks Indiana University: Elinor Ostrom, Robert Goldstone, Fil Menczer, Yajing Wang, Muzaffer Ozakca, Michael Schoon, Tun Myint, David Schwab, Pamela Jagger, Frank van Laerhoven, Rachel Vilensky Thailand: Francois Bousquet, Kobchai Worrapimphong, Chutapa Khunsuk, Sonthaya Jumparnin, Pongchai Dumrongrojwatthana Colombia: Juan-Camilo Cardenas, Daniel Castillo, Jorge Maldonado, Rocio Moreno, Silene Gómez, Maria Quintero, Rocio Polania, Sandra Polania, Adriana Vasquez, Carmen Candelo, Olga Nieto, Ana Roldan, Diana Maya

2. WWW.OPENABM.ORG2 The commons dilemma Dilemma between individual and group interests –Group interest: cooperation –Individual interest: free riding on efforts of others Public goods and common pool resources Expectation with rational selfish agents –No public goods –Overharvesting of common pool resources But, many empirical examples of self-governance

3. WWW.OPENABM.ORG3 Repeated interactions Face-to-Face communication Information on past actions on participant Monitoring and sanctioning by subjects themselves Diversity in motivation: Not all humans are selfish and rational What contributes to cooperation in commons dilemmas? (based on research with artificial agents and humans) But problem is not binary: cooperate or defect. Important is defining the rules of the games and enforcing them.

4. WWW.OPENABM.ORG4 Grammar of Rules Rules are defined as shared understanding about enforced prescriptions, concerning what actions (or outcomes) are required, prohibited, or permitted (Ostrom, 2005). Rules in use vs rules on paper Formal rules vs informal rules (formal rules have explicit consequences defined for when the rules are broken (sanctions) and can be enforced by a third party)

5. WWW.OPENABM.ORG5 Puzzles In what way do users of a common resource change the rules? What makes communication effective? How do this relate to experience? And to ecological dynamics?

6. WWW.OPENABM.ORG6 Combining experiments and agent- based models Traditionally agent-based models on cooperation very abstract Experiments in lab and field challenge simplistic models of behavior Micro-level data to test models Going back and forth between experiments and modeling may stimulate theory development

7. WWW.OPENABM.ORG7 Common research questions Laboratory experiments models Field experiments models “role games” Statistical analysis Surveys Interviews Artificial worlds Statistical analysis, Surveys Text analysis,..

8. WWW.OPENABM.ORG8 Field experiments 3 types of games in 3 types of villages in Thailand and Colombia Pencil and paper experiments First 10 rounds: open access Voting round: 3 types of rules: lottery, rotation, private property Survey on rule options Second set of 10 rounds with chosen rule Survey In depth interviews with a few villagers

9. WWW.OPENABM.ORG9 Field experiments (2) Fishery game: –where to fish (A,B) –how much effort Irrigation game (different position; upstream): –How much investment in public good (water) –What amount to take from (remaining) water Forestry game: –How much harvest

10. WWW.OPENABM.ORG10 Fishery village (Baru) Water irrigation village (Lenguazaque) Logging village (Salahonda)

11. WWW.OPENABM.ORG11 Phetchaburi river Forest village Irrigation village Fishery village

12. WWW.OPENABM.ORG12 Rule choice

13. WWW.OPENABM.ORG13 Forestry game

14. WWW.OPENABM.ORG14 Fishery game

15. WWW.OPENABM.ORG15 Irrigation game

16. WWW.OPENABM.ORG16

17. WWW.OPENABM.ORG17 Laboratory experiments Various spatially explicit real-time virtual environments for small groups. Various rounds Treatments include different options of rule choice and/or participants chat on informal rules

18. WWW.OPENABM.ORG18 Experiments from Spring 2007 Renewable resource, density dependent regrowth Resource is 28x28 cells 4 participants Duration round 4 minutes First round is individual round (14x14 cells) Text chat between the rounds Option to reduce tokens of others at the end of each round (at a cost) Explicit and implicit mode Different resource growth experiments: Low growth (6 groups) High growth (4 groups) High / Low growth (6 groups) Mixed growth (6 groups)

19. WWW.OPENABM.ORG19 Tokens in the resource during the rounds Low Mixed High High-Low

20. WWW.OPENABM.ORG20 Average number of tokens collected (blue) and left over (red) for the 5 rounds H L HLMix

21. WWW.OPENABM.ORG21 Text analysis Coding the text: kind of rules, making sure people understand agreement, off-topic chat, meaning of experiment, etc. Is there a relation between the type of conversation and the performance of the group? We would expect that groups who are more explicit on the rules and make clear people understand it do better. In some groups there is a clear dominance of one person, how does this affect the outcome?

22. WWW.OPENABM.ORG22 Initial results

23. WWW.OPENABM.ORG23 Models of Rule changes Laboratory experiments will give us basic empirical information to develop agent-based model. ABM will be used to explore rule evolution is agents adjust rules

24. WWW.OPENABM.ORG24 Reasons for making a model of the experimental data Testing alternative assumptions of behavior ( compare model with naïve models) Methodological challenge: What do we mean with calibrating an agent-based model? Future option: experiments with artificial agents and humans Using the “informed” agent-based model for exploring theoretical questions in an artificial world.

25. WWW.OPENABM.ORG25 Model outline Timestep: 1 second. Actions: move and harvest (explicit mode) Each agent has a basic default speed (moves per second), and number of moves can vary a little bit between seconds. Define direction (target): –the more nearby a token is to the agent, the more valuable –the more nearby a token is to the current target, the more valuable –the more other agents nearby a token, the less valuable –tokens who are straight ahead in the current path of direction of the agent are more valuable. Harvest (expl mode); probabilistic choice depending on number of tokens nearby

26. WWW.OPENABM.ORG26 Testing the model Calibration on multiple metrics using genetic algorithms Comparing calibrated model with naïve models (random movement; greedy agents, no heterogeneity) Turing tests

27. WWW.OPENABM.ORG27 Towards a theoretical model of the evolution of rules Artificial world where agents play many rounds and adjust the rules of the game. What kind of rule sets will evolve? Are there attractors of rule sets? How is this dependent on the ecological dynamics? How is this dependent on the rule to change the rules (constitution)?

28. WWW.OPENABM.ORG28 Coding rules Grammar of Institutions (Crawford and Ostrom, 1995) Rules is build up from 5 components: –Attributes (characteristics of the agents) –Deontic: may/must/must not –Aim: action of the agent –Conditions: when, where and how –Or else: sanctions when not following a rule

29. WWW.OPENABM.ORG29 Process of constructing a rule from the libraries IF “other agent” in “my area” it MUST NOT “collect tokens” ELSE “penalty”

30. WWW.OPENABM.ORG30 Rule space based on experiments (Not yet in building blocks) Explicit mode required of not Start time harvesting Time left before “going crazy” Spatial allocation (none, corners, horizontal, vertical) Speed limit

31. WWW.OPENABM.ORG31 Including monitoring and sanctioning Monitoring: –None –One monitor who cannot harvest are receives a quarter of the income –Everybody monitors, and sanctioning is costly –Monitoring rotates every x seconds (when monitoring one cannot harvest)

32. WWW.OPENABM.ORG32 Tinkering the rules After every round agents update their preferences for rules (reinforcement learning), propose which rule set for next round, after which one of the proposed rule sets is chosen and implemented.

33. WWW.OPENABM.ORG33 Agents breaking rules Agents can break rules. If an action is not allowed, it might break a rule with a probability related to the opportunities available (amount of tokens available nearby)

34. WWW.OPENABM.ORG34

35. WWW.OPENABM.ORG35 Distribution of total earnings (100 evolutions of 100 rounds)

36. WWW.OPENABM.ORG36 Initial experiments Multiple (100) runs with 100 rounds with agents who conditionally cheat. Best solution: Low growth (one)Low growth (everybody) Speed limit7.55 ModeExplNot expl BoundariesVertical Start-time90110 Time to go crazy210140 Earnings (tokens)337409

37. WWW.OPENABM.ORG37 From ABM back to experiments Further analysis may provide us expectations of outcomes for experiments with human participants. Additional experiments can be done to test those.

38. WWW.OPENABM.ORG38 Areas to explore in model analysis Do clusters of rules evolve? And do these clusters change with different tendencies of agents breaking the rules. Co-evolution of cheating behavior and rules (incl. monitoring/sanctioning) What are path-dependent trajectories? What if growth rates change between rounds? How will this affect the evolved rule sets? How will differences in constitutional rules will affect the ability to derive high performance.

39. WWW.OPENABM.ORG39 Concluding remarks Combining agent-based models with experiments in the field and the lab. The aim is not to make predictive models, but theoretical models grounded in empirical observations. Challenges: –Calibration of agent-based models (multiple metrics) –Modeling communication –Large scale controlled experiments with humans

40. WWW.OPENABM.ORG40 Questions?