Changing the rules of the game: experiments with humans and virtual agents Marco Janssen School of Human Evolution and Social Change, School of Computing and Informatics, Center for the Study of Institutional Diversity In cooperation with: ASU: Allen Lee, Deepali Bhagvat, Marty Anderies, Sanket Joshi, Daniel Merritt, Clint Bushman, Marcel Hurtado, Takao Sasaki, Priyanka Vanjari, Christine Hendricks Indiana University: Elinor Ostrom, Robert Goldstone, Fil Menczer, Yajing Wang, Muzaffer Ozakca, Michael Schoon, Tun Myint, David Schwab, Pamela Jagger, Frank van Laerhoven, Rachel Vilensky Thailand: Francois Bousquet, Kobchai Worrapimphong, Chutapa Khunsuk, Sonthaya Jumparnin, Pongchai Dumrongrojwatthana Colombia: Juan-Camilo Cardenas, Daniel Castillo, Jorge Maldonado, Rocio Moreno, Silene Gómez, Maria Quintero, Rocio Polania, Sandra Polania, Adriana Vasquez, Carmen Candelo, Olga Nieto, Ana Roldan, Diana Maya

The commons dilemma Dilemma between individual and group interests –Group interest: cooperation –Individual interest: free riding on efforts of others Public goods and common pool resources Expectation with rational selfish agents –No public goods –Overharvesting of common pool resources But, many empirical examples of self-governance

Repeated interactions Face-to-Face communication Information on past actions on participant Monitoring and sanctioning by subjects themselves Diversity in motivation: Not all humans are selfish and rational What contributes to cooperation in commons dilemmas? (based on research with artificial agents and humans) But problem is not binary: cooperate or defect. Important is defining the rules of the games and enforcing them.

Grammar of Rules Rules are defined as shared understanding about enforced prescriptions, concerning what actions (or outcomes) are required, prohibited, or permitted (Ostrom, 2005). Rules in use vs rules on paper Formal rules vs informal rules (formal rules have explicit consequences defined for when the rules are broken (sanctions) and can be enforced by a third party)

Puzzles In what way do users of a common resource change the rules? What makes communication effective? How do this relate to experience? And to ecological dynamics?

Combining experiments and agent- based models Traditionally agent-based models on cooperation very abstract Experiments in lab and field challenge simplistic models of behavior Micro-level data to test models Going back and forth between experiments and modeling may stimulate theory development

Common research questions Laboratory experiments models Field experiments models “role games” Statistical analysis Surveys Interviews Artificial worlds Statistical analysis, Surveys Text analysis,..

Field experiments 3 types of games in 3 types of villages in Thailand and Colombia Pencil and paper experiments First 10 rounds: open access Voting round: 3 types of rules: lottery, rotation, private property Survey on rule options Second set of 10 rounds with chosen rule Survey In depth interviews with a few villagers

Field experiments (2) Fishery game: –where to fish (A,B) –how much effort Irrigation game (different position; upstream): –How much investment in public good (water) –What amount to take from (remaining) water Forestry game: –How much harvest

Fishery village (Baru) Water irrigation village (Lenguazaque) Logging village (Salahonda)

Phetchaburi river Forest village Irrigation village Fishery village

Rule choice

Forestry game

Fishery game

Irrigation game

Laboratory experiments Various spatially explicit real-time virtual environments for small groups. Various rounds Treatments include different options of rule choice and/or participants chat on informal rules

Experiments from Spring 2007 Renewable resource, density dependent regrowth Resource is 28x28 cells 4 participants Duration round 4 minutes First round is individual round (14x14 cells) Text chat between the rounds Option to reduce tokens of others at the end of each round (at a cost) Explicit and implicit mode Different resource growth experiments: Low growth (6 groups) High growth (4 groups) High / Low growth (6 groups) Mixed growth (6 groups)

Tokens in the resource during the rounds Low Mixed High High-Low

Average number of tokens collected (blue) and left over (red) for the 5 rounds H L HLMix

Text analysis Coding the text: kind of rules, making sure people understand agreement, off-topic chat, meaning of experiment, etc. Is there a relation between the type of conversation and the performance of the group? We would expect that groups who are more explicit on the rules and make clear people understand it do better. In some groups there is a clear dominance of one person, how does this affect the outcome?

Initial results

Models of Rule changes Laboratory experiments will give us basic empirical information to develop agent-based model. ABM will be used to explore rule evolution is agents adjust rules

Reasons for making a model of the experimental data Testing alternative assumptions of behavior ( compare model with naïve models) Methodological challenge: What do we mean with calibrating an agent-based model? Future option: experiments with artificial agents and humans Using the “informed” agent-based model for exploring theoretical questions in an artificial world.

Model outline Timestep: 1 second. Actions: move and harvest (explicit mode) Each agent has a basic default speed (moves per second), and number of moves can vary a little bit between seconds. Define direction (target): –the more nearby a token is to the agent, the more valuable –the more nearby a token is to the current target, the more valuable –the more other agents nearby a token, the less valuable –tokens who are straight ahead in the current path of direction of the agent are more valuable. Harvest (expl mode); probabilistic choice depending on number of tokens nearby

Testing the model Calibration on multiple metrics using genetic algorithms Comparing calibrated model with naïve models (random movement; greedy agents, no heterogeneity) Turing tests

Towards a theoretical model of the evolution of rules Artificial world where agents play many rounds and adjust the rules of the game. What kind of rule sets will evolve? Are there attractors of rule sets? How is this dependent on the ecological dynamics? How is this dependent on the rule to change the rules (constitution)?

Coding rules Grammar of Institutions (Crawford and Ostrom, 1995) Rules is build up from 5 components: –Attributes (characteristics of the agents) –Deontic: may/must/must not –Aim: action of the agent –Conditions: when, where and how –Or else: sanctions when not following a rule

Process of constructing a rule from the libraries IF “other agent” in “my area” it MUST NOT “collect tokens” ELSE “penalty”

Rule space based on experiments (Not yet in building blocks) Explicit mode required of not Start time harvesting Time left before “going crazy” Spatial allocation (none, corners, horizontal, vertical) Speed limit

Including monitoring and sanctioning Monitoring: –None –One monitor who cannot harvest are receives a quarter of the income –Everybody monitors, and sanctioning is costly –Monitoring rotates every x seconds (when monitoring one cannot harvest)

Tinkering the rules After every round agents update their preferences for rules (reinforcement learning), propose which rule set for next round, after which one of the proposed rule sets is chosen and implemented.

Agents breaking rules Agents can break rules. If an action is not allowed, it might break a rule with a probability related to the opportunities available (amount of tokens available nearby)

Distribution of total earnings (100 evolutions of 100 rounds)

Initial experiments Multiple (100) runs with 100 rounds with agents who conditionally cheat. Best solution: Low growth (one)Low growth (everybody) Speed limit7.55 ModeExplNot expl BoundariesVertical Start-time90110 Time to go crazy Earnings (tokens)337409

From ABM back to experiments Further analysis may provide us expectations of outcomes for experiments with human participants. Additional experiments can be done to test those.

Areas to explore in model analysis Do clusters of rules evolve? And do these clusters change with different tendencies of agents breaking the rules. Co-evolution of cheating behavior and rules (incl. monitoring/sanctioning) What are path-dependent trajectories? What if growth rates change between rounds? How will this affect the evolved rule sets? How will differences in constitutional rules will affect the ability to derive high performance.

Concluding remarks Combining agent-based models with experiments in the field and the lab. The aim is not to make predictive models, but theoretical models grounded in empirical observations. Challenges: –Calibration of agent-based models (multiple metrics) –Modeling communication –Large scale controlled experiments with humans

Questions?