1. Elnaz Nouri, David Traum Institute for Creative Technologies, USC A Cross-cultural Study of Playing Simple Economic Games Online with Humans and Virtual Humans HCII 2013 - 25 th July Las Vegas

2. Abstract We ran simple online economic interactions between virtual human and people from two countries (India and US). We compare our results to: Previously reported findings of similar interactions run in the laboratory.

3. We are interested in Online interactionsInteraction with agents Reliability of self-reported demographics on Mechanical Turk showed to be above 97% economic games run on Mechanical Turk are comparable to those run in laboratory setting even when with low stakes for payment. Agents (Virtual humans) can act as social cues in the interaction. Previous studies on effect of agent’s facial expressions in prisoner’s dilemma ()

4. We are interested in

5. Questions Do we observe deviating from self-interested behavior from participants when playing with virtual humans? Does the online version of games replicated findings from previous in lab and on field studies? Can the values held by participants account for these differences? Are there cultural differences between participants from different countries? What causes these differences in behavior? What impact does the type of game have on players’ decisions and values?

6. Questions we try to answer Opponent’s effect: How similar or different do participants feel and act when playing a virtual human versus another person? Culture’s effect: How different are players from the United States from Players in India? Game’s effect: What impact does the type of game have on players’ decisions and values?

7. In this talk Previous work: Cultural differences in decision-making games and values MARV model and MARV survey Social aspects of human-agent interaction On-line AMT Study and experiment details US and Indian participants Human vs. Virtual Humans Dictator or Ultimatum game Results Cross-cultural differences in game play and personal values Game effect Opponent effect Conclusion

8. Previous Work In-person games have been for understanding people's economic decision making behavior. (Camerer, 2003) Ultimatum game(Guth, 1983), Dictator game(Bolton, 1998), Prisoner’s Dilemma (Rapoport, 1965) …

9. An example: Ultimatum Game Expected Results: Two players can split a certain amount of money (Güth, 1982). Two turn game: 1.Proposer: make an offer 2.Responder: accept or reject the offer Offer the minimum amount possibleAccept any offer greater than zero

10. Cultural Variations in Ultimatum Game Observations: Proposers offer about 40 – 50$ on average. Responders reject offers of 20$ or less. (Camerer, 2003) (Roth, 1993) Considerable variation of offers and acceptance rates across 4 cultures (Roth 1993; Camerer 2003)

11. MARV = Multi-Attribute Relational Values Goal: Internal computational model of decision making for agents that is sensitive to culture, and produces behavior consistent with observations of that culture Approach: applying multi-attribute decision-making model to calculate utility of decisions assigning appropriate weights to each of the following attributes: MARV Model (Nouri, 2011)

12. Methods for setting up the weights Previous Methods for determining culture-specific weights on attributes: 1. Intuitions based on Hofstede’s dimensional model of culture (Nouri & Traum CMVC 2011) 2. Machine learning from human behavior data (Nouri et al CogSci 2012) New method  Directly ask participants how much they care about the weights (Nouri and Traum GDN 2013) by assigning weights from -5 to 5. (-5 means they don’t care about that value whereas 5 shows they care significantly about the value.) Hofstede dimensions of cultural values PDI: Power Distance (large vs. small), IDV: Individualism vs. Collectivism, MAS: Masculinity vs. Femininity, UAI: Uncertainty Avoidance (strong vs. weak), LTO: Long- vs. Short-Term Orientation, IVR: Indulgence vs. Restraint, and MON: Monumentalism vs. Self- Effacement. VS08 Hofestede Survey Questions: 1.have sufficient time for your personal or home life (IDV) 2.Q2 have a boss (direct superior) you can respect (PDI) 3.get recognition for good performance (MAS) 4.have security of employment (IDV) 5.have pleasant people to work with (MAS) 6.do work that is interesting (IDV) 7.be consulted by your boss in decisions involving your work (PDI) 8.live in a desirable area (MAS) 9.have a job respected by your family and friends (IDV) 10.have chances for promotion (MAS) 11.keeping time free for fun (IVR) 12.moderation: having few desires (IVR) 13.being generous to other people (MON) 14.modesty: looking small, not big (MON) 15.If there is something expensive you really want to buy but you do not have enough money, what do you do? (LTO) 16.How often do you feel nervous or tense?(UAI) 17.Are you a happy person? (IVR) 18.Are you the same person at work (or at school if you’re a student) and at home? (LTO) 19.…

13. Hofstede Culture Model: dimensions of cultural values PDI: Power Distance (large vs. small), IDV: Individualism vs. Collectivism, MAS: Masculinity vs. Femininity, UAI: Uncertainty Avoidance (strong vs. weak), LTO: Long- vs. Short-Term Orientation, IVR: Indulgence vs. Restraint, and MON: Monumentalism vs. Self-Effacement.

14. VS08 Hofestede Survey Questions 1. have sufficient time for your personal or home life (IDV) 2. Q2 have a boss (direct superior) you can respect (PDI) 3. get recognition for good performance (MAS) 4. have security of employment (IDV) 5. have pleasant people to work with (MAS) 6. do work that is interesting (IDV) 7. be consulted by your boss in decisions involving your work (PDI) 8. live in a desirable area (MAS) 9. have a job respected by your family and friends (IDV) 10. have chances for promotion (MAS) 11. keeping time free for fun (IVR) 12. moderation: having few desires (IVR) 13. being generous to other people (MON) 14. modesty: looking small, not big (MON) 15. If there is something expensive you really want to buy but you do not have enough money, what do you do? (LTO) 16. How often do you feel nervous or tense?(UAI) 17. Are you a happy person? (IVR) 18. Are you the same person at work (or at school if you’re a student) and at home? (LTO) 19. Do other people or circumstances ever prevent you from doing what you really want to (IVR) 20. how would you describe your state of health these days? (UAI) 21. How important is religion in your life?(MON) 22. How proud are you to be a citizen of your country? (MON) 23. How often, in your experience, are subordinates afraid to contradict their boss (or students their teacher?) (PDI) 24. One can be a good manager without having a precise answer to every question that a subordinate may raise about his or her work (UAI) 25. Persistent efforts are the surest way to results (LTO) 26. An organization structure in which certain subordinates have two bosses should be avoided at all cost (PDI) 27. A company's or organization's rules should not be broken - not even when the employee thinks breaking the rule would be in the organization's best interest (UAI) 28. To what extent We should honor our heroes from the past (LTO)

15. MARV Survey AbbreviationValue Description V self Getting a lot of points V other The other player getting a lot of points V compete Getting more points than the other player V fairness having the same number of points as the other player V joint Making sure that if we add our points together we got as many points as possible V rawls The player with fewest points (whoever that is) gets as many as possible V lower bound Making sure to get some points (even if not as many as possible) V chance The chance to get a lot of points (even if there's also a chance not to get any points) Rating scale: from -5 (no importance at all) to 5 (significantly imporant)

16. Experiment Set Up Played online version Ultimatum Game or Dictator Game over 100 points The Ultimatum Game. as described. The Dictator Game. played exactly like the standard Ultimatum Game, except that the responder is not given an opportunity to accept or reject the offer. Paid based on their performance in the game: $0.5 show up fee Could earn another $0.05 for each additional 10 points that they accumulated in the game.

17. Snapshot of the SimCoach character http://labs.simcoach.org/simcoach/?space=mini&character=3072 http://labs.simcoach.org/simcoach/?space=mini&character=3072

18. Experiment Procedure Before Game: 1. Fill out the VS08 Hofstede Survey and demographic information questions 2. Receive instructions about the game (Dictator Game or Ultimatum Game) denoting they would be playing with another participant from their country. The Game: 1. Play the proposer in Dictator Game or Ultimatum game 2. Fill out the MARV Decision-making values survey 3. (in the case of the ultimatum game) Receive their partner’s move and their final reward.

19. Study Participants Indian and US participants recruited on Amazon Mechanical Turk. Number of players from India Dictator GameUltimatum Game Human107101 Virtual Human3847 Number of players from US Dictator GameUltimatum Game Human107101 Virtual Human4653

20. Results

21. Results: Game Effect Offer Distribution Offers: The offers made in the two games are significantly different from one another. Average Offer in Dictator Game Average Offer in Ultimatum Game 39.647.6 Values: are significantly different between the two games: {V other, V compete, V equal, V joint, V rawls, V lower bound }

22. Results: Culture’s Effect India Offers: Significant difference between the two cultures when playing ultimatum game with Virtual humans (p value<0.05). Values: Significance difference between the values reported by Indians and Americans (across all conditions): { V self, V compete, V chance } When playing with Virtual Humans { V other in dictator game, V lowerbound in ultimatum game} Average US offers 44.16 Average India offers 41.44 US

23. Results: Opponent’s Effect Offers: Playing against a virtual human or a human does not bring about significant difference in the offers made in the games. Indians played differently when playing ultimatum game with a virtual human as opposed to a human (p value<0.05). Values: Significant differences in the values reported { V self, V other, V compete, V rawls, V lower bound, V chance }

24. Prediction of offers Accuracy of prediction Percent correctDictator GameUltimatum Game Country (US or India)39.55%51.61% Hofstede Scores (7 dimensions)39.56%50.05% Hofstede Questions (28 questions)39.24%53.52% Decision-making Values (8 values)52.86%54.90% Random baseline: frequency of offers in the data 21%32% Most common offer baseline (50%)38%51%

25. Prediction of Culture percent correctDictator Game Ultimatum Game Offers (11 values)53.40%54.11% Hofstede Scores (7 dimensions)64.42%69.85% Hofstede Questions (28 questions)76.39%77.79% Decision-making Values (8 values)60.09%65.70% Random Baseline50%

26. Conclusion *** Our results are consistent with reported results in the literature. Opponent’s effect: People from US and India both treat virtual humans similar to how they would have treated another human. We conclude that virtual humans can be a reasonable substitute to humans in online economic interactions. (eg. Selling and negotiation) Culture’s effect: Values held by people from the two countries are different under similar conditions and the reasons should be further investigated. Game’s effect: The most prominent cause affecting the game behavior and the offer values is the type of the game being played.

27. Future Work More data collection More cultures More types of games (potential for other values to be distinguished) Modeling Culture-specific agent models based on reported values Correlations between Hoftstede questions/dimensions and values (for cultures with no values data reported)

28. Thank you! Questions?

29. Results Offers in the Ultimatum Game and the Dictator Game Condition ( mean,std) Dictator GameUltimatum Game US39.81, 21.2348.51, 16.08 India37.75, 27.9645.14, 20.52 Both38.78, 24.7846.83, 18.47

30. Results – Hofstede’s Dimensional values calculated DimensionsPDIIDVMASUAILTOIVRMON US scores reported by Hofstede 4091624629680 India scores reported by Hofstede 7748564061260 Initial calculated US 15.2111.10-2.18-56.379.1830.31-2.93 Initial calculated India 17.950.675.38-53.174.8356.0070.98 Constants using US baseline 24.7879.8964.18102.3719.8137.682.93 ANOVA p-values 0.600.05*0.130.620.470.00**

31. Results – Comparison with previous scores reported by Hoftstede

32. Cultural Differences for VSM 2008 Questions Question Number ANOVA p-value QuestionRelated Dimension Q3 0.01** get recognition for good performanceMAS Q4 0.02* have security of employmentIDV Q5 0.03* have pleasant people to work withMAS Q6 0.02* do work that is interestingIDV Q9 0.00*** have a job respected by your family and friendsIDV Q10 0.01** have chances for promotionMAS Q13 0.01** being generous to other peopleMON Q14 0.00** modesty: looking small, not bigMON Q16 0.02* How often do you feel nervous or tense?UAI Q17 0.00*** Are you a happy person?IVR Q20 0.01** how would you describe your state of health these days?UAI Q21 0.00*** How important is religion in your life?MON Q22 0.00*** How proud are you to be a citizen of your country?MON Q24 0.00*** One can be a good manager without having a precise answer to every question that a subordinate may raise about his or her work UAI Q26 0.00*** An organization structure in which certain subordinates have two bosses should be avoided at all cost PDI Q27 0.00** A company's or organization's rules should not be broken - not even when the employee thinks breaking the rule would be in the organization's best interest UAI

33. Results IndiaUS Offer Distribution in Dictator Game

34. Results IndiaUS Offer Distribution in Ultimatum Game

35. Results IndiaUS Reported MARV values in Dictator Game

36. Results IndiaUS Reported MARV values in Ultimatum Game