1. MSDM AAMAS-09
Two Level Recursive Reasoning by Humans Playing Sequential Fixed-Sum Games
Authors:
Adam Goodie, Prashant Doshi, Diana Young
Depts. of Psychology and Computer Science
University of Georgia

2. Outline Introduction Experimental study Results Discussion
Recursive reasoning
Related work
Experimental study
Problem setting
Participants
Methodology
Results
Discussion

3. Recursive reasoning
Strategic recursive reasoning in multi-agent settings (what do I think that you think that I think...)
Multi-agent decision making frameworks
RMM
I-POMDP
Theory-of-Mind
Real-world application settings
UAV

4. Related work (I) Harsanyi (1967) Mertens and Zamir (1985)
agent types
common knowledge
Mertens and Zamir (1985)
hierarchical belief system
Aumann (1999)
recursive beliefs

5. Related work (II) TOM and Behavioral game theory
Stahl and Wilson (1995)
a symmetric 3×3 matrix game
4% of subjects attributed recursive reasoning to their opponents
Hedden and Zhang (2002)
a sequential, two player, general-sum game(Centipede game)
subjects predominantly began with first-level reasoning
low percentage of subjects use second-level reasoning, when pitted against first-level co-players
Ficici and Pfeffer (2008)
a 3-player, oneshot negotiation game
subjects reasoned about others while negotiating
insufficient evidence to distinguish whether level two models better fit the observed data than level one models

6. Experimental study Problem setting Participants Methodology
Opponent models
Payoff structures
Design of task

7. Problem Setting Two-player alternating-move Fixed-sum
Complete and perfect information

8. UAV cover story scenario

9. Probabilities for players I and II in the cover story scenario

10. Participants 162 subjects
Undergraduate students enrolled in lower-level Psychology courses at the University of Georgia
Incentives
performance-contingent monetary rewards
partial course credit

11. Methodology Opponent models Payoff structure Design of task myopic
predictive
Payoff structure
Design of task
training phase
test phase

12. Opponent models Myopic (First-level reasoning)
Player II chooses its action based on the outcomes at states B and C

13. 2. predictive (Second-level reasoning)
Player II chooses its action by reasoning what player I will do rationally.

14. Payoff Structure trivial games diagnostic game D < C < B < A
A < B < C < D
diagnostic game
C < B < A < D
Different action choices for different opponent models

15. Design of Task Training phase trivial games criterion initial phase
no rationality errors in the 5 most recent games
initial phase
15 games
kickoff
failed to meet the criterion after 40 total training games

16. Test phase 40 diagnostic games
intersperse with 40 C < A < B < D and D < B < A < C
groups based on opponents
half against myopic ones
half against predictive ones
In each opponent model group
half played with abstract version
half played with the UAV cover story and the abstract version

17. Results Time period Monetary incentives Training Phase Test Phase
three months(September-November 2008)
Monetary incentives
50 cents/correct action, average $30/participant
Training Phase
162 subjects ( 26 kicked off)
Test Phase
136 participants (70 female)

18. More accurate choices when opponent is predictive model
No significant difference for two versions of games
mean proportion of accurate choices across all participants in each of the 4 groups

19. mean proportions marginalized over the abstract and realistic versions

20. mean proportions marginalized over myopic and predictive opponents

21. Count of participants grouped according to different proportions of accurate choice

22. Discussion
UAV cover story neither improved nor reduced the performance
This particular cover story had no effect
Did the subjects employ Minimax or Backward Induction?
Exit questionnaire revealed most subjects did not use these
Independent evaluators concurred that most subjects thought recursively
In some settings humans tend to reason at higher levels of recursion

23. Thank you Questions?