Coalition Formation between Self-Interested Heterogeneous Actors Arlette van Wissen Bart Kamphorst Virginia DignumKobi Gal

Introduction Allowing people to interact with computer systems in a manner similar to the way that people interact with each other. To achieve successful interactions, we must understand the factors involved. This may provide new insights into: -how decisions are made -how agent’s behaviors are perceived Types of interactions: -cooperative and helpful interactions -competitive interactions -cooperative interactions in competitive scenarios Human-Agent Interaction

Introduction utility function a real number that represents the level of satisfaction of an actor in that state of the world. non-cooperative games - self-interested actors who can make non-binding commitments - basic modeling unit is the individual cooperative games - players can make binding commitments - communication and negotiation between the players is allowed - groups of players (coalitions) may enforce cooperative behavior - basic modeling unit is the group Self-Interested Actors

Introduction A coalition is a subset S of the set of players N where: 1.agreements take place involving approval by every player in S 2.no agreement between any other members of S and any member of N – S is permitted A coalitional game with transferable utility is a pair ( N, v ) where 1. N is the set of all players 2. v associates with each coalition S ⊆ N a real-valued payoff v(S) that the coalition’s members can distribute among themselves. We assume that v( ∅ ) = 0. Coalition Formation

Related Work To become part of successful coalitions, one has to deal with the following problems (Gentry, 1987) : 1.how to identify the members of a winning coalition 2.how to reach an agreement on the allocation of the payoffs 3.how to make a tradeoff between payoffs and non-utilitarian strategy preferences Humans are not fully rational when it comes to these formation decisions. Coalition Formation

Related Work Ultimatum Game Allocation of Payoff

Related Work Ultimatum Game Humans are not completely rational actors (Kagel & Roth, 1995) Social factors influence human behavior: Inequity aversion (Loewenstein, 1989) Intention (Ross & Fletcher, 1985) Altruism, fairness, reciprocity (Camerer, 2003 ) Social factors also influence cooperation between humans and intelligent agents (van Wissen et al., 2009) Allocation of Payoff

Related Work Human Coalition Formation Relative payoffs clearly matter in the formation of coalitions. There exists a strong correlation between relative payoffs and rejection behavior. (Bolton & Brosig, 2007) In 2- and 3-player coalition games, people respond by acting negatively reciprocally. They punish proposers by rejecting unfair offers. (Okada & Riedl, 2005) Less attention paid to human-agent interaction. Existing models generally do not consider trust or nature.

Related Work Trust For an actor a to be said to trust another actor b with respect to a particular goal g, a must have the following beliefs (castelfranchi, 1998, 2001) : Competence Belief: b is useful for achieving g and is able to provide the expected result Disposition Belief: b is not only capable, but also willing to do what is necessary to achieve g Dependence Belief: the results and rewards of achieving g depend on the involvement of b Fulfillment Belief: g will come about due to b ’s involvement

Related Work Trust experience based vs recommendation based?

Research Question How do nature and trust influence people’s decisions in mixed coalition formations? To what extent do trust and fairness influence team formation in mixed networks? How does the nature of actors affect the way people relate to their actions? Do actors develop stable relations over time?

Hypotheses Hypothesis 1 The nature of the actor affects the cooperation between participants. Hypothesis 2 A trust relationship between the actors affects the cooperation between them.

Conceptual Design Assumptions in CF Groups Prev work The members of coalitions are predefined. Our work The members of a coalition can be freely chosen. Interchangeability Prev work All potential members bring the same utility to a coalition. Our work Some potential members are preferred over others. Membership Prev work Membership to a coalition is equally available to everyone. Our work Members have to meet certain requirements for membership to be available to them. Conflict Prev work Conflict is eliminated by making agreements binding. Our work No conflict-free environment since agreements are not binding.

Conceptual Design Examine interactions between self-motivated agents that must adapt their behavior depending on the behavior of other agents in the environment. Package Delivery Problem

Experimental Design ``A key determinant of CT design was the goal of providing a vehicle for comparing the decision- making strategies people deploy when they interact with other people with those they deploy when computer systems are members of their groups.’’ Grosz et al., 2004 Colored Trails - Board - Chips - Proposals - Phases

Experimental Design Package Delivery Problem in CT

Conceptual Design Hypothesis 1 The nature of the actor affects the cooperation between participants. Deception Hypothesis 2 A trust relationship between the actors affects the cooperation between them. Repeated interaction Hypotheses revisited Transferable payoff & ability to choose team members

Experimental Design Package Delivery Problem in CT Coalition formation game with: N = 6 v ( N i ) = 3 v ( N i N j ) = 60 v ( N i N j N k ) = 180 superadditivity : A game G = ( N,v ) is superadditive if v ( ST ) ≥ v ( S ) + v ( T ) for all S, T ∈ N such that S ∩ T = ∅ imperfect information : players do not have knowledge of all actions and behavior of the players in the game and have partial visibility 5 rounds of t minutes, where 7 ≥ t ≤ 10

Experimental Design Package Delivery Problem in CT -6 players per game ( P ) -3 colors (=P/2)

Experimental Design Package Delivery Problem in CT -6 players per game ( P ) -3 colors (=P/2)

Experimental Design Package Delivery Problem in CT -6 players per game ( P ) -3 colors (=P/2) -6 large packages(= P ) -12 small packages(=2 P )

Experimental Design movie

Experimental Design 1.Survey -preferences -strategies 2.Log -proposals -coalitions -defections

Results

Hypothesis 1 The nature of the actor affects the cooperation between participants. Do players make more proposals to humans than to agents? Do players offer more points (more fair splits) to humans than to agents? Do players join more coalitions with a human as an initiator than with an agent as initiator? Hypothesis 2 A trust relationship between the actors affects the cooperation between them. Do players who defect less obtain a higher score than those who defect more? Do initiators who propose more fair splits obtain a higher score than those who propose less fair splits? Do players who are more often initiator obtain a higher score than those who are more often member? Do players offer less points to those who defect more than to those who defect less?

Results What strategy did you use for choosing your team members? ``Whether members were reliable. I could give computers a smaller share without feeling guilty. I did build a good history with one other human player.’’ ``I chose the computers mostly, since I thought that they would demand less points.’’ What strategy did you use for proposing a split of points? ``Split evenly with humans, keep more for myself when playing with computers.’’ ``Offered computer players less points, human players more.’’ How did your preference for humans/computers as a leader reflect in your strategy of joining and defecting from teams? ``I tended to feel worse about defecting from a human.’’

Discussion & Future Work 1.Write BDI agents based on a model of human decision making in this scenario (CTAPL framework). 2.Create a condition where the emphasis is on reciprocity: helping without direct payoff. 3.Have people play longer rounds to enable them to develop better trust relations.

Conclusion

Questions/Suggestions?

Game Flow Initiator

Game Flow Initiator

Game Flow Member

Game Flow Member