# Combinatorial Agency Michal Feldman ( Hebrew University) Joint with: Moshe Babaioff (UC Berkeley) Noam Nisan (Hebrew University)

## Presentation on theme: "Combinatorial Agency Michal Feldman ( Hebrew University) Joint with: Moshe Babaioff (UC Berkeley) Noam Nisan (Hebrew University)"— Presentation transcript:

Combinatorial Agency Michal Feldman ( Hebrew University) Joint with: Moshe Babaioff (UC Berkeley) Noam Nisan (Hebrew University)

Hidden Actions Algorithmic Mechanism Design: computational mechanisms to handle Private Information. (Classical) Mechanism Design  Private Information  Hidden Actions We study hidden actions in multi-agents computational settings

Example Quality of Service (QoS) Routing [FCSS’05]:  We have some value from message delivery.  Each agent controls an edge: succeeds with low probability by default. succeeds with high probability if exerts costly effort  Message delivered if there is a successful source- sink path.  Effort is not observable, only the final outcome. source sink

Modeling: Principal-Agent Model Agent Principal exerts effort cost: c >0 Does not exert effort cost: 0 Project succeeds with high probability Project succeeds with low probability Motivating rational agents to exert costly effort toward the welfare of the principal, when she cannot contract on the effort level, only on the final outcome “Success Contingent” contract. The agent gets a high payment if project succeeds, gets a low payment if project fails Our focus is on multi-agents technologies

Our Model n agents Each agent has two actions (binary-action):  effort ( a i = 1), with cost c>0 (c i (1)=c)  no effort ( a i = 0), with cost 0 (c i (0)=0) There are two possible outcomes (binary outcome):  project succeeds, principal gets value v  project fails, principal gets value 0 Monotone technology function t: maps an action profile to a success probability:  t: {0,1} n  [0,1] t(a 1,…,a n )=success probability given (a 1,…,a n )   i t(1, a -i ) > t(0,a -i ) (monotonic) Principal designs a contract for each agent  Project succeeds  agent i receives p i (otherwise he gets 0) Players’ utilities, under action profile a=(a 1,…,a n ) and value v:  Agent i: u i (a) = t(a)·p i – c i (a i )  Principal: u(a,v) = t(a)·(v –Σ i p i ) Agents are in a game, reach Nash equilibrium. The Principal’s design parameter: Used to induce the desired equilibrium The Principal’s “input” parameter.

Example: Read-Once Networks A graph with a given source and sink  Each agent controls an edge, independently succeeds or fails in his individual task (delivering on his edge) Succeeds with probability ɣ<½ with no effort Succeeds with probability 1- ɣ (>½ > ɣ) with effort  The project succeeds if the successful edges form a source-sink path. example: t(1, 1, 0) = Pr { x 1  (x 2  x 3 ) =1 | a=(1,1,0) } = (1- ɣ ) (1- ɣ (1- ɣ )) source sink a 1 =1 a 2 =1 a 3 =0 Pr {x 1 =1}=1- ɣ Pr {x 2 =1}=1- ɣ Pr {x 3 =1}= ɣ

Nash Equilibrium Principal’s best contract to induce eq. a=(a 1,…,a n ):  p i = c /  i (a -i ) for agent i with a i =1  p i = 0 for agent i with a i =0 e.g., (1,0) (1,1) Agent i’s utility u i ( 1,a -i ) = p i · t( 1,a -i ) – cu i ( 0,a -i ) = p i · t(0,a -i ) exerts effortDoes not exert effort  i (a -i ) P 2 =0

Optimal Contract the principal chooses a profile a*(v) that maximizes her optimal equilibrium utility Probability of success Total payments

Research Questions How does the technology affect the structure of the optimal contracts?  Several examples (AND, OR, Majority …)  General technologies What is the damage to the society due to the inability to monitor individual actions?  “price of unaccountability” What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges from the graph?

Optimal Contracts: simple AND technology 2 agents,  = ¼, c=1 t(0,0) =  2 = (¼) 2 =1/16 t(1,0) =t(0,1)=  = 3/16;  0 =t(1,0)-t(0,0)=3/16 - 1/16 = 1/8 t(1,1) =   = 9/16 Principal’s Utility 0 agents exert effort: u((0,0),v) = t(0,0)·v = v/16 1 agent exerts effort: u((1,0),v) = t(1,0)·(v-c/  0 ) = =3/16(v-1/(1/8))=(3/16)v-3/2 2 agents exert effort: u((1,1),v) = t(1,1)·(v-2c/  1 ) = 9v/16-3 st x1x1 x2x2 At value of 6 there is a “jump” from 0 to 2 agents

 vv Optimal Contract Transitions in AND and OR AND st x1x1 x2x2 st x1x1 OR x2x2 ɣ =1/4  optimal to contract with 0 agents up to 6, then with 2 agent 2

Optimal Contract Transitions in AND and OR Theorem: For any AND technology, there is only one transition, from 0 to n agents. Theorem: For any OR technology, there are always n transition (any number of agents is optimal for some value). We characterize all technologies with 1 transition and with n transitions.

Proofs Idea-AND’s single transition Observation (monotonicity): number of contracted agents monotonically non-decreasing in v. Proof for AND’s single transition:  At the indifference value between 0 and n agents, contracting with 0 { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/13/3621916/slides/slide_13.jpg", "name": "Proofs Idea-AND’s single transition Observation (monotonicity): number of contracted agents monotonically non-decreasing in v.", "description": "Proof for AND’s single transition:  At the indifference value between 0 and n agents, contracting with 0

Transitions in AND and OR Proof (AND): k: number of contracted agents this function has a single minimum point, thus maximized at one of the edges 0 or n

Proofs Idea – OR’s n transitions Let v k be the indifference point between k and k+1 agents ( u(k,v k ) = u(k+1,v k ) ) We show that for OR: v k+1 > v k This ensures that k is optimal from v k-1 to v k v 0 : The 0,1 indifference value. v 1 : The 1,2 indifference value. v 1 >v 0

Transitions in AND and OR k: number of contracted agents solve for v: u(k) = u(k+1), and let v(k) be the solution we have to show: v(k+1) > v(k)   E.g., n=3  v(0) v(1) v(2)

Majority, 5 agents v ɣ

General Technologies In general we need to know which agents exert effort in the optimal contract Examples: In potential, any subset of agents (out of 2 n subsets) that exert effort could be optimal for some v. Which subsets can we get as an optimal contract?

And-of-Ors (AOO) Technology Example: 2x2 AOO technology Theorem: The optimal contract in any AOO network (with identical OR components) has the same number of agents in each OR-component Proof: by induction based on following lemmas:  Decomposition lemma: if S=TUR is optimal on f=h  g on some v, then T is optimal for h on v · t g (R) and R is optimal for g on v·t h (T)  Component monotonicity lemma: the function v  t h (T) is monotone non-decreasing (same for v  t g (R) )  v {A1,B1}{A1,B1}{A1,B1,A2,B2}{A1,B1,A2,B2} f = h  g T R

Decomposition Lemma Proof: f = h  g T R if S=TUR is optimal on f=h  g on some v, then T is optimal for h on v · t g (R) and R is optimal for g on v·t h (T)

Component Monotonicity Lemma Proof:  S 1 = T 1 U R 1 optimal on v 1  S 2 = T 2 U R 2 optimal on v 2 v 2, and g(R 1 ) > g(R 2 ), T 1 is optimal for h on a larger value than T 2.  Thus, by monotonicity lemma, h(T 1 ) ≥ h(T 2 ) h g T1T1 R1R1 f: T2T2 R2 The function v  t h (T) is monotone non- decreasing (same for v  t g (R) )

And-of-Ors Theorem: The optimal contract in any AOO network, composed of n c OR-components (of size n l ) contracts with the same number of agents in each OR-component. Thus, |orbit(AOO)| ≤ n l +1 Proof: by induction on n c  Base: n c =2 assume (k 1,k 2 ) is optimal on some v, assume by contradiction k 1 >k 2 (wlog), thus h(k 1 )>h(k 2 ). By decomposition lemma: k 1 optimal for h on v·h(k 2 ) k 2 optimal for h on v·h(k 1 )>v·h(k 2 ) but if k 2 optimal for a larger value, k 2 ≥k 1. in contradiction.

And-of-Ors assume (induction) that claim holds for any number of OR components < n c  Assume 1 st component has k 1 contracted agents  Let g be the conjunction of the other (n c -1) comp.  By decomposition lemma, contract on g is optimal at v·h(k 1 ), thus by induction hypothesis has same number of agents, k 2, on each OR component.  Let h 2 be conjunction of first two comp.  By decomp. Lemma, contract on h 2 is optimal for some value and by induction hypothesis has same number of agents, k 3  We get k 1 =k 3 (in first comp. k 1 agents contracted), and k 2 =k 3 (in second comp. k 2 agents contracted), thus k 1 =k 2 g hhhh k1k1 k2k2 k2k2 k2k2 h2h2 k3k3 k3k3 ===

The Collection of Optimal Contracts Given t we wish to understand how the optimal contract changes with v (the “orbit”). Monotonicity Lemma: The optimal contract success probability t(a*(v)) is monotonic non- decreasing with v  So is the utility of the principal, and the total payment Thus, there are at most 2 n -1 changes to the optimal contracts (|Orbit(t)| ≤ 2 n ) Is there a structure on the collection of optimal contracts of t?

The Collection of Optimal Contracts Observation 1: in the observable-actions case, only one set of size k can be optimal (set with highest probability of success) Observation 2: not all 2 n subsets can be obtained  Only a single set of size 1 can be optimal (set with highest probability of success) Thm: There exists a tech. with optimal contracts Open question 1: is there a read-once network with exponential number of optimal contracts? Can a technology have exponentially many different optimal contracts?

Exponential number of optimal contracts (1) Thm: There exists a tech. with optimal contracts Proof sketch:  Lemma 1: all k-size sets in any k-admissible collection can be obtained as optimal contracts of some t  Lemma 2: For any k, there exists a k-admissible collection of k-size sets of size Based on error correcting code  Lemma 3: for k=n/2 we get a k-admissible collection of k-size sets of size, as required. S1S1 S2S2 S3S3 S4S4 Collection of sets of size k, in which every two sets in it differ by at least two elements

Proof of Lemma 1 S S\i k k-1 1 n t(S)= ½ -  S t(S\i)= ½ - 2  S S’ S’\i t(S’)= ½ -  S’ t(S’\i)= ½ - 2  S’ marginal contribution of i  S is: t(S) – t(S\i) =  S Define t to ensure that the marginal contribution of at least one agent is very small Claim: at v s =(ck) / 2  S 2, the set S is optimal: S better than any other set in col. (by derivative of u(S,v)) S better then any other set not in col. (too high payments)

 Let v s be v s.t.

Proof (k-orbit)  : admissible collection of k-size sets Z : {all S   U all S\i } For sets T  Z:  if  z  Z | z  T  else: t(T)= ε ·|T |  Let v s be v s.t.,S chosen at v s S1S1 S2S2 S3S3  : 3-size sets S4S4 Pick ε S  (0.17,0.2] i t(S)=½ - ε S t(S\{i})=½ - 2ε S Pick ε S =0.2 S4S4 T t(T)=max z|T  z, z  Z (t(z)+( |T|-|z| )·ε)

Proof (cont’d) Need to show: s yields higher utility at v s than any other set s’  |s’| = k-1 If s’  {s\i} : t(s’)=ε·|s’|  can be arbitrarily small If s’  {s\i} : t(s) ≥ ½ - ε s > 0.3, t(s’) ≤ ½ - 2ε s < 0.16  |s’| > k  at least one agent is paid 1 , so pick  s.t. payment > v s v u(s 0.2 ) u(s 0.18 ) V 0.2 V 0.18 Size k and (k-1) u(s’,v 0.2 ) 0.16

Exponential number of optimal contracts (1) Thm: There exists a tech. with optimal contracts Proof sketch: constructive  Lemma 1: any admissible collection can be obtained as the k-orbit of some t Define t as follows:  for every set in the collection, Pick  S, and define: t(S)=½ - ε S and t(S\{i})=½ - 2ε S (thus, marginal contribution of i  S is  S )  for every set not in the collection, define t to ensure that the marginal contribution of each agent is very small Claim: at v s =(ck) / 2  s 2, the set S is optimal  S better than any other set in col. (by derivative of U(S,v))  S better then any other set not in col. (too high payments) S1S1 S2S2 S3S3 S4S4 collection of optimal sets of size exactly k Collection in which every two sets in it differ by at least two elements

Exponential number of optimal contracts (2) Lemma: For any n ≥ k, there exists an admissible collection of k-size sets of size Proof: take error correcting code that corrects 1 error.  Hamming distance ≥ 3  admissible  Known:  codes with  (2 n /n) code words.  Construct a code with sufficient # of k-weight words XOR every code word with a random word r. weight k w/ prob Expected number of k-weight code words There exists r such that the expectation is achieved or exceeded

Research Questions How does the technology affect the structure of the optimal contracts? What is the damage to the society / principal due to the inability to monitor individual actions?  “price of unaccountability” What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges from the graph?

Observable-Actions Benchmark (first best) Actions are observable Payment: an agent that exerts effort is paid his cost (c) Principal’s utility: u(a,v) = v·t(a) –  i|a i =1 c Principal’s utility = social welfare sw(a,v). The principal chooses a* OA, the profile with maximum social welfare.

Social Price of Unaccountability Definition: The Social Price Of Unaccountability (POU S ) of a technology is the worst ratio (over v) between the social welfare in the observable-action case, and the social welfare in the hidden-action case: a* - optimal contract for v in the hidden-action case a* OA - optimal contract for v in the observable-action case Example: AND of 2 agents: v 02 Hidden actions Observable actions 02 st

Principal’s Price of Unaccountability Definition: The Principal’s Price Of Unaccountability (POU P ) of a technology is the worst ratio (over v) between the principal’s utility in the observable- action case, and the principal’s utility in the hidden- action case: a* - optimal contract for v in the hidden-action case a* OA - optimal contract for v in the observable-action case

Price of Unaccountability - Results Theorem: The POU of AND technology is  unbounded for any fixed n≥2, when     unbounded for any fixed  ½ when n   Theorem: The POU of OR technology is bounded by 2.5 for any n

Research Questions How does the technology affect the structure of the optimal contracts? What is the damage to the society due to the inability to monitor individual actions?  “price of unaccountability” What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges from the graph?

Complexity of Finding the Optimal Contract Theorem: There exists a polynomial time algorithm to compute (a*,p), if t is given by a table (exponential input). Theorem: If t is given by a black box, exponentially many queries may be required to find (a*,p). Theorem: For read-once networks, the optimal contract problem is #p-hard (under Turing reduction) (proof: reduction from network reliability problem)  Open problem 3: is it polynomial for series-parallel networks?  Open problem 4: does it have a good approximation? Input: value v, description of t Output: optimal contract: (a*,p)

Complexity of Finding the Optimal Contract Theorem: There exists a polynomial time algorithm to compute (a*,p), if t is given by a table (exponential input). Theorem: If t is given by a black box, exponentially many queries may be required to find (a*,p). Proof:  for value v = c(k+ ½), S’ is optimal  Any algorithm must query all sets of size k=n/2 to find S’ in the worst case Input: value v, description of t Output: optimal contract: (a*,p) t(S)=0 t(S)=1 100000 sets of size n/2 sets of size 1 sets of size n S’

Complexity of Finding the Optimal Contract Theorem: For read-once networks, the optimal contract problem is #p-hard  Proof: reduction from network reliability problem Open problem 3: is it polynomial for series- parallel networks? Open problem 4: does it have a good approximation? Input: value v, description of t Output: optimal contract: (a*,p)

Best Contract Computation in Read-Once Networks Proof (sketch): an algorithm for this problem can be used to compute t(E) (probability of success) Player x will enter the contract only for very large value of v (only after all other agents are contracted), call this value v c At v c, principal is indifferent between E and EU{x} G stt G’  x  ½

Research Questions How does the technology affect the structure of the optimal contracts? What is the damage to the society due to the inability to monitor individual actions?  “price of unaccountability” What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges from the graph?

Mixed Strategies In the non-strategic case: NO (convex combination) What about the agency case? Extended game:  q i : probability that agent i exerts effort  t( q i,q -i ) = q i ·t(1,q -i )+ (1-q i )·t(0,q -i )  Marginal contribution:  i (q -I ) = t(1,q -i ) - t(0,q -i ) ≥ 0 Can mixed-strategies help the principal ? What is the price of purity ?

Nash Equilibrium in Mixed Strategies Claim: agent i’s best-response is to mix with probability q  (0,1) only if she is indifferent between 0 and 1 Agent i’s utility: Principal’s utility: Agent i’s utility u i ( 1,q -i ) = p i · t( 1,q -i ) – c i u i ( 0,q -i ) = p i · t(0,q -i ) High effortLow effort

Example: OR with two agents Optimal contract for v=110  Pure strategies: both agents contracted: u = 88.12...  Mixed strategies: q 1 =q 2 =0.96..: u=88.24... Two observations:  q 1 =q 2 in optimal contract  Principal’s utility is improved, but only slightly How general are these observations?

Optimal Contract in OR Technology Lemma: For any anonymous OR (any ,n,c,v), k  {0,1,…,n} agents exert effort with equal probabilities q 1 =…=q k  (0,1], and n-k agents shirk. i.e. optimal profile: (0 n-k, q k ) Proof (skecth): suppose by contradiction that (q i,q j,q -ij ) s.t. q i,q j  (0,1) and q i > q j is optimal qjqj qiqi (q i,q j, q -ij ) (q i -ε,q j +y ε, q-ij) For a sufficiently small ε, success probability increases, and total payments decrease. In contradiction to optimality

Optimal Contract in OR Technology Example: OR with 2 agents:

Price of Purity (POP) Definition: POP is the ratio between principal’s utility in mixed strategies and in pure strategies Optimal pure contract Optimal mixed contract

Price of Purity Definition: technology t exhibits  increasing returns to scale (IRS) if for any i and any b ≥ a t(b i,b -i )-t(a i,b -i ) ≥ t(b i,a -i )-t(a i,a -i )  decreasing returns to scale (DRS) if for any i and any b ≥ a t(b i,b -i )-t(a i,b -i ) ≥ t(b i,a -i )-t(a i,a -i ) Observations: AND exhibits IRS, OR exhibits DRS Theorem: for any technology that exhibits IRS, optimal contract is obtained in pure strategies  e.g., AND

Price of Purity For any anonymous DRS technology, POP ≤ n For anonymous OR with n agents, POP ≤ 1.154.. For any anonymous technology with 2 agents, POP ≤ 1.5 For any technology (not necessarily anonymous, but with identical costs) with 2 agents, POP ≤ 2 Observation: the payment to each agent in a mixed profile is greater than the min payment in a pure profile and smaller than the max payment in a pure profile

Research Questions How does the technology affect the structure of the optimal contracts? What is the damage to the society due to the inability to monitor individual actions?  “price of unaccountability” What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges from the graph?

as before Free-Labor So far, technology was exogenously given Now, suppose the principal has control over the technology in that he can ex-ante remove some agents from the graph  Example: OR with 2 agents Action set of agent i: a i  {1,0,  }  1: exert effort – succeed with probability d. cost=c  0: do not exert effort - succeed with probability  d. cost=0   : do not participate – succeed with probability 0. cost=0 Action  “wastes free-labor” since action “0” increases the success probability with no additional cost

Free-Labor The answer is: YES Example: OR technology, n=2,  =0.2 Theorem: for technologies with increasing marginal contribution (e.g., AND), utilizing all free-labor is always optimal Are there scenarios in which the principal gains utility from “wasting free-labor”? v 012 1 removed

Analysis of OR Lemma: for any OR with n agents and  which is small enough, there exists a value for which in the optimal contract one agent exerts effort and no other agent participates  =0.49  =0.25  =0.01

Version of the Braess’s Paradox A project is composed of 2 essential components: A and B And-of-Ors (AOO): allow interaction between teams Or-of-Ands (OOA): don’t allow interaction between teams Obviously, AOO is superior in terms of success probability st B2B2 B1B1 A2A2 A1A1 s t A1A1 B1B1 B2B2 A2A2 project succeeds if at least one of the following pairs succeed: (A 1,B 1 ) ; (A 1,B 2 ) ; (A 2,B 1 ) ; (A 2,B 2 ) project succeeds if at least one of the following pairs succeed: (A 1,B 1 ) ; (A 2,B 2 )

Version of the Braess’s Paradox st  i =1 B2B2 B1B1 A1A1 A2A2 s t A1A1 B1B1 B2B2 A2A2 remove middle edge st B2B2 B1B1 A2A2 A1A1 don’t remove middle edge Or-of-Ands “wastes free-labor”. Could the principal gain utility from removing middle edge? u(2,2) = 75.59.. Example:  =0.2, v=110 u(1,1) = 74.17.. > Conclusion: it may be beneficial for the principal to isolate the teams And- of-Ors Or-of- Ands

Summary “Combinatorial Agency”: hidden actions in combinatorial settings Computing the optimal contract in general is hard Natural research directions:  technologies whose contract can be computed in polynomial time  Approximation algorithms Many open questions remain

Thank You mfeldman@cs.huji.ac.il

Related Literature [Winter2004] Incentives and discrimination  The effect of technology on optimal contract (full implementation) [Winter2005] Optimal incentives with information about peers [Ronen2005][Smorodinsky and Tennenholtz2004,2005]  Multi-party computation with costly information [Holmstrom82] Moral hazard in teams  Budget-balanced sharing rules

Download ppt "Combinatorial Agency Michal Feldman ( Hebrew University) Joint with: Moshe Babaioff (UC Berkeley) Noam Nisan (Hebrew University)"

Similar presentations