combinatorial agency michal feldman ( hebrew university)
DESCRIPTION
Combinatorial Agency Michal Feldman ( Hebrew University). Joint with: Moshe Babaioff (UC Berkeley) Noam Nisan (Hebrew University). Hidden Actions. Algorithmic Mechanism Design: computational mechanisms to handle Private Information . (Classical) Mechanism Design Private Information - PowerPoint PPT PresentationTRANSCRIPT
Combinatorial Agency
Michal Feldman(Hebrew University)
Joint with: Moshe Babaioff (UC Berkeley)
Noam Nisan (Hebrew University)
Hidden Actions
Algorithmic Mechanism Design: computational mechanisms to handle Private Information.
(Classical) Mechanism Design Private Information Hidden Actions
We study hidden actions in multi-agents computational settings
Example Quality of Service (QoS) Routing [FCSS’05]:
We have some value from message delivery. Each agent controls an edge:
succeeds with low probability by default. succeeds with high probability if exerts costly effort
Message delivered if there is a successful source-sink path.
Effort is not observable, only the final outcome.
source sink
Modeling: Principal-Agent Model
AgentPrincipal
exerts effortcost: c >0
Does not exert effortcost: 0
Project succeeds with high probability
Project succeeds with low probability
Motivating rational agents to exert costly effort toward the welfare of the principal, when she cannot contract on the effort level, only on the final outcome
“Success Contingent” contract. The agent
gets a high payment if project succeeds,
gets a low payment if project fails
Our focus is on multi-agents technologies
Our Model n agents Each agent has two actions (binary-action):
effort (ai=1), with cost c>0 (ci(1)=c) no effort (ai=0), with cost 0 (ci(0)=0)
There are two possible outcomes (binary outcome): project succeeds, principal gets value v project fails, principal gets value 0
Monotone technology function t: maps an action profile to a success probability: t: {0,1}n [0,1] t(a1,…,an)=success probability given (a1,…,an) i t(1, a-i) > t(0,a-i) (monotonic)
Principal designs a contract for each agent Project succeeds agent i receives pi (otherwise he gets 0)
Players’ utilities, under action profile a=(a1,…,an) and value v: Agent i: ui(a) = t(a)·pi – ci(ai) Principal: u(a,v) = t(a)·(v –Σipi)
Agents are in a game, reach Nash equilibrium.
The Principal’s design parameter: Used to induce the desired equilibrium
The Principal’s “input” parameter.
Example: Read-Once Networks A graph with a given source and sink
Each agent controls an edge, independently succeeds or fails in his individual task (delivering on his edge) Succeeds with probability ɣ<½ with no effort Succeeds with probability 1-ɣ (>½>ɣ) with effort
The project succeeds if the successful edges form a source-sink path.
example: t(1, 1, 0) = Pr { x1 (x2 x3) =1 | a=(1,1,0) } = (1- ɣ) (1- ɣ(1-ɣ))
source sinka1=1
a2=1
a3=0Pr {x1=1}=1- ɣ
Pr {x2=1}=1- ɣ
Pr {x3=1}=ɣ
Nash Equilibrium
Principal’s best contract to induce eq. a=(a1,…,an):
pi= c / i(a-i) for agent i with ai=1
pi= 0 for agent i with ai=0 e.g., (1,0) (1,1)
Agent i’s utility
ui( 1,a-i ) = pi· t( 1,a-i ) – c ui( 0,a-i ) = pi· t(0,a-i )
exerts effort Does not exert effort
),0(),1(),0(),1(
iiiiiii atat
cpauau
i(a-i)
)0,0()0,1(1 tt
cp
)0,1()1,1(2 tt
cp
)1,0()1,1(1 tt
cp
P2=0
Optimal Contract
the principal chooses a profile a*(v) that maximizes her optimal equilibrium utility
1: ),0(),1(
)(),(iai ii atat
cvatvau
Probability of success
Total payments
Research Questions
How does the technology affect the structure of the optimal contracts? Several examples (AND, OR, Majority …) General technologies
What is the damage to the society due to the inability to monitor individual actions? “price of unaccountability”
What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges
from the graph?
Optimal Contracts: simple AND technology2 agents, = ¼, c=1
t(0,0) = 2 = (¼)2=1/16 t(1,0) =t(0,1)= = 3/16; 0 =t(1,0)-t(0,0)=3/16 - 1/16 = 1/8 t(1,1) = = 9/16Principal’s Utility 0 agents exert effort: u((0,0),v) = t(0,0)·v = v/16 1 agent exerts effort: u((1,0),v) = t(1,0)·(v-c/0) = =3/16(v-1/(1/8))=(3/16)v-3/2 2 agents exert effort: u((1,1),v) = t(1,1)·(v-2c/1) = 9v/16-3
s tx1 x2
1: ),0(),1()(),(
iai ii atat
cvatvau
-4
-2
0
2
4
6
8
0 5 10 15
v
U(v)
At value of 6 there is a “jump” from 0 to 2 agents
v v
Optimal Contract Transitions in AND and OR
AND
s tx1 x2
s t
x1
OR
x2ɣ=1/4 optimal to contract with 0 agents up to 6, then with 2 agent
2
Optimal Contract Transitions in AND and OR
Theorem: For any AND technology, there is only one transition, from 0 to n agents.
Theorem: For any OR technology, there are always n transition (any number of agents is optimal for some value).
• We characterize all technologies with 1 transition and with n transitions.
Proofs Idea-AND’s single transition Observation (monotonicity): number of contracted
agents monotonically non-decreasing in v. Proof for AND’s single transition:
At the indifference value between 0 and n agents, contracting with 0<k<n agent has lower utility.
By the above observation, a single transition.
-4
-2
0
2
4
6
8
0 5 10 15
v
U(v
)
The 0 and n indifference value
Transitions in AND and OR
Proof (AND):k: number of contracted agents
this function has a single minimum point, thus maximized at one of the edges 0 or n
)21()1()1(
)1()()()(
1
kknkkn ckv
ktkt
ckvktku
Proofs Idea – OR’s n transitions Let vk be the indifference point between k
and k+1 agents ( u(k,vk) = u(k+1,vk) )
We show that for OR: vk+1> vk
This ensures that k is optimal from vk-1 to vk
-12
-2
8
18
28
38
48
58
0 20 40 60 80
v
U(v
) 0
1
2v0: The 0 ,1
indifference value.
v1: The 1 ,2
indifference value.
v1>v0
Transitions in AND and OR
k: number of contracted agents
solve for v: u(k) = u(k+1), and let v(k) be the solution
we have to show: v(k+1) > v(k) E.g., n=3
v(0)v(1)
v(2)
General Technologies In general we need to know which agents exert
effort in the optimal contract Examples:
In potential, any subset of agents (out of 2n subsets) that exert effort could be optimal for some v.
Which subsets can we get as an optimal contract?
s s
(a) OR-of-ANDs technology
t
A1
A2B2
(b) AND-of-ORs technology
A1
A2
B1
B1
B2
And-of-Ors (AOO) Technology Example: 2x2 AOO technology
Theorem: The optimal contract in any AOO network (with identical OR components) has the same number of agents in each OR-component
Proof: by induction based on following lemmas: Decomposition lemma: if S=TUR is optimal on
f=hg on some v, then T is optimal for h on v·tg(R) and R is optimal for g on v·th(T)
Component monotonicity lemma: the function vth(T) is monotone non-decreasing (same for vtg(R) )
s
A1
A2
B1
B2
t
v{A1,B1} {A1,B1,A2,B2}
s ts t
f = h g
T R
Decomposition Lemma
Proof:
Rigi
i
Tihi
i
Rigi
i
Tihi
i
Rifi
i
Tifi
i
Sifi
i
iR
cRg
iT
cvRgTh
iRTh
cRgTh
iTRg
cRgvRgTh
iS
c
iS
cvRgTh
iS
cvSf
)\()(
)\()()(
)\()()()(
)\()(
)()()(
)\()\()()(
)\()(),( vSU
)\()()\(,,
)\()()()\,0()()\,1()\(,
iRThiSRiSimilarly
iTRgRgiThRgiThiSTigi
fi
hi
fi
f = h g
T R
if S=TUR is optimal on f=hg on some v, then T is optimal for h on v·tg(R) and R is optimal for g on v·th(T)
Component Monotonicity Lemma
Proof: S1 = T1 U R1 optimal on v1
S2 = T2 U R2 optimal on v2<v1
By monotonicity lemma: f(S1) ≥ f(S2) Since f=g·h, f(S1)=h(T1)·g(R1) ≥ h(T2)·g(R2) = f(S2) Assume in contradiction that h(T1) < h(T2).
Since h(T1)·g(R1) ≥ h(T2)·g(R2) , we get g(R1) > g(R2). By decomposition lemma, T1 is optimal for h on v1·g(R1), and T2 is
optimal for h on v2·g(R2) As v1 > v2, and g(R1) > g(R2), T1 is optimal for h on a larger value than T2. Thus, by monotonicity lemma, h(T1) ≥ h(T2)
h gT1R1
f:T2
R2
The function vth(T) is monotone non-decreasing (same for vtg(R) )
And-of-Ors
Theorem: The optimal contract in any AOO network, composed of nc OR-components (of size nl) contracts with the same number of agents in each OR-component. Thus, |orbit(AOO)| ≤ nl+1
Proof: by induction on nc
Base: nc=2assume (k1,k2) is optimal on some v, assume by contradiction k1>k2 (wlog), thus h(k1)>h(k2).By decomposition lemma:
k1 optimal for h on v·h(k2)k2 optimal for h on v·h(k1)>v·h(k2)
but if k2 optimal for a larger value, k2≥k1. in contradiction.
s t
x11
xnlnc
x1nc
xnl1
And-of-Ors
assume (induction) that claim holds for any number of OR components < nc
Assume 1st component has k1 contracted agents
Let g be the conjunction of the other (nc-1) comp.
By decomposition lemma, contract on g is optimal at v·h(k1), thus by induction hypothesis has same number of agents, k2, on each OR component.
Let h2 be conjunction of first two comp.
By decomp. Lemma, contract on h2 is optimal for some value and by induction hypothesis has same number of agents, k3
We get k1=k3 (in first comp. k1 agents contracted), and k2=k3 (in second comp. k2 agents contracted), thus k1=k2
g
hhhh
k1 k2k2k2
h2
k3k3= ==
The Collection of Optimal Contracts Given t we wish to understand how the optimal
contract changes with v (the “orbit”).
Monotonicity Lemma: The optimal contract success probability t(a*(v)) is monotonic non-decreasing with v So is the utility of the principal, and the total
payment Thus, there are at most 2n-1 changes to the
optimal contracts (|Orbit(t)| ≤ 2n)
Is there a structure on the collection of optimal contracts of t?
The Collection of Optimal Contracts Observation 1: in the observable-actions case, only one set of
size k can be optimal (set with highest probability of success)
Observation 2: not all 2n subsets can be obtained Only a single set of size 1 can be optimal (set with highest
probability of success)
Thm: There exists a tech. with optimal contracts
Open question 1: is there a read-once network with exponential number of optimal contracts?
nn
n2
Can a technology have exponentially many different optimal contracts?
Exponential number of optimal contracts (1) Thm: There exists a tech. with optimal contracts Proof sketch:
Lemma 1: all k-size sets in any k-admissible collection can be obtained as optimal contracts of some t
Lemma 2: For any k, there exists a k-admissible collection of k-size sets of size Based on error correcting code
Lemma 3: for k=n/2 we get a k-admissible collection of k-size sets of size , as required.
k
n
n
1
nn
n2
S1
S2
S3
S4
Collection of sets of size k, in which every two sets in it differ by at least two elements
nn
n2
Proof of Lemma 1
S
S\i S\i
k
k-1
1
n
t(S)= ½ - S
t(S\i)= ½ - 2S
S’
S’\i S’\i
t(S’)= ½ - S’
t(S’\i)= ½ - 2S’
• marginal contribution of i S is: t(S) – t(S\i) = S
Define t to ensure that the marginal contribution of at least one agent is very small
Claim: at vs=(ck) / 2S2, the set S is
optimal:• S better than any other set in col. (by derivative of u(S,v))• S better then any other set not in col. (too high payments)
Si iStSt
cvStvSu
)\()()(),(
vkcvSu
SS
20),(
Let vs be v s.t. vkc
S 2
Exponential number of optimal contracts (2) Lemma: For any n ≥ k, there exists an admissible collection of
k-size sets of size
Proof: take error correcting code that corrects 1 error. Hamming distance ≥ 3 admissible Known: codes with (2n/n) code words. Construct a code with sufficient # of k-weight words
XOR every code word with a random word r. weight k w/ prob
Expected number of k-weight code words There exists r such that the expectation is achieved or
exceeded
k
n
n
1
n
k
n2/
k
n
n
1
Research Questions
How does the technology affect the structure of the optimal contracts?
What is the damage to the society / principal due to the inability to monitor individual actions? “price of unaccountability”
What is the complexity of computing the optimal contract?
Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing
edges from the graph?
Observable-Actions Benchmark (first best) Actions are observable Payment: an agent that exerts effort is paid
his cost (c)
Principal’s utility: u(a,v) = v·t(a) – i|ai=1 c
Principal’s utility = social welfare sw(a,v). The principal chooses a*OA, the profile with
maximum social welfare.
Social Price of Unaccountability Definition: The Social Price Of Unaccountability
(POUS) of a technology is the worst ratio (over v) between the social welfare in the observable-action case, and the social welfare in the hidden-action case:
a* - optimal contract for v in the hidden-action case a*OA - optimal contract for v in the observable-action case
Example: AND of 2 agents:
),(
),(sup
*
*
0 vasw
vaswPOU OA
vS
v
0 2Hidden actionsObservable actions 0 2
s t
Principal’s Price of Unaccountability Definition: The Principal’s Price Of Unaccountability
(POUP) of a technology is the worst ratio (over v) between the principal’s utility in the observable-action case, and the principal’s utility in the hidden-action case:
a* - optimal contract for v in the hidden-action case a*OA - optimal contract for v in the observable-action case
),(
),(sup
*
*
0 vau
vauPOU
p
OApvP
Price of Unaccountability - Results Theorem: The POU of AND technology is
unbounded for any fixed n≥2, when unbounded for any fixed ½ when n
Theorem: The POU of OR technology is bounded by 2.5 for any n
111
11n
POU
Research Questions
How does the technology affect the structure of the optimal contracts?
What is the damage to the society due to the inability to monitor individual actions? “price of unaccountability”
What is the complexity of computing the optimal contract?
Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing
edges from the graph?
Complexity of Finding the Optimal Contract
Theorem: There exists a polynomial time algorithm to compute (a*,p), if t is given by a table (exponential input).
Theorem: If t is given by a black box, exponentially many queries may be required to find (a*,p).
Proof: for value v = c(k+ ½),
S’ is optimal Any algorithm must query
all sets of size k=n/2 to find S’ in the worst case
Input: value v, description of t Output: optimal contract: (a*,p)
t(S)=0
t(S)=1
100 0 00 sets of size n/2
sets of size 1
sets of size n
S’
Complexity of Finding the Optimal Contract
Theorem: For read-once networks, the optimal contract problem is #p-hard Proof: reduction from network reliability problem
Open problem 3: is it polynomial for series-parallel networks?
Open problem 4: does it have a good approximation?
Input: value v, description of t Output: optimal contract: (a*,p)
Best Contract Computationin Read-Once Networks
Proof (sketch): an algorithm for this problem can be used to compute t(E) (probability of success)
Player x will enter the contract only for very large value of v (only after all other agents are contracted), call this value vc
At vc, principal is indifferent between E and EU{x}
Gs t t
G’
x ½
v
cEt
Et
c
iE
cvEt
iE
cvEt
x
x
Ei xtix
xEi
tix
x
2)21(
)1()(
)21)(()\()1()1()(
)\()(
Research Questions
How does the technology affect the structure of the optimal contracts?
What is the damage to the society due to the inability to monitor individual actions? “price of unaccountability”
What is the complexity of computing the optimal contract?
Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing
edges from the graph?
Mixed Strategies
In the non-strategic case: NO (convex combination) What about the agency case?
Extended game: qi : probability that agent i exerts effort
t( qi,q-i ) = qi·t(1,q-i )+ (1-qi )·t(0,q-i )
Marginal contribution: i(q-I ) = t(1,q-i ) - t(0,q-i ) ≥ 0
Can mixed-strategies help the principal ?
What is the price of purity ?
Nash Equilibrium in Mixed Strategies Claim: agent i’s best-response is to mix with probability
q (0,1) only if she is indifferent between 0 and 1
Agent i’s utility:
Principal’s utility:
),0(),1(),0(),1(
ii
iiiiii qtqt
cpququ
i
iiii q
q
qtcqu
)(
)()(
0| )()(),(
iqi ii
i
q
cvqtvqu
Agent i’s utility
ui( 1,q-i ) = pi· t( 1,q-i ) – ci ui( 0,q-i ) = pi· t(0,q-i )
High effort Low effort
Example:OR with two agents Optimal contract for v=110
Pure strategies: both agents contracted: u = 88.12... Mixed strategies: q1=q2=0.96..: u=88.24...
Two observations: q1=q2 in optimal contract Principal’s utility is improved, but only slightly
How general are these observations?
s t
=0.25
=0.25
Optimal Contract in OR Technology Lemma: For any anonymous OR (any ,n,c,v), k{0,1,…,n}
agents exert effort with equal probabilities q1=…=qk (0,1], and n-k agents shirk. i.e. optimal profile: (0n-k, qk)
Proof (skecth): suppose by contradiction that (qi,qj,q-ij) s.t. qi,qj (0,1) and qi > qj is optimal
qj
qi
(qi,qj,q-ij)
(qi-ε,qj+yε,q-ij)j
i
j
i
q
q
qt
qty
)12(1
)12(1
/
/
For a sufficiently small ε , success probability increases, and total payments decrease. In contradiction to optimality
Optimal Contract in OR TechnologyExample: OR with 2 agents:
Price of Purity (POP)
Definition: POP is the ratio between principal’s utility in mixed strategies and in pure strategies
)*(
0)(|*
0
)())(*(
))(())(*(
)(*
vSi ii
i
vqi ii
i
v
ac
vvSt
vqc
vvqt
SuptPOP i
Optimal pure contract
Optimal mixed contract
Price of Purity
Definition: technology t exhibits increasing returns to scale (IRS) if for any i and any b ≥ a
t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i) decreasing returns to scale (DRS) if for any i and any b ≥ a
t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i)
Observations: AND exhibits IRS, OR exhibits DRS
Theorem: for any technology that exhibits IRS, optimal contract is obtained in pure strategies e.g., AND
Price of Purity
For any anonymous DRS technology, POP ≤ n For anonymous OR with n agents, POP ≤ 1.154.. For any anonymous technology with 2 agents, POP ≤
1.5 For any technology (not necessarily anonymous, but with
identical costs) with 2 agents, POP ≤ 2
Observation: the payment to each agent in a mixed profile is greater than the min payment in a pure profile and smaller than the max payment in a pure profile
Research Questions
How does the technology affect the structure of the optimal contracts?
What is the damage to the society due to the inability to monitor individual actions? “price of unaccountability”
What is the complexity of computing the optimal contract?
Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing
edges from the graph?
as before
Free-Labor So far, technology was exogenously given Now, suppose the principal has control over the technology in
that he can ex-ante remove some agents from the graph
Example: OR with 2 agents
Action set of agent i: ai {1,0,} 1: exert effort – succeed with probability d. cost=c 0: do not exert effort - succeed with probability d.
cost=0 : do not participate – succeed with probability 0. cost=0
Action “wastes free-labor” since action “0” increases the success probability with no additional cost
s t s t
Free-Labor
The answer is: YES Example: OR technology, n=2, =0.2
Theorem: for technologies with increasing marginal contribution (e.g., AND), utilizing all free-labor is always optimal
Are there scenarios in which the principal gains utility from “wasting free-labor”?
s t
=0.2
=0.2
v0 1 2
1 removed
Analysis of OR
Lemma: for any OR with n agents and which is small enough, there exists a value for which in the optimal contract one agent exerts effort and no other agent participates
=0.49=0.25=0.01
Version of the Braess’s Paradox A project is composed of 2 essential components: A and B And-of-Ors (AOO): allow interaction between teams
Or-of-Ands (OOA): don’t allow interaction between teams
Obviously, AOO is superior in terms of success probability
s t
B2
B1
A2
A1
s t
A1 B1
B2A2
project succeeds if at least one of the following pairs succeed: (A1,B1) ; (A1,B2) ; (A2,B1) ; (A2,B2)
project succeeds if at least one of the following pairs succeed: (A1,B1) ; (A2,B2)
Version of the Braess’s Paradox
s ti =1
B2
B1A1
A2
s t
A1 B1
B2A2
remove middle edge
s t
B2
B1
A2
A1
don’t remove middle edge
Or-of-Ands “wastes free-labor”. Could the principal gain utility from removing middle edge?
s t s t
u(2,2) = 75.59..
Example: =0.2, v=110
u(1,1) = 74.17..>
Conclusion: it may be beneficial for the principal to isolate the teams
And-of-Ors
Or-of-Ands
Summary “Combinatorial Agency”: hidden actions in combinatorial
settings
Computing the optimal contract in general is hard
Natural research directions: technologies whose contract can be computed in
polynomial time Approximation algorithms
Many open questions remain
Thank You
Related Literature
[Winter2004] Incentives and discrimination The effect of technology on optimal contract (full implementation)
[Winter2005] Optimal incentives with information about peers
[Ronen2005][Smorodinsky and Tennenholtz2004,2005] Multi-party computation with costly information
[Holmstrom82] Moral hazard in teams Budget-balanced sharing rules