Cooperative/coalitional game theory

A composite of slides taken from Vincent Conitzer and Giovanni Neglia

(Modified by Vicki Allan)

1

Prisoners Dilemma rules

1. Binding agreements are not possible. Note in Prisoners dilemma, if binding agreements were possible, there would be no dilemma

2. Utility is given directly to individuals as a result of individual actions. So, I am not worried about collective utility.

Solution: cooperative game theory

2

Cooperative Games

• Coalitions – set of agents• Grand coalition – all agents work together• Characteristic function: v:2Ag R subsets of agents are assigned a

value.• Simple coalitional game: a coalition has value 0

or 1. A voting system can be thought of as a simple game (have enough votes to win or do not)

3

Three Parts of Cooperative Action

1. Coalition Structure Generation – who should work together. Know only characteristic function. Want to earn a lot, but so does everyone else.

2. Optimization Problem – given coalitions and tasks, which coalition should do each problem.

3. Dividing the utility- What is fair? What is stable?

4

In Multi-Agent Systems

• Agents create their own coalitions rather than have a centralized decision.

5

Scenario 1 – Bargain Buy(supply-demand)

• Store “Bargain Buy” advertises a great price

• 300 people show up• 5 in stock• Everyone sees the advertised

price, but it just isn’t possible for all to achieve it

Scenario 2 – hiring a new PhD(strategy)

• Universities ranked 1,2,3• Students ranked a,b,cDilemma for second tier

university• offer to “a” student• likely rejected• delay for acceptance• “b” students are gone

Scenario 3 – selecting a spouse(agency)

• Bob knows all the characteristics of the perfect wife

• Bob seeks out such a wife• Why would the perfect

woman want Bob?

Scenario 4 Blowhard (trust)What if one person talks a good story, but his claims of skills are really inflated?

He isn’t capable of performing. the task.

Scenario 5 Power in diversityYou consult local traffic statistics to find a good route home from work

But so does everyone else

The coalition is completed and rewards are earned. How are they fairly divided among agents with various contributions?If organizer is greedy, why wouldn’t others replace him with a cheaper agent?

Consider the Stable Marriage Problem – small coalition

• In a small town there are n men and n women who wish to be wed. Each person would be happy to be married to any of the people of the opposite sex but has a definite preference ranking of the possible marriage partners.

• If marriages are arranged arbitrarily, some of the marriages can be unstable in the following sense. Suppose Alice marries Bob and Carol marries Dave. If Alice prefers Dave to Bob and Dave prefers Alice to Carol, then Alice and Dave will leave their partners; in this situation, we say that the marriages of Alice and Dave are unstable.

• Goal is stability not just the most partners: the difference between local and global utility.

12

Core• Ex: software engineering group project. Who would you

choose to work with? Who would also choose you? Coalition doesn’t form unless everyone is happy with it.

• Stability is necessary but not sufficient for a coalition to form. If it isn’t stable, someone will defect.

If there are multiple stable coalitions, another may form instead.

• core: set of feasible distributions of payoff to members of the grand coalition so none will defect.

• We require the outcome (payments) to be both feasible (able to pay) and efficient (all utility distributed).

• Pareto efficiency is the efficiency referred to.

13

• We say a coalition objects to (or blocks) an outcome if every member of the coalition is strictly better off in some other feasible outcome.

• Asking if the grand coalition is stable means, “Is the core non-empty?”

• The point of the core is to study stability not fairness.

14

Example• v{a} = 1 v{b} = 3 v{a,b} = 5• How should we divide the profit if they work together?

Which are feasible, fair, efficient, stable?• <a=5,b=0>• <a=4,b=1>• <a=3,b=2>• <a=2,b=3>• <a=1,b=4>• <a=1.5,b=3.5>• <a=0,b=5>• <a=2,b=2>• <a=2,b=4>

15

So what are concerns?

• What if there are no outcomes in the core?• What if there are multiple outcomes in the

core, how do you pick?• Looking at all possible distributions of utility is

exponential as you have 2n possible objecting subsets to consider.

16

Example• v{1} = 2 v{2} = 2 v{3} = 2 • v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6• How should we divide the profit if they work

together? Which are feasible, fair, efficient, stable?

• <2,2,2>• <3,3,0>• <1,2,3>• <1,2.5,2.5>• <2,2.5, 1.5>

17

More Examples Emptiness & multiplicity

18

• Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0)– v({1, 2, 3}) = 6, – v({1, 2}) = v({1, 3}) = v({2, 3}) = 6,– v({1}) = v({2}) = v({3}) = 0

More Examples Emptiness & multiplicity

19

• Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0)– v({1, 2, 3}) = 6, – v({1, 2}) = v({1, 3}) = v({2, 3}) = 6,– v({1}) = v({2}) = v({3}) = 0

• Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each).

More Examples Emptiness & multiplicity

20

• Example 2:– v({1, 2, 3}) = 18, – v({1, 2}) = v({1, 3}) = v({2, 3}) = 10,– v({1}) = v({2}) = v({3}) = 0

More Examples Emptiness & multiplicity

21

• Example 2:– v({1, 2, 3}) = 18, – v({1, 2}) = v({1, 3}) = v({2, 3}) = 10,– v({1}) = v({2}) = v({3}) = 0

• Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), …

Fairness• Could we get people to join a coalition by

saying, “Will you agree to a fair solution even though it is not in the core?”

• In examplev{1} =0 v{2} = 0 v{3} = 0 v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6What would a fair but unstable solution be?• If we had extra (that no one could really

demand), how would we divide any surplus?

22

Terms• Marginal Contribution (value added): μi(C) = v(C U {i}) – v(C)

• shi is what i is given (its share)• Symmetry: Agents that make the same contribution should get the

same payoff• Dummy player – never increases the value of the coalition beyond

what it could earn alone μi(C) = v(C U {i}) – v(C) = v{i} for every C

• Alternate definition of Dummy: μi(C) = 0 for every C• Additivity: if we add two games defined by v and w by letting (v+w)

(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w• most controversial axiom (for example, participant i’s cost-share of a

runway and terminal is it’s cost-share of the runway plus his cost-share of the terminal)

23

Superadditivity

24

• v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B)

• Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate.

• There is a synergy – if not, coalitions make no sense.• Usually makes sense• Previous examples were all superadditive• Given this, always efficient for grand coalition to form• Without superadditivity, finding a core is not possible.

The Shapley value [Shapley 1953]

25

• In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition)

• The marginal contribution scheme is unfair because it depends on the ordering of the agents

• One way to make it fair: average over all possible orderings• Let MC(i, π) be the marginal contribution of i in ordering π• Then i’s Shapley value is ΣπMC(i, π)/(n!)• The Shapley value is always in the core for convex games• … but not in general, even when core is nonempty, e.g.

– v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,– v = 0 everywhere else

Example: v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,v = 0 everywhere else

1 2 3

123

132

213

231

312

321

avg

26

Compute the Shapley value for each.

Is the solution in the core?

Axiomatic characterization of the Shapley value

27

• The Shapley value is the unique solution concept that satisfies:– (Pareto) Efficiency: the total utility is the value of the grand

coalition, Σi in Nu(i) = v(N)– Symmetry: two symmetric players (add the same amount to

coalitions they join) must receive the same utility– Dummy: if v(S {i}) = v(S) for all S, then i must get 0– Additivity: if we add two games defined by v and w by

letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w.

Additivity Example• Game a

28

Coalition Value 1 2 31 0 123 0 5 12 0 132 0 6 03 0 213 5 0 1

12 5 231 6 0 013 0 312 0 6 023 0 312 6 0 0

123 6 17/6 17/6 2/6

Coalition Value 1 2 31 0 123 0 5 02 0 132 0 0 53 0 213 5 0 0

12 5 231 0 0 513 5 312 5 0 023 5 312 0 5 0

123 5 10/6 10/6 10/6

• Game b

a+b

29

Coalition Value 1 2 3

1 0 123 0 10 1

2 0 132 0 6 5

3 0 213 10 0 1

12 10 231 6 0 5

13 5 312 5 6 0

23 5 312 6 5 0

123 11 27/6 27/6 12/6

Notice how their value in the combined game is the sum of their values in the original games

Shapley Result• Satisfies all three fairness axioms (the last

three listed)• If you came up with some other method that

satisfied all fairness axioms, it would be Shapley. In other words, the Shapley value is the UNIQUE value that satisfies all fairness axioms.

30

Computing a solution in the core

31

• How do you even represent the characteristic function which defines the problem? exponential in the number of agents.

• Can use linear programming:– Variables: u(i)– Distribution constraint: Σi in Nu(i) = v(N)– Non-blocking constraints: for every S, Σi in Su(i) ≥ v(S)

• Problem: number of constraints exponential in number of players (as you have values for all possible subsets)

• … but is this practical?

Convexity

32

• v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B)

• In other words, the amount A adds to B (in forming the union) is at least as much it adds to the intersection.

• One interpretation: the marginal contribution of an agent is increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution. Precisely, the marginal contribution of A to B is v(AUB)-v(B)

• Example, suppose we have three independent researchers. When we combine them at the same university, the value added by A is greater if the set is larger.

Example• v{1} = 0 v{2} = 0 v{3} = 0 • v{1,2} = 8 v{2,3} = 8 v{1,2,3} = 12

• Let A = {1,2} and B={2,3}

• v(AUB)-v(B) ≥ v(A)-v(A∩B)

33

Convexity

34

• In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.)

• One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1})– Marginal contribution scheme- each agent is rewarded by what it adds

to the union.– Works for any ordering of the agents

Theory of cooperative games with sidepayments

• It starts with von Neumann and Morgenstern (1944)

• Two main (related) questions:– which coalitions should form?– how should a coalition (which forms) divide its

winnings among its members?• The specific strategy the coalition will

follow is not of particular concern...• Note: there are also cooperative games

without sidepayments35

Example: Minimum Spanning Tree game

• For some games the characteristic form representation is immediate

• Communities 1,2 & 3 want to be connected to a nearby power source– Possible transmission links & costs as in figure

36

source

1

2

3100

40

5020

4040

Example: Minimum Spanning Tree game

• Communities 1,2 & 3 want to be connected to a nearby power source

37

v(void) = 0v(1) = 0v(2) = 0v(3) = 0v(12) = -90 + 100 + 50 = 60v(13) = -80 + 100 + 40 = 60v(23) = -60 + 50+ 40 = 30v(123) = -100 + 100 + 50+ 40 = 90

A strategically equivalent game.We show what is gained from the

coalition. How to divide the gain?

source

1

2

3100

40

5020

4040

The Core• What about MST game? We use value to mean

what is saved by going with a group.– v(void)= v(1) = v(2) = v(3)=0– v(12) = 60, v(13) = 60, v(23) = 30– v(123) = 90

• Analitically, in getting to a group of three, you must make sure you do better than the group of 2 cases:– x1+x2>=60, iff x3<=30– x1+x3>=60, iff x2<=30– x2+x3>=30, iff x1<=60

381 2

The Core• Let’s choose an imputation in the core:

x=(60,25,5)• The payoffs represent the savings, the

costs under x are – c(1)=100-60=40, – c(2)=50-25=25– c(3)=40-5=35

39

source

1

2

3100

40

5020

4040

FAIR?

The Shapley value: computation• MST game

– v(void) = v(1) = v(2) = v(3)=0– v(1,2) = 60, v(1,3) = 60, v(2,3) = 30, v(1,2,3) = 90

40

1 2 3

123

132

213

231

312

321

avg

Value added by

Coa

litio

ns

The Shapley value: computation• MST game

– v(void) = v(1) = v(2) = v(3)=0– v(12) = 60, v(13) = 60, v(23) = 30, v(123) = 90

41

1 2 3

123

0 60 30

132

213

231

312

321

avg

Value added by

Coa

litio

ns

The Shapley value: computation

• MST game– v(void) = v(1) = v(2) = v(3)=0– v(12) = 60, v(13) = 60, v(23) = 30, v(123) = 90

1 2 3

123

0 60 30

132

0 30 60

213

60 0 30

231

60 0 30

312

60 30 0

321

60 30 0

avg

40 25 25

Value added by

Coa

litio

ns

Discountper person

The Shapley value: computation

• A faster way• The amount player i contributes to

coalition S, of size s, is v(S)-v(S-i)• This contribution occurs for those

orderings in which i is preceded by the s-1 other players in S, and followed by the n-s players not in S

• ki = 1/n! S:i in S (s-1)! (n-s)! (v(S)-v(S-i))

43

The Shapley value has been used for cost sharing. Suppose threeplanes share a runway. The planes require 1, 2, and 3 KM to land. Let’s label those planes 1, 2, and 3.Thus, a runway of 3 must be build, but how much should each pay?

Instead of looking at utility given, look at how much increased cost was required.

1 2 3

123

132

213

231

312

321

avg

The Shapley value has been used for cost sharing. Suppose threeplanes share a runway. The planesrequire 1, 2, and 3 KM to land.Thus, a runway of 3 must bebuild, but how much should each pay?

Instead of looking at utility given, look at how much increased cost was required.

An application: voting power

• A voting game is a pair (N,W) where N is the set of players (voters) and W is the collection of winning coalitions, s.t.– the empty set is not in W (it is a losing coalition) – N is in W (the coalition of all voters is winning)– if S is in W and S is a subset of T then T is in W

• Also weighted voting game can be considered in which each player has w voting weight an a quota is required to win.

• The Shapley value of a voting game is a measure of voting power (Shapley-Shubik power index)– The winning coalitions have payoff 1– The losing coalitions have payoff 0

46

All powers sum to 1. What is power?

• {99,99,1: 100} Any two win, so all have equal power 1/3.

• {6,4,2: 10} Having non-zero weight does not guarantee power. 2 has zero power. It is a dummy.

• If the same agent exists in all winning coalitions, the core is non-empty (as no subgroup can pull out and win).

• Simple game: any coalition is either winning (1) or losing (0)

• It seems clear that some simple games could not be formulated in these terms. (incomplete)

47

An application: voting power

• The United Nations Security Council in 1954– 5 permanent members (P)– 6 non-permanent members (N)– the winning coalitions had to have at least 7 members,– but the permanent members had veto power

• A winning coalition had to have at least seven members including all the permanent members

• The seventh member joining the coalition is the pivotal one: he makes the coalition winning

• Is this a fair voting system?

48

An application: voting power• 11! possible orderings• Power of non permanent members

– (PPPPPN)N(NNNN)– 5 ways to pick which N comes before and 4! possible arrangements for

those that come after. (5! total choices)– 6! possible arrangements for (PPPPPN)– The total number of arrangements in which an N is pivotal is 6!5!– The power of all non permanent members is 6!5!/11!, each getting .0022

• The power of each permanent members is.1974• The ratio of power of a P member to a N member is 91:2• In 1965

– 5 permanent members (P)– 10 non-permanent members (N)– the winning coalitions has to have at least 9 members,– the permanent members keep the veto power

• Similar calculations lead to a ratio of power of a P member to a N member equal to 105:1 (.19627:.001865)

49

How to represent the characteristic function (in condensed way)?

• Induced subgraphsNode represents agents. Arc represents added

benefit if source/target agents are both in the same coalition

50

Example

51

V(ab) = 3v(ac) = 2v(abc) = 5v(bd) = 6v(d) = 5v(cd) = 9

v(abc) = 7 How would we change the diagram?

Example

52

What do you think of this method? Any concerns?

Enforces superadditivity.

Clearly, in linear space, we can’t expect to represent an exponentially rich set of rewards.

What if B added to either A or C adds 3, but the gain isn’t experienced twice?

Do we need to worry about cases where B adds 3 to A, but if C is also present, the gain is actually negative?

We say the method is …

• succinct – as it is much easier to represent• incomplete – as there are characteristic

functions for which the method doesn’t work• How would we use the weighted graph to

compute Shapley value? It seems like each node should get half the weight of the arc it shares.

• We can prove that by breaking in into small problems and using Shapley’s additivity axiom.

53

Marginal Contribution Nets

• Extension of idea of induced subgraphs where total weight is sum of weight due to parts.

• We have a series of patterns (boolean expressions involving agents) and values. To determine the characteristic value for a subset, you add up the values (positive or negative) associated with the rule.

54

Example

#1: a Λ b 5 marginal increase#2: b 2#3: c 4#4: b Λ ¬c -2 so

55

Possible with induced graphs, right?

Coalition Value Rules

a 0noneb 0#2,#4c 4#3ab 5#1,#2,#4ac 4#3bc 6#2. #3abc 11#1,#2,#3

Thought questions?• Are the rules powerful enough? According to researchers, all characteristic

functions can be represented, but we aren’t told how to generate the rules!!!

• Plus, is it really any simpler? Instead of showing all combinations, you now have all the original combinations PLUS the combinations with negative components.

56

Majority game

• A group of agents S has a value of 1 if |S| > n/2.

• Can we do this with Induced subgraph?• Can we do this with marginal contribution

nets?

57