PDL as a Multi-Agent Strategy Logic

Jan van EijckCWI & ILLC, Amsterdam

September 17, 2012

Abstract

We propose a new perspective on PDL as a multi-agent strategic logic (MASL). This logicfor strategic reasoning has group strategies as first class citizens, and brings game logic closerto standard modal logic. We show that MASL can express key notions of game theory, socialchoice theory and voting theory in a natural way. We then present a sound and complete proofsystem for MASL. We end by tracing connections to a number of other logics for reasoningabout strategies.

Overview

• The Pebble Puzzle

• Reasoning About Programs

• Reasoning About Actions

• Strategic Games: the Prisoner’s Dilemma

• Group Strategies in Games

• Key Notions: Best Response, Nash Equilibrium

• Voting as a Multi-Agent Game

• MASL: Language and Expressiveness

• Soundness and Completeness

• Connections, Further Work

The Pebble Puzzle

An urn cointains 70 pebbles; 35 of them are white and 35 are black.There is a pile of black pebbles available outside the urn.

Pebble Algorithm

• While there are still enough pebbles in the urn:

– pick two pebbles;if they have the same colour, put back a black pebbleotherwise, put back the white pebble.

In every step of the algorithm one pebble gets removed. After 69steps, there is one pebble left. What is its colour?

module Pebbles where

data Color = W | B deriving (Eq,Show)

drawPebble :: [Color] -> [Color]drawPebble [] = []drawPebble [x] = [x]drawPebble (W:W:xs) = drawPebble (B:xs)drawPebble (B:B:xs) = drawPebble (B:xs)drawPebble (W:B:xs) = drawPebble (W:xs)drawPebble (B:W:xs) = drawPebble (W:xs)

numberW :: [Color] -> IntnumberW = length . (filter (\x -> x == W))

parityW :: [Color] -> IntparityW xs = mod (numberW xs) 2

prop_invariant = \xs ->parityW xs == parityW (drawPebble xs)

Sir Tony Hoare

Formal Specification With Hoare Triples

In general a triple

initial state – statement – final state{P} S {Q}

has the following operational meaning:

If execution of S in a state that satisfies P terminates, thenthe termination state is guaranteed to satisfy Q.

Such triples {P}S {Q} are called Hoare triples after Tony Hoare.

The predicate for the initial state is called the precondition, and thepredicate for the final state is called the postcondition.

assignment {ϕva} v := a {ϕ}

skip {ϕ} SKIP {ϕ}

sequence{ϕ} C1 {ψ} {ψ} C2 {χ}

{ϕ} C1;C2 {χ}

conditional choice{ϕ ∧B} C1 {ψ} {ϕ ∧ ¬B} C2 {ψ}

{ϕ} if B then C1 else C2 {ψ}

guarded iteration{ϕ ∧B} C {ϕ}

{ϕ} while B do C {ϕ ∧ ¬B}

precondition strengtheningN |= ϕ′ → ϕ {ϕ} C {ψ}

{ϕ′} C {ψ}

postcondition weakening{ϕ} C {ψ} N |= ψ → ψ′

{ϕ} C {ψ′}

Vaughan Pratt

Hoare Logic as a Fragment of Dynamic Logic

Hoare logic is a fragment of a more general system of (propositional)dynamic logic.

The language of propositional dynamic logic was defined by Pratt in[13, 14] as a generic language for reasoning about computation. Ax-iomatisations were given independently by Segerberg [16], Fisher/Ladner[8], and Parikh [10]. These axiomatisations make the connection be-tween propositional dynamic logic and modal logic very clear.

PDL Language

Let p range over the set of basic propositions P , and let a range overa set of basic actions A. Then the formulae ϕ and programs α ofpropositional dynamic logic are given by:

ϕ ::= > | p | ¬ϕ | ϕ1 ∨ ϕ2 | 〈α〉ϕα ::= a |?ϕ | α1;α2 | α1 ∪ α2 | α∗

Abbreviation:[α]ϕ abbreviates¬〈α〉¬ϕ.

Expressing Hoare Triples in PDL

Floyd-Hoare correctness assertions are expressible in PDL, as fol-lows. If ϕ, ψ are PDL formulae and α is a PDL program, then

{ϕ} α {ψ}

translates intoϕ→ [α]ψ.

Clearly, {ϕ} α {ψ} holds in a state in a model iff ϕ→ [α]ψ is true inthat state in that model.

PDL Axiomatisation

Aioms are all propositional tautologies, plus the following axioms(we give box ([α])versions here, but every axiom has an equivalentdiamond (〈α〉) version):

(K) ` [α](ϕ→ ψ)→ ([α]ϕ→ [α]ψ)

(test) ` [?ϕ1]ϕ2 ↔ (ϕ1 → ϕ2)

(sequence) ` [α1;α2]ϕ↔ [α1][α2]ϕ

(choice) ` [α1 ∪ α2]ϕ↔ [α1]ϕ ∧ [α2]ϕ

(mix) ` [α∗]ϕ↔ ϕ ∧ [α][α∗]ϕ

(induction) ` (ϕ ∧ [α∗](ϕ→ [α]ϕ))→ [α∗]ϕ

and the following rules of inference:

(modus ponens) From ` ϕ1 and ` ϕ1 → ϕ2, infer ` ϕ2.

(modal generalisation) From ` ϕ, infer ` [α]ϕ.

The Loop Invariance Rule

In the presence of the other axioms, the induction axiom is equivalentto the loop invariance rule:

ϕ→ [α]ϕ

ϕ→ [α∗]ϕ

Deriving Hoare Rules in PDL

The Floyd-Hoare inference rules can now be derived in PDL. As anexample we derive the rule for guarded iteration:

{ϕ ∧ ψ} α {ψ}{ψ}WHILE ϕ DO α {¬ϕ ∧ ψ}

Let the premise {ϕ ∧ ψ} α {ψ} be given, i.e. assume (1).

` (ϕ ∧ ψ)→ [α]ψ. (1)

We wish to derive the conclusion

` {ψ}WHILE ϕ DO α {¬ϕ ∧ ψ},

i.e. we wish to derive (2).

` ψ → [(?ϕ;α)∗; ?¬ϕ](¬ϕ ∧ ψ). (2)

From (1) by means of propositional reasoning:

` ψ → (ϕ→ [α]ψ).

From this, by means of the test and sequence axioms:

` ψ → [?ϕ;α]ψ.

Applying the loop invariance rule gives:

` ψ → [(?ϕ;α)∗]ψ.

Since ψ is propositionally equivalent with ¬ϕ → (¬ϕ ∧ ψ), we getfrom this by propositional reasoning:

` ψ → [(?ϕ;α)∗](¬ϕ→ (¬ϕ ∧ ψ)).

The test axiom and the sequencing axiom yield the desired result (2).

Strategic Games: The Prisoner’s Dilemma

cooperate defectcooperate c, c c, d

defect d, c d, d

With output function o : {c, d}2 → {x, y, z, u}2:

cooperate defectcooperate x, x y, z

defect z, y u, u

Fixing the preferences of the players: z > x > u > y.

With numerical utilities:

cooperate defectcooperate 2, 2 0, 3

defect 3, 0 1, 1

Group Strategies in PD Game are the Strategy Profiles

cc cd

dc dd

cc cdcd

cc

dc dd

dd

dc

dccc ddcd

Key Notions: Best Response

Let (s′i, s−i) be the strategy profile that is like s for all players excepti, but has si replaced by s′i. A strategy si is a best response in s if

∀s′i ∈ Si ui(s) ≥ ui(s′i, s−i).

Example in PD game. Let s = (d, c). The first player defects, thesecond player cooperates.

Is d a best response for player 1 in (d, c)?

Yes, because (d, c) gives payoff 3 for player 1, while the alternative(c, c) only gives payoff 2. So player 1 cannot do better than play d.

John Nash

Key Notions: Pure Nash Equilibrium

A strategy profile s is a (pure) Nash equilibrium if each si is a bestresponse in s:

∀i ∈ N ∀s′i ∈ Si ui(s) ≥ ui(s′i, s−i).

A game G is Nash if G has a (pure) Nash equilibrium.

(d, d) is a Nash equilibrium for the PD game, so the PD game is Nash.

Charles Dodgson, also known as Lewis Carroll

Voting as a Multi-Agent Game

Voting can be seen as a form of multi-agent decision making, with thevoters as agents [7].

Voting is the process of selecting an item or a set of items from a finiteset A of alternatives, on the basis of the stated preferences of a set ofvoters.

We assume that the preferences of a voter are represented by a ballot:a linear ordering of A. Let ord(A) be the set of all ballots on A.

If there are three alternatives a, b, c, and a voter prefers a over b and bover c, then her ballot is abc.

Example

• Assume there are three voters {1, 2, 3}.

• Assume there are three alternatives {a, b, c}.

• Then profiles are vectors of ballots.

• Example profile where the first voter has ballot abc, the secondvoter has ballot abc, the third voter has ballot bca, and so on:

(abc, abc, bca).

Voting Rules

A voting rule V for set of alternatives A is a function from A-profilesto P+(A) (the set of non-empty subsets of A).

If V (P) = B, then the members of B are called the winners of Punder V .

A voting rule is resolute if V (P) is a singleton for any profile P.

Example voting rule: voting by absolute majority.

Selects an alternative with more than 50 % of the votes as winner, andreturns the whole set of alternatives otherwise.

(abc, abc, bca).

Absolute majority selects a as winner, for a has two votes, b has one.

Strategizing in Voting: Gibbard-Satterthwaite

Ballot Profile Vector of ballots.

Resolute Voting Rule Function V from ballot profiles to alternatives.

P ∼i P′ P and P′ differ at most in the ballot for i.

Strategy-Proofness V is strategy-proof if P ∼i P′ implies that, fromthe perspective of P, V (P) is at least as good for i as V (P′).

(Weak) Non-Imposition V has at least three possible outcomes.

Dictatorship V is a dictatorship if there is some voter k such that Vmaps any profile to the top-ranking alternative in the k-ballot.

GS Theorem Any resolute voting rule that is strategy-proof and weaklynon-imposed is a dictatorship.

Group Actions in Voting Games

Voters: 1, 2, 3. Alternatives: a, b, c. Voting rule: absolute majority.

1 is the row player, 2 the column player, and 3 the table player

a:

a b ca a a ab a b a, b, cc a a, b, c c

b:

a b ca a b a, b, cb b b bc a, b, c b c

c:

a b ca a a, b, c cb a, b, c b cc c c c

Payoffs

• Assume ballot abc.

• Then a > b > c, and plausibly a > {a, b, c} > c.

• Assume {a, b, c} and b give the same payoff.

• Fix the payoff function for voters i of type abc as

ui(a) = 2, ui(b) = ui({a, b, c}) = 1, ui(c) = 0.

• If we do similarly for the other voter types, then this fixes thestrategic game for voting according to the majority rule over theset of alternatives {a, b, c}.

Suppose 1 has ballot abc, 2 has bca, and has 3 cab

This fixes the strategic game:

a:

a b ca (2, 0, 1) (2, 0, 1) (2, 0, 1)

b (2, 0, 1) (1, 2, 0) (1, 1, 1)

c (2, 0, 1) (1, 1, 1) (0, 1, 2)

b:

a b ca (2, 0, 1) (1, 2, 0) (1, 1, 1)

b (1, 2, 0) (1, 2, 0) (1, 2, 0)

c (1, 1, 1) (1, 2, 0) (0, 1, 2)

c:

a b ca (2, 0, 1) (1, 1, 1) (0, 1, 2)

b (1, 1, 1) (1, 2, 0) (0, 1, 2)

c (0, 1, 2) (0, 1, 2) (0, 1, 2)

Nash Equilibrium

If they all cast their vote according to their true ballot, then 1 votes a,2 votes b and 3 votes c.

Then the outcome is a tie, {a, b, c}, with payoff (1, 1, 1).

This is a Nash equilibrium: the vote cast by each player is a bestresponse in the strategy profile.

Changing the Game

Now let’s change the voting rule slightly, by switching to majorityvoting with tie breaking, where abc as the tie breaking order.

This changes the majority rule into a resolute voting rule.

It also changes the strategic game . . .

New Strategic Game

a:

a b ca (2, 0, 1) (2, 0, 1) (2, 0, 1)

b (2, 0, 1) (1, 2, 0) (2, 0, 1)

c (2, 0, 1) (2, 0, 1) (0, 1, 2)

b:

a b ca (2, 0, 1) (1, 2, 0) (2, 0, 1)

b (1, 2, 0) (1, 2, 0) (1, 2, 0)

c (2, 0, 1) (1, 2, 0) (0, 1, 2)

c:

a b ca (2, 0, 1) (2, 0, 1) (0, 1, 2)

b (2, 0, 1) (1, 2, 0) (0, 1, 2)

c (0, 1, 2) (0, 1, 2) (0, 1, 2)

Coalition Formation

• If the players all vote according to their true preference, the out-come is a because of the tie breaking, with payoff given by (2, 0, 1).

• But this is no longer a Nash equilibrium, for player 2 can improvehis payoff from 0 to 1 by casting vote c, which causes the outcometo change into c, with payoff (0, 1, 2).

• The strategy triple (a, c, c) is a Nash equilibrium.

• The voting rule seems to favour voter 1 with ballot abc, becausethe tie breaking rule uses this order for tie breaking.

• Still, the voter with this ballot ends up losing the game, becausethe other two players have an incentive to form a coalition againstplayer 1.

A Language for MASL

• Basic actions: the full strategy profiles.

• Assumption for convenience: the choice of strategies is the samefor each player (as in voting).

• State of the game: a strategy vector where each player has deter-mined her strategy.

• Individual strategies emerge as unions of group strategies.

Example: strategy c for the first player in the PD game

cc cd

dc dd

c,?? c,??c,??

c,?? c,?? c,??c,??

Strategy Terms

• Strategy terms of MASL are:

t ::= a | ?? | !!

• Here a ranges over the set of all player strategies A.

• “??” denotes an individual strategy for an adversary player.

• “!!” denotes the current strategy of a player.

• Random terms serve to model what adversaries do, and currentterms serve to model what happens when players stick to a previ-ous choice.

MASL Actions and Formulas

c ::= (t1 . . . , tn)

The MASL strategy vectors occur as atoms and as modalities in MASLformulas.

Assume that p ranges over a set of game outcome values, that is:assume an outcome function o : AN → P , and let p range over P .

The MASL language is built in the usual PDL manner by mutualrecursion of action expressions and formulas:

ϕ ::= > | c | p | ¬ϕ | ϕ1 ∧ ϕ2 | [γ]ϕ

γ ::= c | ?ϕ | γ1; γ2 | γ1 ∪ γ2 | γ∗

Interpretation

Let s ∈ An. Then s is a strategy profile, with individual strategiestaken from A.

Let i ∈ N .

Then [[·]]A,s,i is a function that maps each c to a subset of A, and [[·]]A,sis a function that maps each group strategy to a set of strategy profiles⊆ An, as follows:

[[a]]A,s,i = {a}[[??]]A,s,i = A

[[!!]]A,s,i = {s[i]}

[[(t1 . . . , tn)]]A,s = [[t1]]A,s,1 × · · · × [[tn]]A,s,n

Examples

• Let A = {a, b, c}, and let n = 3.

• Then a strategic change by the first player to b, while both otherplayers stick to their vote is expressed as (b, !!, !!). In a game state(a, b, b) this is interpreted as {((a, b, b), (b, b, b))}.

• A strategic change by the first player to b, given that the secondplayer sticks to her vote, while the third player may or may notchange, is expressed by (b, !!, ??). In the context of a strategy pro-file s = (a, b, c), this is interpreted as follows:

[[(b, !!, ??)]]A,s = {b} × {b} × {a, b, c}.

• (??, c, c) represents the group strategy where players 2 and 3 bothplay c. This is a strategy for the coalition of 2 and 3 against 1.

• The formula that expresses that the coalition of 1 and 2 can forceoutcome c by both voting c is (abbreviating the singleton outcome{c} as c):

[(??, c, c)]c.

• The strategy (??, ??, c) is different from (!!, !!, c), for the latter ex-presses the individual strategy for player 3 of playing c, in a con-text where the two other players do not change their strategy.

• If we represent a strategy for player i as a relation, then we haveto take into account that the individual choice of i does need in-formation about how the others move to determine the outcome.The relation for the individual choice a of player i is given by

[[(??, · · · , ??, a, ??, · · · , ??)]]A,s = A× · · ·A× {a} × A× · · ·A.

This is computed from all choices that the other players couldmake (all strategies for the other players).

• Compare this with

[[(!!, · · · , !!, a, !!, · · · , !!)]]A,s =

{s[1]} × · · · {s[i− 1]} × {a} × {s[i + 1]} × · · · {s[n]}.This is the action where player i switches to a, while all otherplayers stick to their strategies.

Interpretation of (c, !!) in the PD Game

cc cd

dc dd

c,!! c,!!

c,!! c,!!

Truth Definition (let M = (N,AN , o), and s ∈ AN )

M, s |= > alwaysM, s |= S iff s ∈ [[S]]s

M, s |= p iff s ∈ o−1(p)

M, s |= ¬ϕ iff M, s 6|= ϕ

M, s |= ϕ1 ∧ ϕ2 iff M, s |= ϕ1 and M, s |= ϕ2

M, s |= [γ]ϕ iff for all t with (s, t) ∈ [[γ]]M :

M, t |= ϕ

[[S]]M = {(s, t) | t ∈ [[S]]A,s}[[?ϕ]]M = {(s, s) |M, s |= ϕ}

[[γ1; γ2]]M = [[γ1]]

M ◦ [[γ2]]M

[[γ1 ∪ γ2]]M = [[γ1]]M ∪ [[γ2]]

M

[[γ∗]]M = ([[γ]]M)∗.

Abbreviations

• Let (ia, !!) abbreviate the strategy vector

(!!, · · · , !!, a, !!, · · · , !!),

with a in i-th position, and !! everywhere else.

• Let [(i, !!)]ϕ abbreviate∧a∈A[(ia, !!)]ϕ.

• Then [(i, !!)]ϕ expresses that all strategies to which player i canswitch from the current strategy profile result in a strategy profilewhere ϕ holds (provided that the other players keep their strate-gies fixed).

Abbreviations ctd

• Let (ia, ??) abbreviate the strategy vector

(??, · · · , ??, a, ??, · · · , ??),

with a in i-th position, and ?? everywhere else.

• Let [(i, ??)]ϕ abbreviate∧a∈A[(ia, ??)]ϕ.

• Then [(i, ??)]ϕ expresses that all strategies for i guarantee ϕ, nomatter what the other players do.

• Let (??) abbreviate (??, · · · , ??) (the strategy vector that everywherehas ??).

• Then 〈(??)〉ϕ expresses that in some game state ϕ holds.

Representing Payoffs

• To represent payoffs, assume that basic propositions are payoffvectors u, and that the payoff values are in a finite set U (the setof all utilities that can be assigned in the game).

• Define ui ≥ v as∨w∈U,w≥v u[i] = w

• Then ui ≥ v expresses that player i gets at least v.

• Define ui > v as∨w∈U,w>v u[i] = w.

• Then ui > v expresses that player i gets more than v.

Weak Dominance

Intuitively, and i-strategy a is weakly dominant if a is at least as goodfor i against any moves the other players can make as any alternativeb for a.

In our logic:∧v∈U

∧b∈A−{a}

[(ib, ??)](ui ≥ v → 〈(ia, !!)〉ui ≥ v).

Nash Equilibrium

The following formula expresses that the current strategy profile is aNash equilibrium:∧

i∈N

∨v∈U

(ui ≥ v ∧ [(i, !!)]¬ui > v).

The following formula expresses that the game is Nash:

〈(??)〉∧i∈N

∨v∈U

(ui ≥ v ∧ [(i, !!)]¬ui > v).

Majority Voting

Let Ma be the set of all full strategy vectors where a majority of theplayers play a. Then ∧

x∈A

∧c∈Mx

[c]x

expresses that the game is a voting game with majority rule.

Resoluteness

Assume that the proposition a expresses that a is among the winnersgiven the current profile. A voting rule is resolute if there is alwaysexactly one winner.

Viewing voting according to a voting rule as a game, the followingformula expresses that the game is resolute:

[(??)]∨a∈A

(a ∧∧

b∈A−{a}

¬b).

Strategy-Proofness

A voting rule is strategy proof if it holds for any profile S and forany player (voter) i that changing his vote (action) does not give anoutcome that is better (according to the preferences of i in S) that theoutcome in S. This is expressed by the following formula:

[(??)]∧i∈N

∨v∈U

(ui ≥ v ∧ ¬〈(i, !!)〉ui > v).

Non-Imposedness

A voting rule is (weakly) non-imposed if at least three outcomes arepossible.

Viewing voting as a game, we can use the following formula to ex-press this: ∨

a∈A

∨b∈A−{a}

∨c∈A−{a,b}

(〈(??)〉a ∧ 〈(??)〉b ∧ 〈(??)〉c).

Dictatorship

In a multi-agent game setting, a dictator is a player who can alwaysget what he wants, where getting what you want is getting a payoffthat is at least as good as anything any other player can achieve. Hereis the formula for that, using the abbreviation [(i, ??)]ϕ:

[(i, ??)]∨v∈U

(ui ≥ v ∧∧

j∈N−{i}

¬uj > v).

Gibbard-Satterthwaite in MASL

The classic Gibbard-Satterthwaite theorem [9, 15] states that all rea-sonable voting rules allow strategizing, or put otherwise, that no rea-sonable voting rule is strategy-proof.

Resoluteness, strategy-proofness, non-imposedness and dictatorshipare the four properties in terms of which the Gibbard-Satterthwaitetheorem is formulated.

Statement of the Gibbard-Satterthwaite theorem in our logic:

RES ∧ SP ∧ NI ` DICT.

To do: prove this in MASL.

Meta-Strategies: Tit-for-Tat

Tit-for-Tat as a meta-strategy for the PD game [3] is the instructionto copy one’s opponents last choice, thereby giving immediate, andrancour-free, reward and punishment. Here is a picture of the Tit-for-Tat meta-strategy for player 2, with the states indicating the outcomesof the last play of the game:

cc cd

dc dd

??, c

??, d

??, c

??, c

??, d

??, d

This works because we may think of the current state of the game asthe result of the last play of PD, remembered in the state.

Tit-For-Tat as a MASL Action Expression

The following MASL action expression describes this meta-strategyfor player 2.

(?(c, ??); (??, c) ∪ ?(d, ??); (??, d))∗

What this says is: if the last action by the opponent was a c, thenreward, otherwise (the last action by the opponent was a d) punish.To turn this into a meta-strategy for player 1, just swap all pairs:

(?(??, c); (c, ??) ∪ ?(??, d); (d, ??))∗

Note that tit-for-tat for the PD game boils down to the same thing asthe copycat meta-strategy.

Calculus for MASL

To axiomatize this logic, we can use the well-known proof system forPDL [16, 10], with appropriate axioms for the strategy vectors added.

Call a strategy vector c = (c1 . . . , cn) determinate if for no i ∈ N

ci = ??.

Vector axioms are:

1. Effectivity:[c]c.

2. Seriality:〈c〉>.

3. Functionality:〈c〉ϕ→ [c]ϕ

for all determinate strategy vectors S.

4. Adversary power:

Let c have ?? in position i, and let cia be the result of replacing ??in position i inc by a. Then:

[c]ϕ↔∧a∈A

[cia]ϕ.

Note that this uses the assumption that the set A of available ac-tions is finite.

5. Determinate current choice:

Let c have !! in position i, and let cia be the result of replacing !! atposition i in S by a. Then:

(ia, !!)→ (c↔ cia).

Soundness and Completeness

These axioms are sound for the intended interpretation.

Completeness can be shown by the usual canonical model construc-tion for PDL (see [4]):

Theorem 1 The calculus for MASL is complete.

MASL has the same complexity for model checking and satisfiabilityas PDL:

Model checking for PDL and MASL is PTIME-complete.

Sat solving for PDL and MASL is EXPTIME-complete.

Connections

• Coalition Logic [12]

• Game Logic [11]

• Strategy Logic [5]

• Alternating-time Temporal Logic [2]

• Kn with intersection [1]

• . . .

Further Work

Extend MASL to epistemic MASL.

Language for EMASL:

ϕ ::= > | c | p | ¬ϕ | ϕ1 ∧ ϕ2 | [γ]ϕ | [α]ϕ

γ ::= c | ?ϕ | γ1; γ2 | γ1 ∪ γ2 | γ∗

α ::= i | i |?ϕ | α1;α2 | α1 ∪ α2 | α∗

The interpretations of the i operators (the atoms of α actions) canbe arbitrary. Define Kiϕ as [(i ∪ i )∗]ϕ, and you have a reflexive,symmetric and transitive knowledge operator (see [6]).

References

[1] T. Agotnes and N. Alechina. Reasoning about joint action andcoalitional ability in Kn with intersection. In J. Leite, P. Torroni,T. Agotnes, G. Boella, and L. van der Torre, editors, Computa-tional Logic in Multi-Agent Systems – 12th International Work-shop (CLIMA XII), number 6814 in Lecture Notes in ComputerScience, pages 139–156. Springer, 2011.

[2] R. Alur, T. Henzinger, and O. Kupferman. Alternating-time tem-poral logic. Journal of the ACM, 49:672–713, 2002.

[3] R. Axelrod. The Evolution of Cooperation. Basic Books, NewYork, 1984.

[4] P. Blackburn, M. de Rijke, and Y. Venema. Modal Logic. Cam-bridge Tracts in Theoretical Computer Science. Cambridge Uni-versity Press, 2001.

[5] K. Chatterjee, T. A. Henzinger, and N. Piterman. Strategy logic.Inf. Comput., 208(6):677–693, June 2010.

[6] J. v. Eijck and Y. Wang. Propositional Dynamic Logic as a logicof belief revision. In W. Hodges and R. de Queiros, editors,Proceedings of Wollic’08, number 5110 in Lecture Notes in Ar-tificial Intelligence, pages 136–148. Springer, 2008.

[7] R. Farquharson. Theory of Voting. Blackwell, 1969.

[8] M. Fischer and R. Ladner. Propositional dynamic logic ofregular programs. Journal of Computer and System Sciences,18(2):194–211, 1979.

[9] A. Gibbard. Manipulation of voting schemes: A general result.Econometrica, 41:587–601, 1973.

[10] R. Parikh. The completeness of propositional dynamic logic.

In Mathematical Foundations of Computer Science 1978, pages403–415. Springer, 1978.

[11] R. Parikh. Propositional game logic. In IEEE Symposium onFoundations of Computer Science, pages 195–200, 1983.

[12] M. Pauly. Logic for Social Software. PhD thesis, ILLC, Ams-terdam, 2001.

[13] V. Pratt. Semantical considerations on Floyd–Hoare logic. Pro-ceedings 17th IEEE Symposium on Foundations of ComputerScience, pages 109–121, 1976.

[14] V. Pratt. Application of modal logic to programming. StudiaLogica, 39:257–274, 1980.

[15] M. Satterthwaite. Strategy-proofness and Arrow’s conditions:Existence and correspondence theorems for voting procedures

and social welfare functions. Journal of Economic Theory,10:187–217, 1975.

[16] K. Segerberg. A completeness theorem in the modal logic ofprograms. In T. Traczyck, editor, Universal Algebra and Appli-cations, pages 36–46. Polish Science Publications, 1982.