1. nonzero-sum impulse games 2. veriﬁcation theorem€¦ · 1. nonzero-sum impulse games 2....

1. Nonzero-sum impulse games2. Verification theorem

3. Examples and extension to MFG

Introduction. Consider two countries and the exchange ratebetween their currencies. Unfortunately, the countries disagree:

Country 1 would like the exchange rate to be greater than s1,Country 2 would like the exchange rate to be smaller than s2.

In a static situation (fixed rate), the countries would choose s1+s22 .

Consider now a dynamic situation: the rate varies in time and isstochastic. If the rate gets too low (resp. high), Country 1 (resp. 2)may want to intervene and shift it. When should each countryintervene? What should the new rate be? Generalization to N>2?

Introduction. Consider two countries and the exchange ratebetween their currencies. Unfortunately, the countries disagree:

Country 1 would like the exchange rate to be greater than s1,Country 2 would like the exchange rate to be smaller than s2.

In a static situation (fixed rate), the countries would choose s1+s22 .

Consider now a dynamic situation: the rate varies in time and isstochastic. If the rate gets too low (resp. high), Country 1 (resp. 2)may want to intervene and shift it. When should each countryintervene? What should the new rate be? Generalization to N>2?

More generally, we are here interested in the following problems,which model many practical situations (like the one above).

A stochastic process X : dXt=b(Xt)dt+�(Xt)dWt , with Xt 2Rd .

N players can intervene and shift the process from Xt to Xt + �.

The behaviour of each player is defined by a sequence ofintervention times and shifts: ⇣i = {(⌧i,k , �i,k)}k2N, 1 iN

(impulse control).

Each player aims at maximizing his own payoff, consisting in thesum of: running cost, intervention cost, final cost.

Hence, from a mathematical point of view, we here deal with:N-player nonzero-sum stochastic games with impulse controls.

This is a very recent research area...

More generally, we are here interested in the following problems,which model many practical situations (like the one above).

A stochastic process X : dXt=b(Xt)dt+�(Xt)dWt , with Xt 2Rd .

N players can intervene and shift the process from Xt to Xt + �.

The behaviour of each player is defined by a sequence ofintervention times and shifts: ⇣i = {(⌧i,k , �i,k)}k2N, 1 iN

(impulse control).

Each player aims at maximizing his own payoff, consisting in thesum of: running cost, intervention cost, final cost.

Hence, from a mathematical point of view, we here deal with:N-player nonzero-sum stochastic games with impulse controls.

This is a very recent research area...

Literature, part 1. Quick literature review on stochastic games.

Stopping time Impulse control

One-pl. control problemTwo-pl. zero-sum gameTwo-pl. nonzero-sum game

One-pl. control problem Several authorsTwo-pl. zero-sum gameTwo-pl. nonzero-sum game

Several authors: the player chooses ⌧ so as to maximize

E Z ⌧

0e�⇢t

f (Xt)dt + e�⇢⌧

h(X⌧ )

One-pl. control problem Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game

Friedman: the players choose ⌧1, ⌧2 so as to maximize

E Z ⌧1^⌧2

0e�⇢t

f (Xt)dt + e�⇢(⌧1^⌧2)h(X⌧1^⌧2)

One-pl. control problem Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game Bensoussan-Friedman

Bensoussan-Friedman: the players choose ⌧1, ⌧2 so as to maximize

E Z ⌧1^⌧2

0e�⇢i t fi (Xt)dt + e

�⇢i (⌧1^⌧2)hi (X⌧1^⌧2)

One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game Bensoussan-Friedman

Several authors: the player chooses u = {(⌧k , �k)}k to maximize

0e�⇢t

f (Xt)dt +X

e�⇢⌧k�

�X(⌧k )� , �k

��.

One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game Friedman CossoTwo-pl. nonzero-sum game Bensoussan-Friedman

Cosso: the players choose ui = {(⌧ ik , �

ik)}k so as to maximize

0e�⇢t

f (Xt)dt +X

e�⇢⌧i,k�

�X(⌧i,k )� , �i ,k

e�⇢⌧j,k

�X(⌧j,k )� , �j ,k

��.

One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game Friedman CossoTwo-pl. nonzero-sum game Bensoussan-Friedman ???

Recent problem: the players choose ui = {(⌧ ik , �

ik)}k to maximize

0e�⇢i t fi (Xt)dt +

e�⇢i⌧i,k�i

�X(⌧i,k )� , �i ,k

e�⇢i⌧j,k i

�X(⌧j,k )� , �j ,k

��.

Literature, part 2. Literature on nonzero-sum impulse games.

Formulation and first results for 2-player gamesAïd, Basei, Callegaro, Campi, Vargiolu, Nonzero-sum stochastic

differential games with impulse controls: a verification theorem

with applications, to appear in Mathematics of Operations Research

A numerical method to solve 2-player gamesAïd, Bernal, Mnif, Zabaljauregui, Zubelli, A policy iteration

algorithm for nonzero-sum stochastic impulse games, to appear in

Esaim Proceedings and Surveys

Nonzero impulse games to model a pollution control problemFerrari, Koch, On a strategic model of pollution control, to appear

in Annals of Operations Research

Generalization to N-player games and mean-field gamesBasei, Cao, Guo, Nonzero-sum stoch. games with impulse controls

The process. The underlying process, when none of the playerintervenes, is modelled by dYs = b(Ys)ds + �(Ys)dWs 2 Rd . Thegame ends at ⌧S , the exit time of Y from a fixed subset S ✓ Rd .

Interventions of the players. When player i 2 {1, . . . ,N} decidesto interv. with impulse �, the process is shifted from state x to state�i (x , �), player i pays �i (x , �), player j 6= i pays/receives j ,i (x , �).

Impulse controls. The action of player i is modelled by asequence (impulse control) in the form ui = {(⌧i ,k , �i ,k)}k�1,where {⌧i ,k}k are increasing stopping times (the intervention times)and {�i ,k}k are random variables (the corresponding impulses).

Strategies. The behaviour of the players, modelled by impulsecontrols, is driven by strategies.

A strategy for player i 2 {1, . . . ,N} is a pair 'i = (Ai , ⇠i ), whereAi is a fixed subset of Rd and ⇠i is a continuous function.

Once a N-uple of strategies ' = ('1, . . . ,'N), with 'i = (Ai , ⇠i ),and a starting point x have been chosen, N impulse controls and acontrolled process X = X

x ;' are uniquely defined as follows:

- player i intervenes if and only if the process enters the region Ai ,in which case the impulse is given by ⇠i (y), where y is the state;

- The game ends when the process exits from S .

Strategies. The behaviour of the players, modelled by impulsecontrols, is driven by strategies.

A strategy for player i 2 {1, . . . ,N} is a pair 'i = (Ai , ⇠i ), whereAi is a fixed subset of Rd and ⇠i is a continuous function.

Once a N-uple of strategies ' = ('1, . . . ,'N), with 'i = (Ai , ⇠i ),and a starting point x have been chosen, N impulse controls and acontrolled process X = X

x ;' are uniquely defined as follows:

- player i intervenes if and only if the process enters the region Ai ,in which case the impulse is given by ⇠i (y), where y is the state;

- The game ends when the process exits from S .

Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):

Ji (x ;') := Ex

Z ⌧S

0e�⇢i s fi (Xs)ds +

k2Ne�⇢i⌧i,k�i

⇣X(⌧i,k )� , �i ,k

k2Ne�⇢i⌧j,k i ,j

⇣X(⌧j,k )� , �j ,k

⌘+ e

�⇢i⌧Shi�X(⌧S )�

�{⌧S<+1}

Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

where si ,'i (') substitutes the i-th element in ' by 'i .

Ji (x ;') := Ex

Z ⌧S

⇣X(⌧i,k )� , �i ,k

⇣X(⌧j,k )� , �j ,k

⌘+ e

�{⌧S<+1}

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

Ji (x ;') := Ex

Z ⌧S

⇣X(⌧i,k )� , �i ,k

⇣X(⌧j,k )� , �j ,k

⌘+ e

�{⌧S<+1}

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

Ji (x ;') := Ex

Z ⌧S

⇣X(⌧i,k )� , �i ,k

⇣X(⌧j,k )� , �j ,k

⌘+ e

�{⌧S<+1}

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

Ji (x ;') := Ex

Z ⌧S

⇣X(⌧i,k )� , �i ,k

⇣X(⌧j,k )� , �j ,k

⌘+ e

�{⌧S<+1}

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

Ji (x ;') := Ex

Z ⌧S

⇣X(⌧i,k )� , �i ,k

⇣X(⌧j,k )� , �j ,k

⌘+ e

�{⌧S<+1}

⇤N) s.t.

Vi (x) := Ji (x ;'⇤) � J

i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,

Our goal. Looking for a verification theorem: if N functions areregular enough and satisfy suitable equations, they are actually thevalue funct. of the game and we can characterize a Nash strategy.

First of all, some heuristics about the appropriate equations for Vi

and the Nash equilibria. To simplify, let �i (x , �) = x + �.

Heuristics on Nash equilibria. Assume for a moment we knowthe value functions Vi . Define, for i , j 2{1, . . . ,N}, i 6= j , x 2 S :

{�i (x)} = arg max��Vi�x + �

�+ �i (x , �)

MiVi (x) = Vi�x + �i (x)

�+ �i

�x , �i (x)

Hi ,jVi (x) = Vi�x + �j(x)

�+ i ,j

�x , �j(x)

{�i (x)} = arg max��Vi�x + �

�+ �i (x , �)

�+ �i

�x , �i (x)

�+ i ,j

�x , �j(x)

{�i (x)} = arg max��Vi�x + �

�+ �i (x , �)

�+ �i

�x , �i (x)

�+ i ,j

�x , �j(x)

Let x be the current state of the process. Interpretation:Vi (x) is the value of the game for player i ;�i (x) is the optimal impulse of player i in case of an immediateintervention by player i himself;MiVi (x) (resp. Hi ,jVi (x)) is the value of the game for playeri in case of an immediate interv. by player i (resp. player j).

To help with the interpretation, we here recall the definitions:

Vi (x) = J i (x ;'⇤),

{�i (x)} = arg max�

�x + �

�+ �i (x , �)

MiVi (x) = Vi

�x + �i (x)

�+ �i

�x , �i (x)

Hi,jVi (x) = Vi

�x + �j(x)

�+ i,j

�x , �j(x)

As a consequence, we (heuristically) argue that the Nash policy is:

player i intervenes if and only if MiVi (x) = Vi (x)

and shifts the process from x to x + �i (x).

So, provided that Vi is known, we can build a Nash equilibrium.We now need to characterize Vi , by means of suitable equations.

Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :

Vi = hi , in @S ,

MjVj � Vj 0, in S ,

Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max

�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},

where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).

Vi = hi , in @S ,

First equation. Standard terminal condition.

Vi = hi , in @S ,

Second equation. We expect MjVj � Vj 0 thanks to theinterpretation above. Standard condition in impulse control theory.

Vi = hi , in @S ,

Hi ,jVi � Vi = 0, in {MjVj�Vj =0},

max�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},

Third equation. If player j intervenes (i.e. MjVj � Vj = 0), by thedefinition of Nash equilibrium we expect that player i does not loseanything: this is modelled by Hi ,jVi � Vi = 0.

Vi = hi , in @S ,

Fourth equation. If none of player i ’s competitors intervenes(i.e. MjVj�Vj <0, for each j 6= i), then Vi satisfies the PDE of astandard one-player impulse problem.

Vi = hi , in @S ,

Verification theorem. We are now ready to state and prove theverification theorem for our class of problems.

Verification theoremLet V1, . . . ,VN be functions from S to R satisfying some (veryweak) technical assumptions and such that:

- Vi is a classical solution to (QVI),- Vi 2 C

2(D�i \ @Di ) \ C1(D�i ) \ C (D�i ),

where i , j 2{1, . . . ,N}, Di ={MiVi�Vi <0}, D�i =\j 6=iDj .Then the Vi are the value functions and a Nash equil. is as follows.

Player i intervenes if and only if X enters {MiVi�Vi =0}.

When intervening, player i shifts X from the current state x tothe state x + �i (x), where �i (x) is the (unique) maximizer of� 7! Vi

�x + �

�+ �i (x , �).

Regularity assumptions. When solving the QVI problem, onedeals with functions which are piecewise defined (this will be clearlater). The regularity conditions are set as suitable smooth-pastingconditions: what we get is a system of algebraic equations.

If the regularity conditions are too strong, the system has moreequations than parameters, with no possibility to apply the theorem.

Then, it is fundamental to provide regularity conditions which allowto prove the verification theorem but which also make possible topractically apply this theorem.

Example: a game with explicit solutions

The problem. Let us consider the following one-dimensional game(in the paper, a more general example):

J1(x ;'1,'2)=Ex

0e�⇢s(Xs�s1)ds�

e�⇢⌧1,k c+

e�⇢⌧2,k c

J2(x ;'1,'2)=Ex

0e�⇢s(s2�Xs)ds�

e�⇢⌧2,k c+

e�⇢⌧1,k c

where s1<s2 and, in case of no interventions, we have dXs=�dWs .

Practically, two players control a one-dimensional BM. They havesymmetric linear payoffs: player 1 (resp. player 2) needs a high(resp. low) value for the state. Fixed intervention cost/penalty.Possible interpretation: control of the exchange rate, as above.

The problem. Let us consider the following one-dimensional game(in the paper, a more general example):

J1(x ;'1,'2)=Ex

0e�⇢s(Xs�s1)ds�

e�⇢⌧1,k c+

e�⇢⌧2,k c

J2(x ;'1,'2)=Ex

0e�⇢s(s2�Xs)ds�

e�⇢⌧2,k c+

e�⇢⌧1,k c

where s1<s2 and, in case of no interventions, we have dXs=�dWs .

Practically, two players control a one-dimensional BM. They havesymmetric linear payoffs: player 1 (resp. player 2) needs a high(resp. low) value for the state. Fixed intervention cost/penalty.Possible interpretation: control of the exchange rate, as above.

The solving procedure is as follows.Step 1: we solve the QVI problem, to get a pair of parametriccandidates V1,V2 for the value functions.Step 2: we impose the regularity conditions, to get the value of theparameters in the candidates V1,V2 (solution to algebraic system).Step 3: we apply the verification theorem, to get a Nash equilibriumby means of V1,V2, after checking that all the assumptions hold.

Step 1: building a candidate. As player 1 needs a high value forXt , we assume his interv. region to be in the form ]�1,x1].Similarly, we expect the interv. region of player 2 to be in the form[x2,+1[. The real line is, heuristically, divided into three intervals:

]�1, x1] = {M1V1 � V1 = 0}, where player 1 intervenes,]x1, x2[ = {M1V1�V1<0, M2V2�V2<0}, where no one intervenes,

[x2,+1[ = {M2V2 � V2 = 0}, where player 2 intervenes.

The solving procedure is as follows.Step 1: we solve the QVI problem, to get a pair of parametriccandidates V1,V2 for the value functions.Step 2: we impose the regularity conditions, to get the value of theparameters in the candidates V1,V2 (solution to algebraic system).Step 3: we apply the verification theorem, to get a Nash equilibriumby means of V1,V2, after checking that all the assumptions hold.

Step 1: building a candidate. As player 1 needs a high value forXt , we assume his interv. region to be in the form ]�1,x1].Similarly, we expect the interv. region of player 2 to be in the form[x2,+1[. The real line is, heuristically, divided into three intervals:

]�1, x1] = {M1V1 � V1 = 0}, where player 1 intervenes,]x1, x2[ = {M1V1�V1<0, M2V2�V2<0}, where no one intervenes,

[x2,+1[ = {M2V2 � V2 = 0}, where player 2 intervenes.

The equations in the QVI problem here read

HiVi � Vi = 0, in {MjVj � Vj = 0},max

�AVi � ⇢Vi + fi ,MiVi � Vi} = 0, in {MjVj � Vj < 0},

which we can rewrite as (where 'i is a sol. to AVi � ⇢Vi + fi = 0):

MiVi , in {MiVi � Vi = 0},'i , in {MiVi � Vi < 0,MjVj � Vj < 0},HiVi , in {MjVj � Vj = 0}.

Finally, by the previous partition of the real line, we get

M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,

M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].

M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,

M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].

M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,

M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].

By heuristic arguments we can estimate MiVi and HiVi . Thisleads to the following (class of) candidates, where x

⇤i is a local

maximum of 'i in the interval ]x1, x2[:

eV1(x) =

'1(x⇤1 ;A11,A12)� c , if x 2 ]�1, x1],'1(x ;A11,A12), if x 2 ]x1, x2[,'1(x⇤2 ;A11,A12) + c , if x 2 [x2,+1[,

eV2(x) =

'2(x⇤1 ;A21,A22) + c , if x 2 ]�1, x1],'2(x ;A21,A22), if x 2 ]x1, x2[,'2(x⇤2 ;A21,A22)� c , if x 2 [x2,+1[.

Notice that some free parameters are present at the moment. Wenow set such parameters by imposing the regularity conditions.

By heuristic arguments we can estimate MiVi and HiVi . Thisleads to the following (class of) candidates, where x

⇤i is a local

maximum of 'i in the interval ]x1, x2[:

eV1(x) =

'1(x⇤1 ;A11,A12)� c , if x 2 ]�1, x1],'1(x ;A11,A12), if x 2 ]x1, x2[,'1(x⇤2 ;A11,A12) + c , if x 2 [x2,+1[,

eV2(x) =

'2(x⇤1 ;A21,A22) + c , if x 2 ]�1, x1],'2(x ;A21,A22), if x 2 ]x1, x2[,'2(x⇤2 ;A21,A22)� c , if x 2 [x2,+1[.

Notice that some free parameters are present at the moment. Wenow set such parameters by imposing the regularity conditions.

Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:

⇤1 ;A11,A12) = 0, '00

1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )

'01(x1;A11,A12) = 0, (C 1-pasting in x1)

'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)

⇤2 ;A21,A22) = 0, '00

'02(x2;A21,A22) = 0, (C 1-pasting in x2)

'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)

(x1 < x

⇤1 < x2, (order condition 1)

x1 < x⇤2 < x2, (order condition 2)

where 'i (x ; a, a)=ae✓x+ae

�✓x+(�1)i+1(x � si )/⇢ and ✓=p

2⇢/�.

⇤1 ;A11,A12) = 0, '00

'01(x1;A11,A12) = 0, (C 1-pasting in x1)

⇤2 ;A21,A22) = 0, '00

'02(x2;A21,A22) = 0, (C 1-pasting in x2)

(x1 < x

�✓x+(�1)i+1(x � si )/⇢ and ✓=p

2⇢/�.

⇤1 ;A11,A12) = 0, '00

'01(x1;A11,A12) = 0, (C 1-pasting in x1)

⇤2 ;A21,A22) = 0, '00

'02(x2;A21,A22) = 0, (C 1-pasting in x2)

(x1 < x

�✓x+(�1)i+1(x � si )/⇢ and ✓=p

2⇢/�.Notice: 8 equations

⇤1 ;A11,A12) = 0, '00

'01(x1;A11,A12) = 0, (C 1-pasting in x1)

⇤2 ;A21,A22) = 0, '00

'02(x2;A21,A22) = 0, (C 1-pasting in x2)

(x1 < x

�✓x+(�1)i+1(x � si )/⇢ and ✓=p

2⇢/�.Notice: 8 equations and 8 parameters.

⇤1 ;A11,A12) = 0, '00

'01(x1;A11,A12) = 0, (C 1-pasting in x1)

⇤2 ;A21,A22) = 0, '00

'02(x2;A21,A22) = 0, (C 1-pasting in x2)

(x1 < x

�✓x+(�1)i+1(x � si )/⇢ and ✓=p

2⇢/�.Notice: 8 equations and 8 parameters. We here have explicit solutions!

Step 3: equilibrium. After checking of the assumptions, we finallyapply the theorem and use V1, V2 to get Nash equilibria.

PropositionA family of Nash equilibria is characterized as follows (s 2 R).

Player i intervenes if and only if the process hits xi , where

x2, x1 = s ±1

✓log

"s⌘ + ⇠⌘ � ⇠

s✓(c � c)

4⇠+ 1 +

s✓(c � c)

When intervening, player i shifts the process to the state x⇤i , where

x⇤2 , x

⇤1 = s ±

✓log

"s1 � ⇠1 + ⇠

s✓(c � c)

4⇠+ 1 +

s✓(c � c)

Here, ✓=p

2⇢� and ⇠ is the unique zero of F (x)=2x+✓c�log

⇣1+x1�x

Explicit expressions for both the value functions are available as well.

Step 3: equilibrium. After checking of the assumptions, we finallyapply the theorem and use V1, V2 to get Nash equilibria.

PropositionA family of Nash equilibria is characterized as follows (s 2 R).

Player i intervenes if and only if the process hits xi , where

x2, x1 = s ±1

✓log

"s⌘ + ⇠⌘ � ⇠

s✓(c � c)

4⇠+ 1 +

s✓(c � c)

When intervening, player i shifts the process to the state x⇤i , where

x⇤2 , x

⇤1 = s ±

✓log

"s1 � ⇠1 + ⇠

s✓(c � c)

4⇠+ 1 +

s✓(c � c)

Here, ✓=p

2⇢� and ⇠ is the unique zero of F (x)=2x+✓c�log

⇣1+x1�x

Explicit expressions for both the value functions are available as well.

Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0.

Then, the solution is:

x1 = 0.814, x2 = 1.186, (Intervention points)

x⇤1 = 1.078, x

⇤2 = 0.922. (States after intervention)

0.814 0.922 1.078 1.186

This is what happens practically (assume X0 = 1).As long as the state belongs to ]x1, x2[, no one intervenes.When the state reaches x1 or x2, one of the player intervenes:

- if Xt = x1, player 1 intervenes and moves the state to x⇤1 ,

- if Xt = x2, player 2 intervenes and moves the state to x⇤2 .

...and so on.

Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0. Then, the solution is:

x⇤1 = 1.078, x

0.814 0.922 1.078 1.186

...and so on.

Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0. Then, the solution is:

x⇤1 = 1.078, x

0.814 0.922 1.078 1.186

...and so on.

Here the value function of player 2:

notice the three parts, thesmooth pasting in x1, the maximum point in x

⇤2 .

0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0

x1 x⇤2 x⇤

Here the value function of player 2: notice the three parts,

thesmooth pasting in x1, the maximum point in x

⇤2 .

0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0

x1 x⇤2 x⇤

Here the value function of player 2: notice the three parts, thesmooth pasting in x1,

the maximum point in x⇤2 .

0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0

x1 x⇤2 x⇤

Here the value function of player 2: notice the three parts, thesmooth pasting in x1, the maximum point in x

⇤2 .

0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0

x1 x⇤2 x⇤

Here the value function of player 2: notice the three parts, thesmooth pasting in x1, the maximum point in x

⇤2 . And of player 1.

0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0

x1 x⇤2 x⇤

Properties. The explicit formulas for xi , x⇤i let us state someproperties w.r.t. the parameters c (interv. cost) and c (interv. gain).

As c ! 0+ (interv. cost to zero, assume c = 0), thecontinuation region shrinks and finally collapses to {s}.Reasonable: small costs imply frequent interventions, in the limitsituation the controlled process gets constant.

As c ! +1 (interv. cost to 1), the continuation regionexpands and finally coincides with R.Reasonable: big costs imply rare interventions, in the limit situationthe players never intervene.

If c= c>0, we have x⇤i = xj : infinite sequence of simultaneous

interventions, the Nash strategies are not admissible.

When N ! 1: mean-field games with impulse controls

Idea. Let us consider impulse games where the players’ payoffsdepend on the average of the processes X

1, . . . ,XN :

Ji (x ;') = Ex

0e�⇢s

✓Xs ,

◆ds +

e�⇢⌧k�i (�i,k)

dXs = b(Xs)ds + �(Xs)dWs + impulses.

For example, each player may want to keep her own process closeto the average 1

PNj=1 X

Very complicated to solve...

Mean-field games. N-player games are complicated to solve. Idea:N = 1 (! stationary mean-field games with impulse controls).

J(x ;') = Ex

0e�⇢s

f (Xs ,m)ds +X

e�⇢⌧k�(�k)

dXs = b(Xs)ds + �(Xs)dWs + impulses,

m = lims!1

Ex [Xs ].

Step 1. For a fixed m 2 R, find the opt. strategy '⇤ = '⇤(m).

Step 2. Let �(m) := lims!1 Ex [Xx,'⇤(m)s ].

Step 3. Solve the fixed point problem �(m) = m.

Mean-field games. N-player games are complicated to solve. Idea:N = 1 (! stationary mean-field games with impulse controls).

J(x ;') = Ex

0e�⇢s

f (Xs ,m)ds +X

e�⇢⌧k�(�k)

dXs = b(Xs)ds + �(Xs)dWs + impulses,

m = lims!1

Ex [Xs ].

Step 1. For a fixed m 2 R, find the opt. strategy '⇤ = '⇤(m).

Step 2. Let �(m) := lims!1 Ex [Xx,'⇤(m)s ].

Step 3. Solve the fixed point problem �(m) = m.

A class of problems. For suitable ↵ (e.g., a contraction), consider:

J(x ;') = Ex

0e�⇢s

|Xs � ↵(m)|ds +X

e�⇢⌧k (K + k |�k |)

dXs = �dWs + impulses, m = lims!1

Ex [Xs ].

Following the procedure above (Steps 1-2-3), we get the followingoptimal control.

Let m be the (unique) fixed point of a suitable contraction. Letu,U be determined by a suitable system of algebraic equations.

The controller intervenes when |Xs � ↵(m)| � u.

If Xs � ↵(m) � u (resp. �u), she moves the process so thatXs � ↵(m) = U (resp. = �U).

A class of problems. For suitable ↵ (e.g., a contraction), consider:

J(x ;') = Ex

0e�⇢s

|Xs � ↵(m)|ds +X

e�⇢⌧k (K + k |�k |)

dXs = �dWs + impulses, m = lims!1

Ex [Xs ].

Following the procedure above (Steps 1-2-3), we get the followingoptimal control.

Let m be the (unique) fixed point of a suitable contraction. Letu,U be determined by a suitable system of algebraic equations.

The controller intervenes when |Xs � ↵(m)| � u.

If Xs � ↵(m) � u (resp. �u), she moves the process so thatXs � ↵(m) = U (resp. = �U).

N vs. MFG. Let us compare the N-player and 1-player problems.

(N) V i (x) = J i (x ;'⇤) J i (x ; si,'i ('⇤)), 8i , 'i (Nash eq.)

'⇤ = ('⇤1 , . . . ,'

⇤N), '⇤

i = (A⇤i , ⇠

⇤i ) (action region/function)

J i (x ;') = Ex

0e�⇢s

��Xis �

��ds +X

e�⇢⌧i,kK

(MFG) VMFG (x) = JMFG (x ;'⇤) = inf'

JMFG (x ;') (optimal ctrl.)

'⇤ = (A⇤, ⇠⇤) (action region/function)

JMFG (x ;')=Ex

0e�⇢s

|Xs �m|ds +X

e�⇢⌧kK

�, m= lim

sEx [Xs ]

Question: does the MFG solution ’provide’ a good approximation ofthe N-player solution?

N vs. MFG. Let us compare the N-player and 1-player problems.

(N) V i (x) = J i (x ;'⇤) J i (x ; si,'i ('⇤)), 8i , 'i (Nash eq.)

'⇤ = ('⇤1 , . . . ,'

⇤N), '⇤

i = (A⇤i , ⇠

⇤i ) (action region/function)

J i (x ;') = Ex

0e�⇢s

��Xis �

��ds +X

e�⇢⌧i,kK

(MFG) VMFG (x) = JMFG (x ;'⇤) = inf'

JMFG (x ;') (optimal ctrl.)

'⇤ = (A⇤, ⇠⇤) (action region/function)

JMFG (x ;')=Ex

0e�⇢s

|Xs �m|ds +X

e�⇢⌧kK

�, m= lim

sEx [Xs ]

Question: does the MFG solution ’provide’ a good approximation ofthe N-player solution?

The solution to the MFG is characterized by (�u⇤,�U

⇤,U⇤, u⇤)(±U

⇤ intervention threshold, ±u⇤ new state after intervention).

Consider now the following strategy '⇤ for the N-player game.

If X is �m

⇤� u

⇤ (resp. �u⇤), player i moves the process so

that X is �m

⇤ = U⇤ (resp. = �U

⇤).Suitable priority rules in case of simultaneous interventions.

The following result holds.

PropositionThe strategies '⇤, described as above, form a "-Nash equilibriumfor the N-player game, where " = O( 1p

Notice: "-Nash means that J i (x ;'⇤) Ji (x ; si ,'i ('

⇤)) + ".

The solution to the MFG is characterized by (�u⇤,�U

⇤,U⇤, u⇤)(±U

⇤ intervention threshold, ±u⇤ new state after intervention).

Consider now the following strategy '⇤ for the N-player game.

If X is �m

⇤� u

⇤ (resp. �u⇤), player i moves the process so

that X is �m

⇤ = U⇤ (resp. = �U

⇤).Suitable priority rules in case of simultaneous interventions.

The following result holds.

PropositionThe strategies '⇤, described as above, form a "-Nash equilibriumfor the N-player game, where " = O( 1p

Notice: "-Nash means that J i (x ;'⇤) Ji (x ; si ,'i ('

⇤)) + ".

1. Nonzero-sum impulse games

1.1 Naturally arise in energy finance but never studied

1.2 Our model: strategies ! impulse controls ! controlled process

1.3 Payoff: running cost, intervention costs, intervention gains, terminal cost

2. Verification theorem

2.1 Sufficient conditions to characterize the value functions

2.2 Fundamental assumptions: QVI problem + regularity conditions

2.3 Key-points for the QVI problem: operators MiVi and Hi,jVi

3.1 Example: two players with linear payoff control a one-dimensional BM

3.2 Explicit formulas and asymptotic properties

3.3 Extension to MFG

A. Bensoussan, A. Friedman,Non-zero sum stochastic differential games with stopping timesand free boundary problems,Trans. Amer. Math. Society 231 (1977), no. 2, 275–327.

A. Cosso,Stochastic differential games involving impulse controls anddouble-obstacle quasi-variational inequalities,SIAM J. Control Optim. 51 (2013), no. 3, 2102–2131.

A. Friedman,Stochastic games and variational inequalities,Arch. Rational Mech. Anal. 51 (1973), no. 5, 321–346.

1. nonzero-sum impulse games 2. veriﬁcation theorem€¦ · 1. nonzero-sum impulse games 2....

Documents