large population consensus in an adversarial...
TRANSCRIPT
![Page 1: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/1.jpg)
Mean field gamesConnections to consensus
Large Population Consensus in an AdversarialEnvironment
Quanyan Zhu2 Dario Bauso1 Tamer Basar2
1Universita di Palermo
2University of Illinois Urbana-Champaign
University of Padua, 14 July 2011.
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 2: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/2.jpg)
Mean field gamesConnections to consensus
Outline
Mean field gamesIntroductionExamples, (O. Gueant et al. 2011)
Connections to consensusa game theoretic perspectivefrom state-feedback NE to mean-field NE
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 3: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/3.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Advection
I N →∞ homogeneous agents withdynamics
x(t) = u(x(t))
I x(t) defines vector field
I density m(x, t) in x evolvesaccording to advection equation
∂tm+ div(m · u(x)) = 0
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 4: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/4.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Mean field games
I Agents wish to minimize J =∫ T0
[ 12|u(t)|2︸ ︷︷ ︸
penalty on control
+ g(x(t),m(·, t))︸ ︷︷ ︸...on state & distribution
]dt+G(x(T ),m(·, T ))︸ ︷︷ ︸
...on final state
I optimal control u(t) = −∇xJI coupled partial differential equations
−∂tJ + 12 |∇xJ |2 = g(x,m)
u
��
(HJB) - backward
∂tm+ div(m · u(x)) = 0
m
TT
(advection) - forward
I boundary conditionsm(·, 0) = m0, J(x, T ) = G(x,m(·, T ))
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 5: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/5.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Hamilton Jacobi Bellman
I From Bellman
J(x0, t0)︸ ︷︷ ︸today’s cost
= minu [12|u|2 + g(x,m)]dt︸ ︷︷ ︸
stage cost
+ J(x0 + dx, t0 + dt)︸ ︷︷ ︸future cost
I Taylor expanding future cost
J(x0 + dx, t0 + dt) = J(x0, t0) + ∂tJdt+∇xJxdt
I minu [12|u|2 + g(x,m) + ∂tJ +∇xJ
u︷︸︸︷x ]︸ ︷︷ ︸
Hamiltonian
= 0
I optimal control u(t) = −∇xJ yields
−∂tJ +12|∇xJ |2 = g(x,m) HJB
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 6: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/6.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Stochastic differential game
I stochastic dynamics is
dx = udt+ σdBt
I dBt infinitesimal Brownian motion
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian)
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J = g(x,m)
u
��
(HJB)-backward
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)-forward
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 7: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/7.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Hamilton Jacobi Bellman (with dx = udt+ σdBt)
I From Bellman
J(x0, t0)︸ ︷︷ ︸today’s cost
= minu [12|u|2 + g(x,m)]dt︸ ︷︷ ︸
stage cost
+ EJ(x0 + dx, t0 + dt)︸ ︷︷ ︸exp. future cost
I Taylor expanding future cost (EdBt = 0, EdB2t → dt)
J(x0+dx, t0+dt) = J(x0, t0)+∂tJdt+E∇xJdx︸ ︷︷ ︸∇xJudt
+ E12dx′∇2
xJdx︸ ︷︷ ︸σ2
2∆JEdB2
t
I minu [12|u|2 + g(x,m) + ∂tJ +∇xJu+
σ2
2∆J ]︸ ︷︷ ︸
Hamiltonian
= 0
I optimal control u(t) = −∇xJ yields
−∂tJ +12|∇xJ |2 −
σ2
2∆J = g(x,m) HJB
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 8: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/8.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Average cost
I J = E lim supT→∞1T
∫ T0
[12 |u(t)|2 + g(x(t),m(·, t))
]dt
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian)
λ+ 12 |∇xJ |2 − σ2
2 ∆J = g(x, m)
u
��
(HJB)
div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 9: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/9.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Discounted cost
I J = E∫∞
0 e−ρt[
12 |u(t)|2 + g(x(t),m(·, t))
]dt
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian)
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J + ρJ = g(x,m)
u
��
(HJB)
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 10: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/10.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Mexican wave (mimicry & fashion)
1z
yL
m(y, t)
0
I state x = [y, z], y ∈ [0, L) coordinate, z position:
z ={
1 standing0 seated
, z ∈ (0, 1) intermediate
I dynamics dz = udt (u control)I penalty on state and distribution g(x,m) =
Kzα(1− z)β︸ ︷︷ ︸comfort
+1ε2
∫(z − z)2m(y; t, z)
1εs(y − yε
)dzdy︸ ︷︷ ︸mimicry
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 11: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/11.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Meeting starting time (coordination with externality)
−xmaxx0
m(x, t)
I dynamics dxi = uidt+ σdBt
I τi = mins(xi(s) = 0) arrival time, ts scheduled time,t actual starting time
I penalty on final state and distributionG(x(τi),m(·, τi)) = c1[τi − ts]+︸ ︷︷ ︸
reputation
+ c2[τi − t]+︸ ︷︷ ︸inconvenience
+ c3[t− τ ]+︸ ︷︷ ︸waiting
I people arrived up to time s: F (s) = −∫ s
0 ∂xm(0, v)dvI starting time t = F−1(θ), (θ is quorum)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 12: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/12.jpg)
Mean field gamesConnections to consensus
IntroductionExamples, (O. Gueant et al. 2011)
Large population (herd behaviour)
I behaviour dynamicsdxi = uidt+ σdBt
I penalty
g(x,m) = β(x−∫ym(y, t)dy︸ ︷︷ ︸average
)2
I discounted cost J = E∫∞
0 e−ρt[
12 |u(t)|2 + g(x(t),m(·, t))
]dt
I mean field game with discounted cost
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J + ρJ = g(x,m)
u
��
(HJB)
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 13: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/13.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Consensus
xi(t) = u(xi(t), x(i)(t))
x1(t) = u(x1(t), x(1)(t))
x2(t) = u(x2(t), x(2)(t))
I N “dynamic agents” (vehicles, employes, computers,...)I ...described by differential (difference) equationsI model interaction through communication graphI vector x(i) represents neighbors’ statesI main feature: one agent is influenced only by neighbors:
xi(t) = u(xi(t), x(i)(t))
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 14: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/14.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
why “consensus”?
xi(t) = u(xi(t), x(i)(t))
x1(t) = u(x1(t), x(1)(t))
x2(t) = u(x2(t), x(2)(t))
I use local control u(xi, x(i))I .. to converge to global “consensus value”
x(t)→ ave(x(0))
I consensus originates from computer scienceI shown connections to potential games
[Lynch, Morgan Kaufmann, 1996], [Shamma et al., Trans. on Systems, Man, and Cyb., to appear]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 15: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/15.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Where is game theory here?
xi(t) = u(xi(t), x(i)(t))
x1(t) = u(x1(t), x(1)(t))
x2(t) = u(x2(t), x(2)(t))
J2(x2, x(2))
J1(x1, x(1))
Ji(xi, x(i))
I Assign N local objective functions Ji(xi, x(i)) so that..I if local control u(xi, x(i)) is optimal w.r.t. Ji(xi, x(i))I all states converge to global “consensus value”
x(t)→ ave(x(0))
[Lynch, Morgan Kaufmann, 1996], [Shamma et al., Trans. on Systems, Man, and Cyb., to appear]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 16: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/16.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Linear quadratic consensus problem
I linear dynamics xi = aixi + biui
I objective functionsJi(xi, x(i), ui) = 1
2
∫∞0 e−ρt
[|u(t)|2 + (xi − x(i))2
]dt
I averaging over neighbors x(i) =∑
j∈Ni wijxj
I in compact form
x = Ax+∑N
i=1Biui
Ji(xi, x(i), ui) = 12
∫∞0 e−ρt
[Riu
2i + x′Qx
]dt
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 17: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/17.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Feedback Nash Equilibrium
I Nash equilibrium strategies are linear in state:
ui = µi(x, t) = −R−1i B′iZix
I Zi are solutions to the coupled Riccati equations
Zi
(A−
∑i=N
SiZi
)+(A−
∑i=N
SiZi
)′Zi +ZiSiZi +Qi = ρiZi
I Si = B′iR−1i Ri
I drawback:1. can be convoluted when N is large,2. strategies use full state vector
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 18: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/18.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Linear (averaging): Arithmetic Mean
I local linear control: u(xi, x(i)) =∑
j∈Ni(xj − xi).
I local quadratic objective: F (xi, x(i)) =(∑
j∈Ni(xj − xi))2
;
Ji(xi, x(i), ui) = limT−→∞∫ T
0
(F (xi, x(i)) + ρu2
i
)dt
[Olfati-Saber, Fax, Murray, Proc. of the IEEE, 2007], [Ren, Beard, Atkins, ACC 2005]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 19: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/19.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Nonlinear 1/2 (Geometric Mean)
I local nonlinear control: u(xi, x(i)) = xi∑
j∈Ni(xj − xi).
I local objective: F (xi, x(i)) =(xi∑
j∈Ni(xj − xi))2
;
Ji(xi, x(i), ui) = limT−→∞∫ T
0
(F (xi, x(i)) + ρu2
i
)dt
[Bauso, Giarre, Pesenti, Systems and Control Letters, 2006], [Cortes, Automatica, 2008]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 20: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/20.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Nonlinear 2/2 (Harmonic Mean)
I local nonlinear control: u(xi, x(i)) = −x2i
∑j∈Ni(xj − xi)
I local objective: F (xi, x(i)) =(x2i
∑j∈Ni(xj − xi)
)2;
Ji(xi, x(i), ui) = limT−→∞∫ T
0
(F (xi, x(i)) + ρu2
i
)dt
[Bauso, Giarre, Pesenti, Systems and Control Letters, 2006], [Cortes, Automatica, 2008]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 21: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/21.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Disturbances 1/2 (Stochastic)
I Least-mean square consensus[Xiao, Boyd, Kim, J. Parallel and Distributed Computing, 2007]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 22: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/22.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Time-varying topology 1/3
I discrete-time gossip algorithmsI at t = 1
I Pick (randomly) one edge (i, j) and an increasing oddfunction f(.)
I xi(t+ 1) = xi(t) + f(xi(t)− xj(t))I xj(t+ 1) = xj(t) + f(xj(t)− xi(t))
[Boyd et al., IEEE trans. on Information Theory, 2006], [Giua et al., TAC, in press]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 23: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/23.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Time-varying topology 2/3
I at t = 2I Pick a second edge (k, l)I xk(t+ 1) = xk(t) + f(xk(t)− xl(t))I xl(t+ 1) = xl(t) + f(xl(t)− xk(t))
and so on for t = 3, 4, . . .
[Boyd et al., IEEE trans. on Information Theory, 2006], [Giua et al., TAC, in press]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 24: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/24.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Time-varying topology 3/3
I slow convergence
[Boyd et al., IEEE trans. on Information Theory, 2006], [Giua et al., TAC, in press]
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 25: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/25.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
From state-feedback NE to Mean field-feedback NE
I state feedback NE strategies
u1 = µ1(x, t) = µ1(x1, x2, . . . , x7, t)
⇓
I individual state feedback NEstrategies u1 = µ1(x1,m, t)
I m is aggregate and exogenousrepresentation of x2, . . . , x7
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 26: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/26.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
homogeneous agents and complete graphs
I density m(x, t) = limN→∞1N
∑Ni=1 Ixi
I average m(t) =∫xm(x, t)dx
I track average signal m(t) (exogenous)
Ji(xi, m, ui) =12
∫ ∞0
e−ρt[|u|2 + (xi − m)2
]dt
I initial states distribution m(x, 0) = m0 (xi ∈ R)I distribution evolution
∂tm+ ∂x((ax+ bu)m(x, t)) = 0 (advection)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 27: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/27.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Feedback mean-field equilibrium
I Solve tracking problem via Riccati method and plug theoptimal u in advection (below)
I solve advection and plug resulting m in offset condition(above)
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 28: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/28.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Extensions
I Consensus with maliciousagents
I applications:communication/socialnetworks
I malicious agent s:
Js =12
∫ ∞0
e−ρt[|u|2+(1− αs)(xs − x(s))2︸ ︷︷ ︸
track neighbors
+αs(xs − xs)2︸ ︷︷ ︸track xs
]dt
I stochastic dynamics
dxi = (aixi + biui)dt+ σidBit
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 29: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/29.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Conclusions
I Mean field games require solving coupled partialdifferential equations (HJB-Kolmogorov)
I consensus translates into a mean field game when infinitehomogenous players (large population)
I analyze consensus with malicious agentsI inspect connections with opinion dynamics with stubborn
agents in social networksI mean field stochastic games
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games
![Page 30: Large Population Consensus in an Adversarial Environmentcontrol.dei.unipd.it/tl_files/events/20110716BausoSeminario.pdf · Quanyan Zhu2 Dario Bauso1 Tamer Basar2 1Universit a di Palermo](https://reader036.vdocuments.us/reader036/viewer/2022071409/6102dbe5a28b05149e6fdfda/html5/thumbnails/30.jpg)
Mean field gamesConnections to consensus
a game theoretic perspectivefrom state-feedback NE to mean-field NE
Main references
I J.-M. Lasry and P.-L. Lions. Mean field games. JapaneseJournal of Mathematics, 2(1), Mar. 2007.
I O. Gueant, J.-M. Lasry and P.-L. Lions. Mean field gamesand applications. Lect. notes in math., Springer 2011.
I M. Huang, P.E. Caines, and R.P. Malhame. IEEE TAC,52(9), 2007.
I Bauso, Giarre, Pesenti, Systems and Control Letters, 2006.I H. Tembine, Q. Zhu, and T. Basar, Risk-sensitive mean
field stochastic games, IFAC WC 2011, Milano, Italy, 2011.
Quanyan Zhu, Dario Bauso, Tamer Basar Robust Dynamic TU games