![Page 1: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/1.jpg)
An Architectural View of Game Theoretic Control
Raga Gopalakrishnan and Adam WiermanCalifornia Institute of Technology
Jason R. MardenUniversity of Colorado at Boulder
6/18/2010 Hotmetrics 2010
![Page 2: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/2.jpg)
Distributed Resource AllocationSensor Coverage Wireless Access Point Selection
Wireless Channel Selection Power Control (sensor networks)
![Page 3: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/3.jpg)
Resource Allocation Problem – A Simple Model
• Set of (distributed) agents, N = {1, 2, . . ., n}• Set of resources, R• Action sets, Ai µ 2R for agents i 2 N– Set of action profiles, A = A1 £ A2 £ . . . £ An
– Set of agents choosing resource r in action profile a, {a}r
• Objective function, W : A! R– Linearly separable, i.e., W(a) = r2R Wr ( {a}r )
Goal: Find an allocation a 2 A that maximizes W(a)
![Page 4: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/4.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Game-theoretic Control
![Page 5: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/5.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Game-theoretic Control
Promising new approach Model the agents as “self-interested”
players in a non-cooperative game
Still being explored The solution to the problem emerges
as the equilibrium of the game
![Page 6: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/6.jpg)
Modeling the problem as a game
• Set of players, N = {1, 2, . . ., n}
• Action sets, Ai µ 2R for players i 2 N– Set of action profiles, A = A1 £ A2 £ £ An
– Set of players choosing resource r in action profile a, {a}r
• Utility functions, Ui : A! R for players i 2 N– Linearly separable, i.e., Ui(a) = r2R fr ( i,
{a}r )
• Welfare function W : A! R– Linearly separable, i.e., W(a) = r2R Wr
( {a}r )
Resource Allocation Problem Resource Allocation Game
• Set of agents, N = {1, 2, . . ., n}• Set of resources, R• Action sets, Ai µ 2R for agents i 2 N
– Set of action profiles, A = A1 £ A2 £ . . . £ An
– Set of agents choosing resource r in action profile a, {a}r
• Objective function, W : A! R– Linearly separable, i.e., W(a) = r2R Wr
( {a}r )
![Page 7: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/7.jpg)
Game Theoretic Control (GTC)
Setup the game1
Design the players2
decision makers/players action sets
utility functions
agent decision rules(learning rules)
Desirable globalbehavior emergesas equilibrium ofthe game
Goal:
• A Nash equilibrium is an action profile a*2 A such that for each player i,
• Measures of efficiency for Nash equilibrium:
![Page 8: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/8.jpg)
Game Theoretic Control (GTC)
Setup the game1
Design the players2
decision makers/players action sets
utility functions
agent decision rules(learning rules)
Desirable properties Existence of an equil. Efficiency of an equil. Tractability Locality of information Budget balance …
Desirable properties Locality of information Fast convergence Equilibrium selection Robust convergence …
Learning Design
Utility Design
Inherited
DesignedDesigned
![Page 9: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/9.jpg)
Many other applications: [Akella et al. 2002, Kaumann et al. 2007, Marden et al. 2007, 2008, Mhatre et al. 2007, Komali and MacKenzie 2007, Zou and
Chakrabarty 2004, Campos-Nanez 2008, Marden & Effros 2009]
[Marden, Wierman 2008]
[Campos-Nanez, Garcia, Li 2008]
Applications of GTC
Utility Design
Learning Design
Sensor Coverage
Power Control (sensor networks)Is there a way to view Game Theoretic Control from an application-independent
perspective?
![Page 10: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/10.jpg)
Architectural View for GTC
Utility Design
Learning Design
Class of Games
“Virtualization”layer
IP
NetworkApps
Networkhardware
OS
software
hardware
• Potential Games are games for which there exists a potential function F : A! R such that ∀ i 2 N, ∀a–i 2 A–i , ∀ ai, ai’ 2 Ai , it holds that
F (ai , a–i) – F (ai’ , a–i) = Ui (ai , a–i) – Ui (ai’ , a–i)
• Key Property: Local maxima of F are Nash equilibria
![Page 11: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/11.jpg)
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
Unifying view of several existing designs:
[Akella et al. 2002][Kaumann et al. 2007]
[Marden et al. 2007, 2008][Mhatre et al. 2007]
[Komali and MacKenzie 2007][Zou and Chakrabarty 2004]
[Campos-Nanez 2008][Marden & W 2008]
[Marden & Effros 2009]and many others…
![Page 12: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/12.jpg)
Utility Design (examples)
• Wonderful Life Utility (WLU) [Wolpert et al. 1999]
– Potential game with © = W (hence, price of stability = 1)– Price of anarchy = ½ for sub-modular games
• Shapley Value Utility (SVU) [Shapley 1953]– Potential game– Price of anarchy = Price of stability = ½ for sub-modular games
• Weighted SVU [Shapley 1953]– Similar properties as SVU
Adapted from cost-sharing literature in economic theory [Marden, Wierman]
![Page 13: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/13.jpg)
Learning Design (examples)• Gradient Play [Ermoliev et al. 1997, Shamma et al. 2005]– Convergence to a Nash equilibrium
• Joint Strategy Fictitious Play (JSFP) [Marden et al. 2009]– Convergence to a Nash equilibrium
• Log-Linear Learning [Blume 1993, Marden et al.]– Convergence to the best Nash equilibrium
• Many others . . . [Ozdaglar et al. 2009, Shah et al. 2010]
![Page 14: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/14.jpg)
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility
? Relationships to other approaches
? Limitations
+ Modularity / Decoupling
+ Flexibility
? Relationships to other approaches
? Limitations
![Page 15: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/15.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Relationships to Other Approaches
Game-theoretic Control
![Page 16: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/16.jpg)
• Distributed Constraint Optimization Problem (DCOP)
– Utility Design: WLU– Learning Design: Variety
Chapman, Rogers, Jennings – Benchmarking hybrid algorithms for distributed constraint optimization games [OptMAS ‘08]
Potential Games
WLU
Variety
Distributed Optimization
![Page 17: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/17.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
![Page 18: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/18.jpg)
• Gibbs-sampler-based control―Utility Design: WLU―Learning Design: Log-Linear Learning
Access Point Selection Channel Selection
Kauffmann, Baccelli, Chaintreau, Mhatre, Papagiannaki, Diot – Measurement-based self organization of interfering 802.11 wireless access networks [INFOCOM ‘07]
Potential Games
WLU
Log-Linear Learning
Physics-inspired Control
We prove that
![Page 19: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/19.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
![Page 20: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/20.jpg)
Distributed Approaches
Distributed Optimization
Lyapunov-based Control
Physics-inspired Control
Potential Games
UtilityDesign
LearningDesign
Game-theoretic Control
Relationships to Other Approaches
![Page 21: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/21.jpg)
Potential Games-based Architecture
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches
? Limitations
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches
? Limitations
![Page 22: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/22.jpg)
Do Potential Games Suffice?No utility design with all the
desirable properties
Utility Design
Learning Design
POTENTIAL GAMES
Desirable properties Existence of an equil. Efficiency of an equil.Budget balanceTractability Locality of information …
Not always!
Open Question: What other limitations are there?
Any linearly separable, budget-balanced utility design that guarantees equilibrium existence has PoS · ½
[Marden, Wierman 2009]
![Page 23: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/23.jpg)
Summary
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
? Beyond Potential Games
![Page 24: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/24.jpg)
Conclusion
Utility Design
Learning Design
Potential Games
SVU WonderfulLife
WSVU
GradientPlay
Log-Linear
Learning JSFP
+ Modularity / Decoupling
+ Flexibility Relationships to
other approaches― Not all desirable
properties can be achieved
?
Other choices for virtualization layer
[MW’09,AJWG’09,Sv’09]Strengths and Limitations
A library of architectures
![Page 25: An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado](https://reader038.vdocuments.us/reader038/viewer/2022103015/551c01e8550346b24f8b4c97/html5/thumbnails/25.jpg)
Thank You