informationally efficient multi user communication · 2 • motivation and existing approaches •...

Informationally Efficient Multi‐user communication

Advisor: Professor Mihaela van der Schaar

Electrical Engineering, UCLA

• Motivation and existing approaches

• Informationally efficient multi‐user communication– Vector cases

• Convergence conditions with decentralized information

• Improve efficiency with decentralized information

– Scalar cases• Achieve Pareto efficiency with decentralized information

• Conclusions

Outline

Multi‐user communication networks

Distributed routing

Power control

Peer‐to‐peer system

etc…

Constraints in communication networks

• Resources– Bandwidth, power,

spectrum, etc.

• Information– Real‐time

• Local observation

• Resources– Bandwidth, power,

spectrum, etc.

• Exchanged message

• Resources– Bandwidth, power,spectrum, etc.

• Local observation• Exchanged message

– Non‐real‐time• A‐priori information aboutinter‐user coupling, protocols, etc.

• Resources– Bandwidth, power,spectrum, etc.

• Local observation• Exchanged message

– Non‐real‐time• A‐priori information aboutinter‐user coupling, protocols, etc.

Goal: multi‐user communication without information exchange

A standard strategic game formulation

• Consider a tuple

– The set of players :

– The set of actions: and

– Utility function: and

– Utility region:

In communication networks, different operating

points in can be chosen based on the information

availability

Existing approaches

Nash equilibrium

Existing approaches

Nash equilibrium

• Exchanged messages

Pareto optimality

Price!

Existing approaches

Nash equilibrium

• Exchanged messages

Pareto optimality

Existing results usually assume some

specific action and utility structures!

Price!

• Results with specific action and utility structures– Pure Nash equilibrium

• Concave gamesi) : convex and compact; ii) : quasi‐concave in

• Potential games [Shapley]

• Super‐modular games [Topkis]i) is a lattice; ii)

– Pareto optimality• Network utility maximization [Kelly]

• Convexity is the watershed

Existing approaches (cont’d)

Use gradient play to find NE

Use best response to find NE

Existing approaches (cont’d)Researchers Applications Tools

Altman CDMA uplink power control S‐modular games

Berry Distributed interference compensation S‐modular games

Barbarossa Power control Potential games

Tse Spectrum sharing Repeated games

Kelly End‐to‐end congestion control Pricing

Goodman CDMA uplink power control Pricing

Low End‐to‐end flow control Pricing

Chiang Joint congestion and power control Pricing

Poor Energy efficient power and rate control Equilibrium analysis

Cioffi Power control in DSL systems Equilibrium analysis

Yates Uplink power control for cellular radio Equilibrium analysis

Wicker Selfish users in Aloha Equilibrium analysis

Lazar Non‐cooperative optimal flow control Equilibrium analysis

Special

utility

• Game theory– Equilibrium characterization

– Incentive design

• Optimization theory– Computational complexity

– Distributed algorithms

• Information theory– Fundamental limits

– Encoding and decoding schemes

Information is usually costless

The focus is on strategic interactions among users

Decentralization is not the focus

u2 Pareto boundaryGlobal information

Nash equilibriumDecentralized (limited) information

General modelse.g. concave/potential/supermodular games

Specific multi‐user communication applications

But in many communication systems, information is constrained and no message passing is allowed!

Our goals

Pareto boundaryGlobal (exchanged) information

If information is constrained and

no message passing is allowed…

New classes of communication

When will it convergeto a NE ? And how fast ?

Our goals

How to improve an inefficient NE without message passing ?

Our goals

And can we still achieve Pareto optimality ?

Our goals

• Conclusions

Outline

A reformulation of multi‐user interactions

– The set of players:

– The set of actions:

– State space:

– State determination function:

– Utility function:

In standard strategic game,

It captures the structure of the coupling between action and state

A reformulation of multi‐user interactions

– The set of players:

– The set of actions:

– State space:

– State determination function:

– Utility function:

In standard strategic game,

Many communication

networking applications have

simple , which captures

the aggregate effects of

It captures the structure of the coupling between action and state

• Power control

Communication games with simple states

aggregate interference

• Power control

• Flow controlremaining capacity

• Power control

• Flow control

• Random access

remaining capacity

idle probability

• Conclusions

Outline

• Definition– A multi‐user interaction in which

A1: action set is defined to be

Additively Coupled Sum Constrained Games

A1: action set is defined to be

Structure of the action set:resource is constrained

A2: The utility function satisfies

in which is an increasing and strictly

concave function. Both and

are twice differentiable.

states

Structure of the utility:additive coupling between

action and state

diminishing return per invested action

states

Structure of the utility:additive coupling between

action and state

• Power control in interference channels

Examples of ACSCG

• Delay minimization in Jackson networks

Examples of ACSCG (cont’d)

Nash equilibrium in ACSCG

• Existence of pure NE– A subclass of concave games

• When is the NE unique? When does best response converges to such a NE?– Existing literatures are not immediately applicable

• Diagonal strict convexity condition [Rosen]• Use gradient play and stepsizes need to be carefully chosen

• Super‐modular games [Topkis]• Action space is not a lattice

• Sufficient conditions for specific and [Yu]

• Best response iteration

Best response dynamics

in which is chosen such that

• When does it converges?– By intuition, the weaker the mutual coupling is, the more likely it converges

– How to measure and quantify this coupling strength?

Best response dynamics

Best response dynamicssum constraint

additive coupling

A competition scenario in which every useraggressively uses up all his resources

Best response dynamicssum constraint

additive coupling

Define

represents the maximum impact that user m’s action can make over user n’s state

A measure of the mutual coupling

Convergence conditions

Theorem 1: If

then best response dynamics converges linearly to a unique pure NE for any set of initial conditions.

• The contraction factor is a measure of the overall coupling strength

• If is affine, the condition in Theorem 1 is not impacted by ; otherwise it may depend on .

Contraction mapping

Theorem 1: If

Contraction mapping

is a constant for affine

• If have the same sign, the condition in Theorem 1 can be relaxed to

• This is true in many communication scenarios– Increasing power causes stronger interference

– Increasing input rate congests the server

• If have the same sign, the condition in Theorem 1 can be relaxed to

• This is true in many communication scenarios– Increasing power causes stronger interference

– Increasing input rate congests the server

Strategic complements (or strategic substitutes)

For , define [Walrand]

A special class of

For , define [Walrand]

Define

A special class of

A measure of the similarity between users’ parameters

Theorem 2: If

Theorem 1 Theorem 1

Theorem 2 Theorem 2

Theorem 2: If

Contraction mapping

Conclusion so far…

Nash equilibrium

Pareto boundary

If Information is constrained and

no message passing is available…

Concave games

Power control,Flow control

Conclusion so far…

Nash equilibrium

Pareto boundary

Sufficient conditions that guarantee linear convergence

Concave games

Power control,Flow control

Power control as an ACSCG

Performance comparison • Solutions without information exchange

– Iterative water‐filling algorithm [Yu]

• Solutions with information exchange

k kmn m

m nH P

≠∑

user n’s spectrum

max k kkRω∑

k kmn m

m nH P

≠∑

Performance comparison • Solutions without information exchange

– Iterative water‐filling algorithm [Yu]

• Solutions with information exchange

k kmn m

m nH P

≠∑

user n’s spectrum

max k kkRω∑

k kmn m

m nH P

≠∑

OSB = Optimal

Spectrum

Balancing

ASB = Autonomous

Spectrum

Balancing

• Conclusions

Outline

How to model the mutual coupling

• A reformulation of the coupling– State space

– Utility function

– State determination function

– Belief function

– Conjectural Equilibrium (CE) : a configuration of belief functions and joint action satisfying

n n∈=× NS S:n n nu × →S A R

:n n ns − →A S:n n ns →A S

1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

– Belief function

1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

it captures the

aggregate effect of

the other users’ actions

it models the aggregate effect

of the other users’ actions

– Belief function

1( , , )Ns s∗ ∗1( , , )Na a a∗ ∗ ∗=

( ) ( )n n n ns a s∗ ∗ ∗−= a ( )( )arg max ,

n nn n n n n

aa u s a a∗ ∗

beliefs are realized each user behaves optimally

according to its expectation

it captures the

aggregate effect of

the other users’ actions

it models the aggregate effect

of the other users’ actions

CE in power control games [SuTSP’09]

• One leader and multiple followers

• State space– : the interference caused to user n in channel k

• Utility function

• State determination function

• Belief function (linear form)

21log 1

n k kn nk

⎛ ⎞⎟⎜ ⎟= +⎜ ⎟⎜ ⎟⎜ +⎝ ⎠∑

1,Nk k k

n in ii i nI Pα

= ≠=∑

1 1k k k kI Pβ γ= −

actual play

conceived play

Why Linear belief?is piece‐wise linear; , if the

number of frequency bins is sufficiently large. Linear belief is sufficient to capture the

interference coupling!

∂ = ≠∂

∂∂

∂ = ≠∂

∂∂

12 1f fH P

∂ = ≠∂

∂∂

12 1f fH P

∂ = ≠∂

∂∂

12 1f fH P

∂ = ≠∂

∂∂

12 1f fH P

Main results• Stackelberg equilibrium

– Strategy profile that satisfies

• NE and SE are special CENE:

• Infinite set of CEOpen sets of CE that contain

NE and SE may exist

k k k ki i

iPβ α γ

== =∑

1SER•

1 11 1

1 1, .

k kk k k k

k kI I

I PP P

β γ∂ ∂= − ⋅ = −∂ ∂

( )( )* *1 1,a NE a

( )( ) ( )( )* *1 1 1 1 1 1 1 1, , ,u a NE a u a NE a a≥ ∀ ∈ A

Achieving the desired CE

• Conjecture‐based rate maximization (CRM)

solvable using dual method

leader followers

Discussion about CRM

• Essence of CRM– local approximation of the computation of SE

• Advantages– the structure of the utility function is explored

– only local information is required

– it can be applied in the cases where N>2

– if it converges, the outcome is a CE

Simulation results

Average rate improvements:

2‐user case: 24.4% for user 1; 33.6% for user 2

3‐user case: 26.3% for user 1; 9.7% for user 2&3

( )20.5,k

ijki jα = ≠∑

0.5 1 1.5 2 2.5 30

R1/R1NE

R2/R2NE

0.8 1 1.2 1.4 1.6 1.8 20

R1/R1NE

R2/R2NE

R3/R3NE

( )20.33,k

ijki jα = ≠∑

Concave games

Conclusions so far…

u2Pareto boundary

Nash equilibrium

no message passing is allowed

Power control

Concave games

Overall efficiency may be improved!

u2Pareto boundary

Nash equilibrium

Build belief, learn, and adapt

no message passing is allowed

Power control

• Conclusions

Outline

Linearly coupled games

• A non‐cooperative game model• Users’ states are linearly impacted by their competitor’s actions

• Contributions– Characterize the structures of the utility functions– Explicitly compute Nash equilibrium and Pareto boundary

– A conjectural equilibrium approach to achieve Pareto boundary without real‐time information exchange

A multi‐user interaction is considered a linearly coupled game if the action set is convex and the utility function satisfies

in which . In particular, the basic assumptions about include:

A1: is non‐negative;

A2: is strictly linearly decreasing in ;

is non‐increasing and linear in .

Definition

States are linearly impacted by actions

Denote .

A3: is an affine function,

Definition (cont’d)

Actions are linearly coupled at NE and PB

• For the games satisfying A1‐A4, the utility functions can take two types of form:– Type I [SuJSAC’10]

• e.g. random access

– Type II [SuTR’09]

• e.g. rate control

Two basic types

• For the games satisfying A1‐A4, the utility functions can take two types of form:– Type I [SuJSAC’10]

• e.g. random access

– Type II [SuTR’09]

• e.g. rate control

Two basic types

• Player set: – nodes in a single cell

• Action set:– transmission probability

• Payoff:– throughput

• Key issues– stability, convergence, throughput, and fairness

Type I games: wireless random access

Rx1Tx2

• Individual conjectures– state:

– linear belief:

• Two update mechanisms – Best response

– Gradient play

Conjecture‐based Random Access

actual play

conceived play

Main results

• Existence of CE– all operating points in action space are CE

• Stability and convergence– sufficient conditions

• Throughput performance– the entire throughput region can

be achieved with stable CE

• Fairness issue– conjecture‐based approaches

attain weighted fairness

Protocol design: how to achieve efficient outcomes?

How to select suitable ak?

• Adaptively alter ak when the network size changes

• Adopt aggregated throughput or “idle interval” as the indicator of the system efficiency

• Advantages– No need of a centralized solver– Throughput efficient with fairness guarantee– Stable equilibrium– Autonomously adapt to traffic fluctuation

Engineering interpretation• DCF vs. the best response update

– re‐design the random access protocol

similar different

CBRA makes use of 4-bit information, while DCF only uses 2 bits

Simulation results

• Throughput

• Stability and convergence

5 10 15 20 25 30 35 40 45 5025

Number of nodes

s) Optimal throughputP-MACConjecture-based algorithmsIEEE 802.11 DCF

0 100 200 300 400 500 60031

P-MACBest responseGradient play

DCF: low throughput; P‐MAC: needs to know the number of nodes

P‐MAC: instability due to the online estimation

• Utility function

• Nash equilibrium

• Pareto boundary

• Efficiency loss

Conventional solutions in Type II games

• At stage t,

• Theorem 5: A necessary and sufficient condition for the best response dynamics to converge is

Best response dynamics in Type II games

Determine the eigenvalues of the Jacobian matrix

Observed state Linear belief

• Theorem 6: All the operating points on the Pareto boundary are globally convergent CE under the best response dynamics. The belief configurations lead to Pareto‐optimal operating points if and only if

– : the ratio between the immediate performance degradation and the conjectured long‐term effect

Stability of the Pareto boundary

Theorem 5 and expressions of Pareto boundary and CE

Pricing vs. conjectural equilibrium

• Pricing mechanism in communication networks [Kelly][Chiang]– Users repeatedly exchange coordination signals

• Conjectural equilibrium for linearly coupled games– Coordination is implicitly implemented when the participating users initialize their belief parameters

– Pareto‐optimality can be achieved solely based on local observations on the states

– No message passing is needed during the convergence process

– The key problem is how to design belief functions

u2 Pareto boundaryGlobal (exchanged) information

Decentralized (insufficient) information

The optimal way of designing the beliefs and updating the

actions based on conjectural equilibrium is addressed

Can we still achieve Pareto optimality ?

Concave games

Random Access,Rate control

u2 Pareto boundaryGlobal (exchanged) information

Conjectural equilibrium

Decentralized (insufficient) information

The optimal way of designing the beliefs and updating the

actions based on conjectural equilibrium is addressed

Can we still achieve Pareto optimality ?

Concave games

Random Access,Rate control

Pareto optimality can be achieved!

Conclusions

• We define new classes of games emerging in multi‐user communication networks and investigate the information and efficiency trade‐off– Provide sufficient convergence conditions to NE

– Suggest a conjectural equilibrium based approach to improve efficiency

– Quantify the performance improvement

References• J. Rosen, “Existence and uniqueness of equilibrium points for

concave n‐person games,” Econometrica, vol. 33, no. 3, pp. 520‐534, Jul. 1965.

• D. Monderer and L. S. Shapley, “Potential games,” Games Econ. Behav., vol. 14, no. 1, pp. 124‐143, May 1996.

• D. Topkis, Supermodularity and Complementarity. Princeton University Press, Princeton, 1998.

• F. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control in communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research Society, vol. 49, pp. 237‐252, 1998.

• M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering as optimization decomposition: A mathematical theory of network architectures,” Proc. of the IEEE, vol. 95, no. 1, pp. 255‐312, January 2007.

References (cont’d)

• W. Yu, G. Ginis, and J. Cioffi, “Distributed multiuser power control for digital subscriber lines,” IEEE J. Sel. Areas Commun., vol. 20, no. 5, pp. 1105‐1115, June 2002.

• J. Mo and J. Walrand, “Fair end‐to‐end window‐based congestion control,” IEEE Trans. on Networking, vol. 8, no. 5, pp. 556‐567, Oct. 2000.

References (cont’d)• Y. Su and M. van der Schaar, “Structural solutions for

additively coupled sum constrained games,” UCLA technical Report, 2010.

• Y. Su and M. van der Schaar, “Conjectural equilibrium in multiuser power control games,” IEEE Trans. Signal Processing, vol. 57, no. 9, pp. 3638‐3650, Sep. 2009.

• Y. Su and M. van der Schaar, “A new perspective on multi‐user power control games in interference channels,” IEEE Trans. Wireless Communications, vol. 8, no. 6, pp. 2910‐2919, June 2009.

• Y. Su and M. van der Schaar, “Linearly coupled communication games,” UCLA technical Report, 2009.

• Y. Su and M. van der Schaar, “Dynamic conjectures in random access networks using bio‐inspired learning,” IEEE JSAC special issue on Bio‐Inspired Networking, May 2010.

Linear convergence

• A sequence with limit is linearly convergent if there exists a constant such that

for k sufficiently large.

Solutions with information exchange

• Users aim to solve

• They can pass coordination messages

and user n behaves according to

user n’s impact over user m’s utility

• Gradient play

Theorem 3: If

gradient play converges for a small enough stepsize.Lipschitz continuity and gradient projection algorithm

• Jacobi update

Theorem 4: If

Jacobi update converges for a small enough stepsize.

Lipschitz continuity, descent lemma, and mean value theorem

• Convergence to an operating point that satisfies the KKT conditions is guaranteed

• Total utility is monotonically increasing

• Global optimality is guaranteed if the original problem is convex, otherwise not

• Developed for general non‐convex problem in which convex NUM solutions may not apply in general

Stackelberg equilibrium

• Definition– Leader (foresighted): only one

– Follower (myopic): the remaining ones

– Strategy profile that satisfies

• Existence and computation of SE in the power control games [SuTWC’09]

( )( )* *,n na NE a

( )( ) ( )( )* *, , ,n n n n n n n nu a NE a u a NE a a≥ ∀ ∈ A

A two‐user formulation

• Bi‐level Programming

problem

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

problem

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

1 2 21

2 1 11

max ln 1 ( )

. . , 0, ( )

argmax ln 1 ( )

. . , 0. ( )

k k kkK k kk

k k kk

K k kk

s t P P b

s t P P d

⎛ ⎞⎟⎜ ⎟+⎜ ⎟⎜ ⎟⎜ +⎝ ⎠

≤ ≥

⎛ ⎞′ ⎟⎜ ⎟= +⎜ ⎟⎜ ⎟⎜ +⎝ ⎠

′ ′≤ ≥

2 2 2 2 2 21 1 11 1 12 22 2 2 22 2 21 11, , ,k k k k k k k k k k k kN H H H N H H Hσ α σ α= = = =

Problems with the SE formulation

• Computational complexity– intrinsically hard to compute

• Information required for playing SE– Global information

• Realistic assumption– Local information

– Any appropriate solutions other than SE and NE?

{ } { } { }, ,k kij iα σ max

1 12,N k k k

n nnPα σ

=+∑ max

• Priority‐based fair medium access control– Traffic classes with positive weights

• Conjecture‐based protocol

Weighted Fairness

Some distributed iterative algorithms

• Best response

• Jacobi update

• Gradient play stepsize

informationally efficient multi user communication · 2 • motivation and existing approaches •...

Documents

efficient frontier for multi-objective stochastic

design guidelines for energy-efficient multi-storey ......

on the impossibility of informationally efficient …on the...

efficient multi-scale modelling of path dependent …

development of ros-tms 5.0 for informationally structured

development of highly efficient multi-invariable wireless

robust and efficient multi-robot coverage

multi-prediction particle filter for efficient

efficient multi-order uncertainty computation for

simultaneous multi-processor cores for efficient embedded

strategic trading in informationally complex...

informationally decentralized video streaming over multihop

master thesis: efficient multi-modal route planning

searching for efficient multi-scale architectures for dense...

chancel: efficient multi-client isolation under

chapter 2: the domestic and international financial ... ·...

on the impossibility of informationally efficient...

system support for efficient multi-resolution visual

energy-efficient buildings ppp multi-annual...

efficient multi-match packet classification with tcam