1

A Generic Mean Field Convergence Result for Systems of Interacting Objects

From Micro to Macro

Jean-Yves Le Boudec, EPFL

Joint work with David McDonald, U. of Ottawaand Jochen Mundinger, EPFL

2

The full text of my talk is available in the proceedings of QEST 2007

The paper and this slide show are

also available from my web page

3

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

4

Motivation

Find re-usable approximations of large scale systems

Examples from my fieldPerformance of UWB impulse radio : many sensors, each has a MAC layer stateAd-Hoc networkingReputation Systems

From microscopic description to macroscopic equations

Understand fluid approximation and mean field approximation

5

Example 1 : TCP/ECN

TCP connection n transmits at a rate 2 {s0, …, si, …, sI}

Queue length at router is R(t) With probability q(R(t))

connection i receives an Explicit Congestion Notification (ECN) in next time slot

When connection n does not receive an ECN, it increases its rate:

If rate == si, new rate := si+1 (i<I)

Else it decreases its rate:If rate == si, new rate := sd(i)

ECN router

queue length R(t)

ECN Feedback q(R(t))

N connections

1

n

N

The question is the behaviour when N is large

6

Microscopic Description Time is discrete Connection n runs one Markov chain XN

n(t);

The transition probabilities of the Markov chain XNn(t) depend on

global state R(t) (queue size)

Global state R(t) depends on states of all connectionslet MN

i(t) = nb of connections in state i at time t , C = service rate of

router

ECN received

no ECN received

7

Macroscopic Description The fluid approximation is often given as a simplification of the

previous model

Combined with

we have a macroscopic description of the system

In [17], Tinna. and Makowski show that it holds as large N asymptotics

8

The Mean Field Approximation Assume we want to analyze one TCP connection in detail We can keep the microscopic description for this TCP

connection, and use the fluid approximation for the others: We can call it fast simulation.

i.e. pretend XN1(t) (one connection) and R(t) (global resource) are

independent. This is similar to what is called the mean field approximation in physics

9

Another Example: Robot Swarm

N robots Robot has S = 2 possible states Transition for one robot depends

on this robot’s state + how many other robots are in search state

[11] uses the fluid approximation :

10

A few other Examples …

11

In these and other examples, some authors assume the validity of the fluid / mean field approximation and use the approximation to do performance evaluation, parameter identification, control… Never

again !

… while, in contrast, others spend most of the paper proving the derivation and validity of the approximations in their specific setting

papers in this latter class are intimidatingcost of proof of one approximation result ¼ 1 PhDand not re-usable

Proof of convergence to

Mean field

for TCP/ECN

12

Can we have answers of general applicability to:

When are the fluid approximation and the mean field approximation valid ?

Can we write them in a sound ( = mechanical) way ?

13

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

14

Mean Field Interaction Model

A Generic Model, with generic results Does not cover all useful cases, but is a useful first step

Time is discrete N objects Every object has a state in .

Informally: object n evolves depending only onIts own stateA global resource whose evolution depends only on how many other objects are in each state

15

XNn(t) : state of object n at time t

MNi(t) = proportion of objects that are in state i MN is the “occupancy measure” ¼ the “mean field”

RN(t) = global resource =“history” of occupancy measure

Conditional to history up to time t, objects draws next state independent of each other according to

Model Assumptions

16

Two Mild Assumptions

1. Continuity of the integration function g()

2. For large N, the transition matrix K becomes independent of N and is continuous

17

TCP/ECN Example fits in this Framework

Intuitively satisfies the conditionsState of one connection depends only on buffer contentBuffer contents depends only on how many connections are in each state

Formally:One object = one TCP connectionState of one object = index i of sending rateRN(t) = total buffer occupancy / N

Function g() :

thus

g() is continuousAssumption 1 is satisfied

ECN router

queue length R(t)

ECN Feedback q(R(t))

N connections

1

n

N

18

TCP/ECN Example fits in this Framework

Transition matrix K

Let q(r) = proba of negative feedback when R==r

K is independent of N thus Assumption 2 is is satisfied if q() is continuous

ECN router

queue length R(t)

ECN Feedback q(R(t))

N connections

1

n

N

ECN received

no ECN received

19

A Multiclass Variant

Take same as previous TCP/ECN model but introduce multiclass

Aggressive connections, normal connection

State of an object = (c, i)c : classi : sending rate

Objects may change class or not

Also fits in our framework

Mean Field does not mean all objects are exchangeable !

20

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

21

22

Practical Application : Derivation of the Fluid Approximation

The theorem replaces the stochastic system by a deterministic, dynamical system

This gives a method to write and justify the fluid approximation in the large N regime

Equation for the limiting occupancy measure can be rewritten as

where Ni(t) = N MNi(t) = number of objects in state i at time t

This recovers for example the result in [17]

23

Proof of Theorem

Based on The next theorem (fast simulation)A coupling argumentAn ad-hoc version of the strong law of large numbersThe Glivenko Cantelli lemma

24

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

25

Fast Simulation / Analysis of One Object

Assume we are interested in one object in particularE.g. distribution of time until a TCP connection reaches maximum rate

For large N, since mean field convergence holds, one may do the mean field approximation and replace the set of other objects by the deterministic dynamical system

The next theorem says that, essentially, this is valid

26

Fast Simulation Algorithm

Returns next state for one objectWhen transition matrix is K

State of one specific object

This is the mean field independenceapproximation

Replace true value by deterministiclimit

27

Fast Simulation Result

28

Practical Application

This justifies the mean field approximation for the stochastic evolution of one object in the large N regime

Gives a method for fast simulation or analysisThe state space for Y1 has S states, instead of SN

29

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

30

A Reputation System

My original motivation for this work Illustrates the complete set of steps, including a few modelling

tricks System

N objects = N peersPeers observe one subject and rate itRating is a number in (0,1)Direct observations and spreading of reputationConfirmation bias + forgetting

31

Operation of Reputation System: Forgetting

Zn(t) = reputation rating held by peer n

During a direct observation, subject is perceived as positive (with proba ) or negative (with proba 1-)

In case of direct positive observation

In case of direct negative observation

w is the forgetting factor, close to 1 (0.9 in next slides)

32

Confirmation Bias

Peer also read other peer ratings If overheard rating is z:

is the threshold of the confirmation bias

33

Liars and Honest Peers

Honest peer does as just explained Liar tries to bring the reputation down

Uses different strategies, see later

34

Initially: peers have Z=0, 0.5 or 1

= 0.9

Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69

Example of exact simulation: N=100 peerswith maximal liars (always say Z=0)

ratingprop

orti

on o

f pe

ers

35

3 particular peers, one of each type

= 0.9

time

rating

36

Can we study the system with 106 users instead of 100 ?

37

The problem fits in our framework…

Assume discrete time At every time step a peer

Makes a direct observationOr overhears a liarOr overhears some honest peerOr does nothing

Object = honest peer

Assume first that liars use strategy 1: maximal lying (always say Z=0)

Transition of one honest peer depends onOwn stateDistribution of states of all other peers

=> Fits in our framework with memory R = occupancy measure M

38

Different Liar Strategies

Strategy 1 (maximal lying): liars always say Z= 0 Strategy 2 (infer): liar guesses your rating based on past

experienceTransition of one honest peer depends on

Own stateDistribution of states of all other peersWhat liars remember seeing in the past

=> Fits in our framework with memory R = occupancy measure of ratings at steps t and t-1

Strategy 3 (side information): liars know your rating and is as negative as you acceptnot realistic but serves as benchmark (worst case)

Similar to strategy 1, memory = occupancy measure M

39

We would like to apply the mean field convergence result to analyze very large N

But model has continuous state space

Discretize reputation ratings !Quantize Zn on ca. L bits; replace Zn by Xn = 2L ZN with

Issue: small increments due to “forgetting” coefficient w (e.g. w = 0.9) are set to 0

Solution: use random rounding; replace previous equation by

where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7 E(RANDROUND(x)) = x

40

Transition Matrix K

The transition matrix KN is straightforward but tedious to describe.

Unlike in the TCP/ECN example, it does depend on N

It contains terms such as : the proba that an indirect observation with a honest peer is with someone who has rating equal to k. This proba is equal to

It depends on N, but for large N it converges uniformly to MNk(t),

with no term in N

The limiting matrix K is polynomial in MNk(t), thus continuous,

thus assumption 2 is satisfied

Assumption 1 is trivially satisfied, by inspection

41

Therefore we can apply the theorem and derive the fluid approximation and the mean field approximation

Both are true in the limit N = 1

42

Limiting reputation ratings: 0.9 and 0.1

Discrete event simulation, N = 100 Fluid Approximation

Fast Simulation based on Mean Field Approximation

43

Fluid approximationCan be written using Theorem 4.1 Is a deterministic recurrence with state vector the memory number of dimensions is 2 L+1, where L = number of quantization bits for reputation values (e.g. L=8)

Mean Field Approximation = Fast SimulationSimulation of one Markov chain on state space with 2 L states, with time varying transition probability

44

Different Parameters (few liars)

Few liarsFinal ratings converge to true value

Phase transition

45

Different Initial Conditions

46

Liar Strategy 2(infer)

Liar Strategy 3(side information)

Peers starting after 512 time units

47

Modelling Locality with Multiclass Model

We can model spatial aspectsObject = honest peer ; state = (c, x) with

C = location (in a discrete set of locations)X = rating (same as before)

This allows to account for locality of interaction

48

Contents

E.L.

1. Motivation

2. A Generic Model for a System of Interacting

Objects

3. Convergence to the Mean Field

4. Fast Simulation

5. Full Scale Example: A Reputation System

6. Outlook

49

I have shown how a mean field convergence result can be used to write and validate

the fluid approximation = macroscopic descriptionthe mean field approximation = fast simulation (or analysis)

Applies to cases where objects interact such thatTransition depends on state of this object + current and past distribution of states of all other objectsNumber of objects is large compared to number of states of one object

Extensionsbirth and death of objects transitions that affect several objects simultaneouslygaussian approximations (central limit theorems)

Outlook

50

… thank you for your attention

E. L.