Competing for attention: branching-process models of meme popularity

James P. Gleeson

MACSI, Department of Mathematics and Statistics, University of Limerick, Ireland

#branching

www.ul.ie/gleeson james.gleeson@ul.ie

@gleesonj

NetSci14, Berkeley, 5 June 2014

Branching processes for meme popularity models Overview

Φ

Memory

Network Competition 𝑝𝑘

Branching processes for meme popularity models Part 1

Memory

Network Competition

Motivating examples from empirical work on Twitter

Twitter 15M one-year dataset: collaboration with R. Baños and Y. Moreno

𝛼 = 2

fraction of hashtags with popularity ≥ 𝑛 at age 𝑎

Branching processes for meme popularity models Part 2

Memory

Network Competition

Simon’s model

• Simon, “On a class of skew distribution functions”, Biometrica, 1955 • The basis of “cumulative advantage” and “preferential attachment” models;

see Simkin and Roychowdhury, Phys. Rep., 2011 • During each time step, one word is added to an ordered sequence

• With probability 𝜇, the added word is an innovation (a new word)

• With probability 1 − 𝜇, a previously-used word is copied; the copied word is

chosen at random from all words used to date

time

• Simulation results at age 𝑎 = 25000: seed time is 𝜏, observation time is Ω = 𝜏 + 25000

• Early-mover advantage; fixed-age distributions have exponential tails

[Simkin and Roychowdyury, 2007]

Simon’s model

𝜇 = 0.02

Simon’s model as a branching process

• During each time step, one word is added to an ordered sequence • With probability 𝜇, the added word is an innovation (a new word) • With probability 1 − 𝜇, a previously-used word is copied; the copied word is

chosen at random from all words used to date

𝑡 = 𝜏 𝑡 = Ω

A word on probability generating functions (PGFs)

• PGFs are “transforms” of probability distributions: define PGF 𝑓(𝑥) by • …but “inverse transform” usually requires numerical methods, e.g. Fast

Fourier Transforms [Cavers, 1978] • Some properties:

• PGF for the sum of independent random variables is the product of the

PGFs for each of the random variables e.g., H. S. Wilf, generatingfunctionology, CRC Press, 2005

𝑓 𝑥 = �𝑝𝑘𝑥𝑘∞

𝑘=0

𝑓 1 = �𝑝𝑘

∞

𝑘=0

= 1 𝑓𝑓 1 = �𝑘 𝑝𝑘

∞

𝑘=0

= 𝑧

Branching processes solution of Simon’s model

• Define 𝑞𝑛(𝜏,Ω) as the probability that the word born at time 𝜏 has been used a total of 𝑛 times by the observation time Ω

• Define 𝐻(𝜏,Ω, 𝑥) as the PGF for the popularity distribution

𝐻 𝜏,Ω, 𝑥 = �𝑞𝑛 𝜏,Ω 𝑥𝑛∞

𝑛=1

• Define 𝐺 𝜏,Ω, 𝑥 as the PGF for the excess popularity distribution, so that

𝐻 𝜏,Ω, 𝑥 = 𝑥 𝐺 𝜏,Ω, 𝑥

and 𝐺 Ω,Ω, 𝑥 = 1

Outcome for seed word Probability Contribution to 𝐺 𝜏,Ω, 𝑥

Copied at 𝜏 + Δ𝜏 (1 − 𝜇) Δ𝑡𝜏

𝑥 𝐺 𝜏 + Δ𝑡 2

Not copied 1 − (1 − 𝜇) Δ𝑡𝜏

𝐺 𝜏 + Δ𝑡

𝐺 𝜏,Ω, 𝑥 = 1 − 𝜇

𝛥𝑡𝜏𝑥 𝐺 𝜏 + 𝛥𝑡,𝛺, 𝑥 2 + 1 − (1 − 𝜇)

𝛥𝑡𝜏

𝐺 𝜏 + 𝛥𝑡,𝛺, 𝑥

⇒ −𝜕𝐺𝜕𝜏

≈1 − 𝜇𝜏

𝑥 𝐺2 − 𝐺

𝜏 𝜏 + Δ𝑡

Ω ≫ 𝜏 ≫ Δ𝑡 when

Branching processes solution of Simon’s model

⇒ 𝐺 𝜏,Ω, 𝑥 =𝜏Ω

1−𝜇

1 − 𝑥 1 − 𝜏Ω

1−𝜇

−𝜕𝐺𝜕𝜏

=1 − 𝜇𝜏

𝑥 𝐺2 − 𝐺

Using 𝐻 = 𝑥 𝐺, the corresponding popularity distribution is

𝑞𝑛 𝜏,Ω =𝜏Ω

1−𝜇1 −

𝜏Ω

1−𝜇 𝑛−1

Mean (expected) popularity:

𝑚 𝜏,Ω = �𝑛 𝑞𝑛(𝜏,Ω)∞

𝑛=1

=𝜕𝐻𝜕𝑥

𝜏,Ω, 1 =Ω𝜏

1−𝜇

“Early-mover advantage”

Branching processes solution of Simon’s model

𝜇 = 0.02

• Simulation results at age 𝑎 = 25000: set Ω = 𝜏 + 25000

• Early-mover advantage; fixed-age distributions have exponential tails

[Simkin and Roychowdyury, 2007]

Branching processes solution of Simon’s model

Note 𝛼 ≥ 2

• Power-law distributions arise only after averaging over seed times:

𝑞𝑛 Ω ≡ � 𝑞𝑛 𝜏,Ω1Ω

𝑑𝜏Ω

0

= 1

1 − 𝜇 𝐵 𝑛,

2 − 𝜇1 − 𝜇

∼ 𝑛−𝛼 as 𝑛 → ∞, with 𝛼 = 2−𝜇1−𝜇

Branching processes solution of Simon’s model

A generalization of Simon’s model

Probability that a copying event at time 𝑡 chooses the word from time 𝜏 𝜏 𝑡

𝜙 𝜏, 𝑡 Δ𝑡

Simon’s model: 𝜙 𝜏, 𝑡 = 1𝑡

Copying with memory models: (e.g. Cattuto et al. 2007, Bentley et al. 2011)

𝜙 𝜏, 𝑡 = Φ(𝑡 − 𝜏)

𝐺 𝜏,Ω, 𝑥 ≈ exp (1 − 𝜇)� 𝜙 𝜏, 𝑡 𝑥 𝐺 𝑡,Ω, 𝑥 − 1 𝑑𝑡Ω

𝜏

𝐺 Ω,Ω, 𝑥 = 1 with Ω ≫ 𝜏 ≫ Δ𝑡, when

A generalization of Simon’s model

Probability that a copying event at time 𝑡 chooses the word from time 𝜏 𝜏 𝑡

𝐺 𝜏,Ω, 𝑥 = exp (1 − 𝜇)� 𝜙 𝜏, 𝑡 𝑥 𝐺 𝑡,Ω, 𝑥 − 1 𝑑𝑡Ω

𝜏

Age of seed at observation time is 𝑎 = Ω − 𝜏

For 𝜙 𝜏, 𝑡 = Φ(𝑡 − 𝜏), let 𝐺 𝜏,Ω, 𝑥 = 𝐺�(Ω − 𝜏, 𝑥)

⇒ 𝐺� 𝑎, 𝑥 = exp (1 − 𝜇)� Φ(𝑠) 𝑥 𝐺� 𝑎 − 𝑠, 𝑥 − 1 𝑑𝑠𝑎

0

• In this case, popularity distributions depend only on the age of the seed; there is no early-mover advantage

𝜙 𝜏, 𝑡 Δ𝑡

• Simulation results at age 𝑎 = 25000: set Ω = 𝜏 + 25000

• Memory-time distribution: 𝜙 𝜏, 𝑡 = Φ 𝑡 − 𝜏 = 1𝑇𝑒−(𝑡−𝜏)/𝑇, with 𝑇 = 500

A generalization of Simon’s model

• In this case, popularity distributions depend only on the age of the seed; there is no early-mover advantage

𝜇 = 0.02

𝛼 = 1.5

Competition-induced criticality

Simon’s original model, and the copying-with-memory model both have the following features:

• One word is added in each time step

• Words “compete” for user attention in order to become popular • The words have equal “fitness” – a type of “neutral model” [Pinto and

Muñoz 2011, Bentley et al. 2004 ]

• … except for the early-mover advantage in Simon’s model…

but only the copying-with-memory model gives critical branching processes.

• Gleeson JP, Cellai D, Onnela J-P, Porter MA, Reed-Tsochas F, “A simple generative model of collective online behaviour” arXiv :1305.7440v2

Branching processes for meme popularity models Part 3

Memory

Network Competition

• Each node (of 𝑁) has a memory screen, which holds the meme of current interest to that node. Each screen has capacity for only one meme.

• During each time step (Δ𝑡 = 1/𝑁), one node is chosen at random. • With probability 𝜇, the selected node innovates, i.e., generates a brand-new

meme, that appears on its screen, and is tweeted (broadcast) to all the node's followers.

• Otherwise (with probability 1 − 𝜇), the selected node (re)tweets the meme currently on its screen (if there is one) to all its followers, and the screen is unchanged. If there is no meme on the node's screen, nothing happens.

• When a meme 𝑚 is tweeted, the popularity 𝑛𝑚 of meme 𝑚 is incremented by 1 and the memes currently on the followers' screens are overwritten by meme 𝑚.

The Markovian Twitter model

• Network structure: a node has 𝑘 followers (out-degree 𝑘) with probability 𝑝𝑘.

• In-degree distribution (number of followings) has a Poisson distribution. • Mean degree 𝑧 = ∑ 𝑘𝑝𝑘𝑘 .

• A simplified version of the model of Weng, Flammini, Vespignani, Menczer,

Scientific Reports 2, 335 (2012). • Related to the random-copying “neutral” (Moran-type) models of Bentley

et al. 2004 [Bentley et al. I’ll Have What She’s Having: Mapping Social Behavior, MIT Press, 2011], where the distribution of popularity increments can be obtained analytically [Evans and Plato, 2007].

• Our focus is on the distributions of popularity accumulated over long timescales: when a meme 𝑚 is tweeted, the popularity 𝑛𝑚 of meme 𝑚 is incremented by 1.

The Markovian Twitter model

• When all screens are non-empty, memes compete for the limited resource of user attention

• Random fluctuations lead to some memes becoming very popular, while others languish in obscurity

The Markovian Twitter model

• Random fluctuations lead to some memes becoming very popular, while others languish in obscurity

• The popularity distributions depend on the structure of the network, through the out-degree distribution 𝑝𝑘

𝜇 = 0

𝑝𝑘 = 𝛿𝑘,10

The Markovian Twitter model

• Random fluctuations lead to some memes becoming very popular, while others languish in obscurity

• The popularity distributions depend on the structure of the network, through the out-degree distribution 𝑝𝑘

𝜇 = 0.01

𝑝𝑘 ∝ 𝑘−𝛾; 𝛾 = 2.5

The Markovian Twitter model

overwritten 𝑧 Δ𝑡

𝑡 𝑡 + Δ𝑡

Branching processes solution of Twitter model

Define 𝐺(𝑎, 𝑥) as the PGF for the excess popularity distribution at age 𝑎 of memes that originate from a single randomly-chosen screen (the root screen)

𝑎 𝑎 − Δ𝑡

Outcome for screen 𝑆1 Probability

𝜕𝐺𝜕𝑎

= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝐺𝑓(𝐺) 𝑓 𝑥 = �𝑝𝑘𝑥𝑘∞

𝑘=0

𝐺 0, 𝑥 = 1

selected, innovates 𝜇 Δ𝑡

selected, retweets (1 − 𝜇) Δ𝑡

not selected, survives 1 − (𝑧 + 1) Δ𝑡

𝜕𝐺𝜕𝑎

= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝐺𝑓(𝐺)

𝐻 𝑎, 𝑥 = �𝑞𝑛 𝑎 𝑥𝑛 = 𝑥𝐺 𝑎, 𝑥 𝑓(𝐺 𝑎, 𝑥 )∞

𝑛=0

Analysis of the branching process equation

Mean popularity of age-𝑎 memes:

𝑚 𝑎 = �𝑛𝑞𝑛(𝑎)∞

𝑛=1

=𝜕𝐻𝜕𝑥

𝑎, 1 = 1 + (𝑧 + 1)𝜕𝐺𝜕𝑥

𝑎, 1

So: 𝑑𝑚𝑑𝑎

= (𝑧 + 1)(1 − 𝜇 𝑚)

with 𝑚 0 = 1

𝜕𝐺𝜕𝑎

= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝐺𝑓(𝐺)

𝐻 𝑎, 𝑥 = �𝑞𝑛 𝑎 𝑥𝑛 = 𝑥𝐺 𝑎, 𝑥 𝑓(𝐺 𝑎, 𝑥 )∞

𝑛=0

Analysis of the branching process equation

Mean popularity of age-𝑎 memes:

𝑚 𝑎 = � 1 + 𝑧 + 1 𝑎 if 𝜇 = 01𝜇−

1 − 𝜇𝜇

𝑒−𝜇 𝑧+1 𝑎 if 𝜇 > 0

Analysis of the branching process equation

Mean popularity of age-𝑎 memes:

𝑚 𝑎 = � 1 + 𝑧 + 1 𝑎 if 𝜇 = 01𝜇−

1 − 𝜇𝜇

𝑒−𝜇 𝑧+1 𝑎 if 𝜇 > 0

Long-time (old-age) asymptotics

• If 𝑓′′ 1 < ∞ (finite second moment of 𝑝𝑘),

𝑞𝑛 ∞ ∼ 𝐴 𝑒−𝑛𝜅 𝑛−

32 as 𝑛 → ∞

with 𝜅 = 2𝜇2

𝑓′′ 1 +2𝑧𝑧+1 2

• If 𝑝𝑘 ∝ 𝑘−𝛾 for large 𝑘 with 2 < 𝛾 < 3,

𝑞𝑛 ∞ ∼ �𝐵 𝑛−𝛾

𝛾−1 if 𝜇 = 0𝐶 𝑛−𝛾 if 𝜇 > 0

as 𝑛 → ∞

𝜕𝐺𝜕𝑎

= 0

cf. sandpile SOC on networks [Goh et al. 2003]

Comparing branching process theory with simulations

𝑝𝑘 = 𝛿𝑘,10

𝑝𝑘 ∝ 𝑘−𝛾 𝛾 = 2.5

𝜇 = 0.01

𝜇 = 0

Branching processes for meme popularity models Part 4

Memory

Network Competition

Twitter model with memory

Φ

• During each time step (with time increment Δ𝑡 = 1/𝑁), one node is chosen at random.

• The selected node may innovate (with probability 𝜇), or it may retweet a meme from its memory using the memory distribution Φ(𝑡 − 𝜏).

• Define 𝐺(𝑎, 𝑥) as the PGF for the excess popularity distribution at age 𝑎 of memes that originate from a single randomly-chosen seed (the root)

• The mean popularity 𝑚(𝑎) of age-𝑎 memes has Laplace transform:

Branching process analysis

𝐺 𝑎, 𝑥 = �𝑝𝑘 � 𝑑𝑡 (𝑧 + 𝜇)𝑒− 𝑧+𝜇 𝑡 ×∞

0 𝑘

× exp − 1 − 𝜇 � 𝑑𝑑min 𝒕,𝑎

0

� 𝑑𝜏𝑎−𝑟

0 Φ 𝑎 − 𝑑 − 𝜏 1 − 𝑥 𝐺 𝜏, 𝑥 𝑘

𝑚� 𝑠 = 𝑧 + 𝜇 + 𝑠 + 1 − 𝜇 Φ�(𝑠)

𝑠 𝑧 + 𝜇 + 𝑠 − 1 − 𝜇 𝑧 Φ�(𝑠)

Φ

Memory

Network Competition 𝑝𝑘

Comparing the model to data

𝛾 ≈ 2.13

Φ 𝜏 = Gamma(𝑘,𝜃)

=1

Γ 𝑘 𝜃𝑘 𝜏𝑘−1𝑒−𝜏/𝜃

𝑘 = 0.2; 𝜃 = 355

Φ

𝑚� 𝑠 = 𝑧 + 𝜇 + 𝑠 + 1 − 𝜇 Φ�(𝑠)

𝑠 𝑧 + 𝜇 + 𝑠 − 1 − 𝜇 𝑧 Φ�(𝑠)

Comparing the model to data

𝜇 = 0.02

Comparing the model to data

Data Model

Conclusions: Branching processes for meme popularity models

Φ

Memory

Network Competition 𝑝𝑘

• Competition between memes for the limited resource of user attention induces criticality in this model in the 𝜇 → 0 limit

• Criticality gives power-law popularity distributions and epochs of linear-in-time popularity growth, even for (cf. Weng et al. 2012) – homogeneous out-degree distributions – homogeneous user activity levels

• Despite its simplicity, the model matches the empirical popularity

distribution of real memes (hastags on Twitter) remarkably well

• Generalizations of the model are possible, and remain analytically tractable

Conclusions: Competition-induced criticality

⇒ a useful null model to understand how memory, network structure and competition affect popularity distributions

Davide Cellai, UL Mason Porter, Oxford J-P Onnela, Harvard Felix Reed-Tsochas, Oxford

Jonathan Ward, Leeds Kevin O’Sullivan, UL William Lee, UL

Yamir Moreno, Zaragoza Raquel A Baños, Zaragoza Kristina Lerman, USC

Science Foundation Ireland FP7 FET Proactive PLEXMATH SFI/HEA Irish Centre for High-End

Computing (ICHEC)

Collaborators, funding, references

• “A simple generative model of collective online behaviour” arXiv :1305.7440v2 • Physical Review Letters, 112, 048701 (2014); arXiv:1305.4328

Branching processes for meme popularity models

Φ

Memory

Network Competition 𝑝𝑘

#branching www.ul.ie/gleesonj james.gleeson@ul.ie @gleesonj