generating event storylines from microblogs

CIKM’12

Generating Event Storylinesfrom

Microblogs

ABSTRACT

we explore the problem of generating storylines

from microblogs for user input queries.

Given a query of an ongoing event, we propose

to sketch the real-time storyline of the event by a

two-level solution.

1. propose a language model with dynamic

pseudo relevance feedback to obtain relevant

tweets

2. Generate storylines via graph optimization

INTRODUCTION

Generating Event Storyline from Microblogs

(GESM)

differences between GESM and prior studies：

1. Well edited facts ---- short noisy text

2. GESM provides personalized service

3. A two-level framework is necessary: at the low

level, finding all relevant tweets through the

time-line of the event by a retrieve model; and

at the high level, summarizing relevant tweets

and the latent structure to produce a storyline.

INTRODUCTION

INTRODUCTION

Challenges

1、the dynamic and sparse nature of microblogs

——How to match the underlying event expressed

by the vague event query to potential relevant

tweets which possibly not contain any query terms

2、Numerous duplicate tweets and direct and

undirect re-tweets

INTRODUCTION

contributions

1. generating event storylines from microblogs

2. A dynamic pseudo relevance feedback (DPRF)

language

model

3. a graph-based optimization problem and is

solved by approximation algorithms of

minimum-weight dominating set and directed

Steiner tree

THE FRAMEWORK OVERVIEW

generated storyline should be a graph structure

Node is labeled by a summary

Edge represents causal relationship between two

phases

Offline layer

Online layers

THE RETRIEVAL MODEL

Preliminaries

the original query is usually short and vague

Query expansion

In a pseudo relevance manner, suppose the few top

ranked documents d + by the initial query Q builds a

relevant model θ F , we can set the new query to be

a linear combination of original query Q and

relevant model θF

THE RETRIEVAL MODEL

Dynamic Pseudo Relevance Feedback

K burst periods

Assume that the prior probability of relevant

document d + is dependent on the distance of td+

to the centroid

of burst periods, denoted as Φ = { φ 1 ··· φ K }

three probability functions to model the effective

range of burst period, decay coefficient and

skewness.

1. Mixture Gaussian Distribution

2. Local Power Distribution

3. Skewed Linear Distribution

THE RETRIEVAL MODEL

Mixture Gaussian Distribution

Local Power Distribution

Skewed Linear Distribution

THE RETRIEVAL MODEL

Burst Period Detection

1. appear more frequently than usual

2. be continuously frequent around the time point.

detect burst periods of the event by

1. for each query term, finding the time intervals

with arbitrary length in which the query term

appears constantly frequent;

2. picking the time points within these intervals

with the

largest sum of frequencies over all query terms.

THE RETRIEVAL MODEL

“bursty score”

find time interval Tw,j = <st, et, LS, RS> with the maximal cumulative burst score B ( w, Tw,j )

Compute the score of any query term q at each time point

Rank each time point by ∑q∈QH ( q,t )and choose the largest K time point φk .

STORYLINE GENERATION

1. Representative tweets

2. Depict the evolving structure of the event

3. an optimistic connection

a multi-view tweet graph is constructed

a minimum dominant set on the tweet graph

a minimum steiner tree

STORYLINE GENERATION

three non negative real parameters α, τ1, τ2 , τ1<

τ2 .

define E : text similarity > α

define A : τ1 ≤ t j − t i ≤ τ2

w(vi ) = 1 − score ( Q,vi ).

STORYLINE GENERATION

A subset S of the vertex set of an undirected

graph is a

dominating set if for each vertex u ,either u is in

S or is adjacent to a vertex in S .

STORYLINE GENERATION

greedy algorithm

STORYLINE GENERATION

A Steiner tree of a graph G with respect to a

vertex subset S is the edge-induced sub-tree of G

that contains all the vertices of S having the

minimum total cost, where the cost is

the total weight of the vertices.

STORYLINE GENERATION

STORYLINE GENERATION

EXPERIMENTS

Data Set

EXPERIMENTS

Tweet Retrieval

49 queries

evaluation metric :

precision at top 30 tweets(P@30)

mean average precision(MAP)

precision at top 100 tweets(P@100)

R-precision (R-PREC)

EXPERIMENTS

Comparative Study

EXPERIMENTS

Parameter Tuning

EXPERIMENTS

Summarization Capability

EXPERIMENTS

Parameter Tuning

EXPERIMENTS

A User Study

CONCLUSION

The proposed dynamic pseudo relevance

feedback model

minimum weighted Steiner tree on a dominant set

充分的实验

OMG, I Have to Tweet That!

A Study of Factors that Influence Tweet

Rates

Abstract

key limitation ：

it depends on people self reporting their own

behaviors

and observations.

a large scale quantitative analysis of some of

the factors that influence self reporting bias.

the daily variations in tweet rates about weather

events

Introduction

treating social media as a signal to measure the relative real-world occurrence of events

critical challenge ：the bias introduced by the self-reported nature of

social media

What is it about an event that makes it more or less “tweetable”?

A first large-scale, quantitative analysis of some of the factors that influence self-reporting bias by comparing a year of tweets about weather events in cities across the United States and Canada to ground-truth knowledge about actual weather occurrences.

Introduction

three potential factors ：

1. How extreme is the weather?

2. How expected is the weather given the time-of-

year?

3. How much did the weather change?

Data Preparation

Jun 1, 2010 and Jun 30, 2011

56 different metropolitan areas

historical weather data provided by the National

Oceanic and Atmospheric Administration of the

United States.

Identifying Weather-related Tweets

discovering the rate of weather-related tweets

that occurred per-day across metropolitan areas

1. filtering the full archive of tweets for tweets that

contain at least 1 weather-related word from a

list of 179 weather-related words and phrases

2. build a classifier for weather-related tweets

a simple classifier that estimates the probability

of a tweet being weather related as

Identifying the Location of Tweets

geo-coded

the textual user- provided location field in a user’s Twitter profile

normalize the textual

arbitrary user-provided location information into concrete geo-coded coordinates

1. a mapping from user-provided location fields to latitude-longitude coordinates.

2. merge location fields with similar geo-mappings together to create clusters for roughly metropolitan-sized areas

Identifying the Location of Tweets

Historical Weather Data

calculate daily summaries

For each daily summary of weather data at a

location：

Expectation: how normal the observed weather

is at a location

Extremeness : how extreme the weather is on a

particular day

Change: how different the observed weather data

is from previous days’ weather

Analysis and Results

Tweet Rates and Weather Reports

Analysis and Results

Linear Regression

the relationship between a set of weather-derived

features and the daily rate of weather-related

tweets

Analysis and Results

Correlating Basic Weather Data and Tweet

Rates

Analysis and Results

Correlating Expectation and Tweet Rates

expectation measure adds little information about likely tweet rates beyond what is already contained in basic weather data

Correlating Extremeness and Tweet Rates

extremeness can independently explain more of the variation in weather-related tweet rates than basic weather alone

Correlating Delta Change and Tweet Rates

there is little difference in the amount of information gained from building these delta-change models

Combining Extremeness, Expectation, and Delta Change Models

Analysis and Results

Per-Location Models

Discussion

Additional Factors Likely to Effect Tweet

Rates

Sentiment

Privacy concerns, embarrassments and safety:

Population segments :

Mobile devices

Time-of-Day, day-of-week, holiday, and other

effects of time:

Conclusions

the correlation between daily tweet

rates and the expectation, extremeness, and the

change in

observed weather.

global models

location-specific models

Extremeness>change>expectation