combining social data and semantic content analysis for l ... · combining social data and semantic...

Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network

Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops (Università degli Studi di Bari ‘Aldo Moro’, Italy - SWAP Research Group)

I-CiTies 2015 2015 CINI Annual Workshop on ICT for Smart Cities and Communities

Palermo (Italy) - October 29-30, 2015

2

April 6, 20095.8 magnitude earthquake20 billions damages70,000 people displaced309 people died

Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network. iCities 2015 Workshop, Palermo (Italy) 29.10.2015

L’Aquila

3

2015: six years later7 billions fundings still needed22,000 people still displacedDiaspora


L’Aquila

4

19 ‘new towns’ around l’Aquila 15,200 people today live there


L’Aquila

5

What about the consequences?

Loss of trust, sense of belonging, relationships


L’Aquila

6

Loss of social capitalCataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network. iCities 2015 Workshop, Palermo (Italy) 29.10.2015

L’Aquila

7Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network. iCities 2015 Workshop, Palermo (Italy) 29.10.2015

L’Aquila Social Urban Network



Our contribution!

9

Research Question:Is it possible to extract and process social

media to monitor in real time people feelings, opinions and sentiments about the current

state of the social capital of L’Aquila?



10

A framework for real-time Semantic Analysis of Social Streams


CrowdPulse

11

CrowdPulse

Social Data Extraction

features

Semantic Tagging

Sentiment Analysis Processing & VisualizationCataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network. iCities 2015 Workshop, Palermo (Italy) 29.10.2015

12

workflowCrowdPulse


13

Step 1: Social Data ExtractionCrowdPulse


14

Step 1: Social Data Extraction

Extraction

Source

Heuristics

CrowdPulse


15


Extraction

Source

Heuristics

CrowdPulse


16


Extraction

Source

Heuristics

ContentUser

Geo

Content+Geo

#www2015#democrats

#traffic

@barack_obama@comunefi

#earthquake

Page

Group

CrowdPulse


17


Extraction

Source

Heuristics

ContentUser

Geo

Content+Geo

#icities2015#democrats

#traffic

@barack_obama@comunepalermo

#earthquake

Page

GroupWe only extract public content

CrowdPulse


18

Use CaseL’Aquila Social Urban Network

Heuristics: - Twitter users (local newspapers, mention to politicians) - Twitter content+geo (50km around l’Aquila and/or specific hashtags as #laquila #earthquake, etc)

CROWDPULSE SETTINGS


19


CROWDPULSE SETTINGS

Heuristics: - Facebook groups (identified after a thorough analysis) - Facebook pages (identified after a thorough analysis)


20


Extracted content (example)

Tweets about the fear of new earthquakes.

Facebook posts about citizens’ proposals.

Tweets about people worried of the situation.Tweets about new buildings in the city.

CROWDPULSE SETTINGS


21


Sentiment Analysis and Semantic Tagging of the content

CROWDPULSE SETTINGS


Keyword-based representation introduces a lot of noise in the analysis

22

aquila

??

(eagle)

(italian city)

(italian)

Semantic TaggingMotivations


(Please, do something: l’Aquila is going to die!)(Please, do something: the eagle is going to die!)

“Fate qualcosa per favore, l’Aquila sta morendo!”

?

23

Semantic TaggingMotivations


24

Step 2: Semantic TaggingCrowdPulse

Non-trivial NLP tasks (stopwords removal, n-grams identification, named entities recognition and disambiguation) are automatically performed

identification and disambiguation of the entities mentioned in the text.


25

CrowdPulseStep 3: Sentiment Analysis


26

Sentiment AnalysisMotivations

Is this content conveying any opinion?


27

Sentiment AnalysisMotivations

Is this content conveying any opinion?

This is a crucial issue if people-based findings have to be generated


28

Sentiment AnalysisDefinition

“It is the field of study that analyzes people’s

opinions, sentiments, evaluations, appraisals, attitudes, and emotions towards entities such as

products, services, organizations, individuals, issues, events, topics, and

their attributes “ (*)

(Pang, Bo, and Lillian Lee. "Opinion mining and sentiment analysis." Foundations and trends in information retrieval, 2008)

We concentrated on the polarity detection taskCataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops Combining Social Data and Semantic Content Analysis for L’Aquila Social Urban Network. iCities 2015 Workshop, Palermo (Italy) 29.10.2015

29


Overall sentiment: :-(


30




The process can be iterated over a larger set of content, to get findings about the feeling of the

population regards a certain topic

31




32

CrowdPulseStep 4: Processing & Visualization


33


How to map each content with the social indicator it refers to?

CROWDPULSE SETTINGS


34


Given a fixed set of social capital indicators, we built a classification model to associate each content (along with

its sentiment) to the social indicator it refers to.

CROWDPULSE SETTINGS


35


Tweet about new buildings in the city.

Social Capital Mapper



36



Input: Social indicators + classification model



37


Domain-specific processing: Classification task



38


Output: (multi-class) classification + sentiment



39



The score of a social indicator is the average sentiment of all the content referring to it.


40


CROWDPULSE OUTPUT

Overall score of the social indicators between March and August 2014


41


CROWDPULSE OUTPUT

COMMUNITY PROMOTER

DEFINES SOME INITIATIVES TO EMPOWER THE SOCIAL CAPITAL

MONITORS THE STATE OF THE SOCIAL INDICATORS

Real-world applicationof the output


Lessons Learned


Lessons Learned

43

Pipeline of state of the art techniquesSemantic Processing, Sentiment Analysis, Machine Learning, Data Visualization

Use Case: L’Aquila Social Urban Network

DEFINITION OF A FRAMEWORK FOR REAL-TIME SEMANTIC CONTENT ANALYSIS


Thanks to the huge availability of textual data very complex

phenomena can be analyzed in a totally new way

questions?Cataldo Musto, Ph.D [email protected]

@cataldomusto

mailto:[email protected]

combining social data and semantic content analysis for l ... · combining social data and semantic...

Documents