1 language, science and data science kathleen mckeown department of computer science columbia...
TRANSCRIPT
1
Language, Science and Data Science
Kathleen McKeownDepartment of Computer Science
Columbia University
2
3
Science and Language
Predicting impact of scientific journal articles
Generating updates on disaster
4
Machine learning framework
Data (often labeled)
Extraction of “features” from text data
Prediction of output
5
Machine learning framework
Data (often labeled)
Extraction of “features” from text data
Prediction of output
What data is available for learning?
6
Machine learning framework
Data (often labeled)
Extraction of “features” from text data
Prediction of output
What features yield good predictions?
7
Science and Language
Predicting impact of scientific journal articles
Generating updates on disaster
8
9
What can articles yield?
10
Alexandrescu and Kirchhoff, 2007
Lee and Ng, 2002
Ando, 2006Niu et al., 2005
Stevenson and Wilks, 2001
Ng et al., 2003
Rigau et al., 1997
Yarowsky, 1995
Stetina et al., 1998
Ng and Lee, 1996
Peh and Ng, 1997
Yarowsky, 1992
Khapra et al., 2011
Chan et al., 2007
Tim
e
Qazvinian et al, JAIR 2013Citation Network
11
What can articles yield? Argumentative zoning (Teufel, 1996)
“Here, we present quantitative estimates of the global biological impacts of climate changes.” [AIM]
Citation sentiment and scope “The approach of economists takes a
broader view. … We argue that this approach misses biologically important phenomena.”
12
Prediction of Scientific Impact
Climate change, Climate model
Input: term, document
Extract features from full text and metadata
Predict prominence of
term, document
13
Approach
Metadata features Citation network Author collaboration
network
Text features Argumentative
zoning Citation sentiment Primary concepts Relations
Logistic regression to predict future impact
14
Data
4 million full-text Elsevier journal articles
48 million Web of Science metadata records
Fields: medical, chemistry, biology, computer science, …
Ground truth function
15
ExperimentsExperimental comparisons of text features vs metadata features
Data: Selected Elsevier articles and counterparts in WOS
16
What have We Learned?
17
What have we learned?
18
Text features alone outperform metadataElsevier data
Text
Metadata
19
Metadata adds value when combined with textElsevier data
Text
Text+metadata
20
ExperimentsExperimental comparisons of text features vs metadata features
Data: All WOS and Elsevier data
21
What have We Learned?
22
Text better than metadata on Web of Science
Text
Metadata
23
Science and Language
Predicting impact of scientific journal articles
Generating updates on disaster
24
It’s October 28th, 2012
25
26
27
28
29
30
Monitor events over time
Input: streaming data
News, social media, web pages
At every hour, what’s new
31
Data NIST: Trec Temporal Summarization
Track hourly web crawl October 2011 - February 2013 16.1TB
Natural disasters, industrial disasters, social unrest, …
Training Data drawn from Wikipedia
32
33
The U.S. Pacific Tsunami Warning Center said there was a possibility of a local tsunami, within 100 or 200 miles of the epicenter,
34
Would you like to contribute to this story?Start a discussion.
35
Add to Digg. Add to del.icio.us. Add to Facebook.
36
System Update
37
Temporal Summarization Approach
At time t:
1. Predict salience for input sentences Disaster-specific features for predicting
salience
2. Remove redundant sentences
3. Cluster and select exemplar sentences for t
Incorporate salience prediction as a prior
Kedzie & al, Bloomberg Social Good Workshop, KDD 2014Kedzie & al, ACL 2015
38
Predicting Salience: Model FeaturesBasic sentence level features
sentence length punctuation count number of capitalized words number of event type synonyms, hypernyms, and hyponyms
39
Predicting Salience: Model FeaturesBasic sentence level features
sentence length punctuation count number of capitalized words number of event type synonyms, hypernyms, and hyponyms
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
40
Predicting Salience: Model FeaturesBasic sentence level features
sentence length punctuation count number of capitalized words number of event type synonyms, hypernyms, and hyponyms
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
41
Predicting Salience: Model FeaturesBasic sentence level features
sentence length punctuation count number of capitalized words number of event type synonyms, hypernyms, and hyponyms
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
42
Predicting Salience: Model FeaturesLanguage Models (5-gram Kneser-Ney model)
generic news corpus (10 years AP and NY Times articles) domain specific corpus (disaster related Wikipedia
articles)
43
Predicting Salience: Model FeaturesLanguage Models (5-gram Kneser-Ney model)
generic news corpus (10 years AP and NY Times articles) domain specific corpus (disaster related Wikipedia
articles)
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
44
Predicting Salience: Model FeaturesLanguage Models (5-gram Kneser-Ney model)
generic news corpus (10 years AP and NY Times articles) domain specific corpus (disaster related Wikipedia
articles)
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
45
Predicting Salience: Model FeaturesLanguage Models (5-gram Kneser-Ney model)
generic news corpus (10 years AP and NY Times articles) domain specific corpus (disaster related Wikipedia
articles)
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
46
Predicting Salience: Model FeaturesGeographic Features
tag input with Named-Entity tagger get coordinates for locations and mean distance to event
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
47
Predicting Salience: Model FeaturesGeographic Features
tag input with Named-Entity tagger get coordinates for locations and mean distance to event
High SalienceNicaragua's disaster management said it had issued a local tsunami alert.
Medium SaliencePeople streamed out of homes, schools and oce buildings as far north as Mexico City.
Low SalienceAdd to Digg Add to del.icio.us Add to Facebook Add to Myspace
48
Predicting Salience: Model FeaturesTemporal Features measuring burstiness of words
average tf-idf at the current time t0 difference of average tf-idf at time t0 and t - 1 difference of average tf-idf at time t0 and t - 5 hours since event start time
49
SubEvent Identification
➔Decompose articles on a main event into related sub-events:
Manhattan Blackout Breezy Point fire Public Transit Outage
Hurricane Sandy
50
What have We Learned?
51
What have We Learned?
Salience predictions increase precision
52
What have We Learned?
Salience predictions increase precision
Penalizing redundancy removes too many important facts
53
Integrate lens of the media and scientific knowledge
To better predict unfolding impact oof disaster
Social media to feed into modelsof resilience
54
In Sum
Can exploit the words on the web
Language analysis can help the world of science
55
Thank You!
The research presented here has been supported in part by DARPA BOLT, IARPA FUSE, and NSF.