![Page 1: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/1.jpg)
Determining Negation Scope and Strengthin Sentiment Analysis
SMC 2011
Paul van ItersonErasmus School of EconomicsErasmus University Rotterdam
Bas HeerschopErasmus School of EconomicsErasmus University Rotterdam
Flavius FrasincarErasmus School of EconomicsErasmus University Rotterdam
Uzay KaymakErasmus School of EconomicsErasmus University Rotterdam
October 12, 2011
Alexander HogenboomErasmus School of EconomicsErasmus University Rotterdam
![Page 2: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/2.jpg)
Outline
• Introduction
• Sentiment Analysis
• Accounting for Negation
• Framework
• Evaluation
• Conclusions
• Future WorkSMC 2011
2
![Page 3: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/3.jpg)
Introduction (1)
• Need for information monitoring tools for tracking sentiment in today’s complex systems
• The Web offers an overwhelming amount of textual data, containing traces of sentiment
SMC 2011
3
![Page 4: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/4.jpg)
Introduction (2)
• Existing sentiment analysis approaches are based on word frequencies
• There is a tendency of involving various other aspects of content in automated sentiment analysis
• Accounting for negation seems promising, but how to model the influence of negation keywords on the conveyed sentiment?
SMC 2011
4
![Page 5: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/5.jpg)
Sentiment Analysis
• Sentiment analysis is typically focused on determining the polarity of natural language texts
• Applications in summarizing reviews, determining a general mood (consumer confidence, politics)
• Common approach to sentiment analysis:– Creation of lexicon (list of words and their sentiment scores)– Utilization of lexicon to determine sentiment in text
• Sentiment analysis approaches differ on several distinguishing characteristic features, e.g.,– Analysis level and focus– Handling of syntactic variations, amplification, and negation
55
SMC 2011
![Page 6: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/6.jpg)
Accounting for Negation (1)
• Common approach: exploitation of negation keywords
• Challenge lies in finding the negation scope
• Sophisticated approaches involve complex rules, compositional semantics, or machine learning
• Many existing sentiment analysis frameworks use rather simple conceptualizations of negation scope
66
SMC 2011
![Page 7: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/7.jpg)
Accounting for Negation (2)
• Let us consider the following positive sentence:– Example: Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
• Rest of Sentence (RoS):– Following: Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes! – Around:Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
• First Sentiment-Carrying Word (FSW):– Following: Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!– Around:Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
77
SMC 2011
![Page 8: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/8.jpg)
Accounting for Negation (3)
• Let us consider the following positive sentence:– Example: Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
• Next Non-Adverb (NNA):– Following: Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
• Fixed Window Length (FWL):– Following (3): Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes! – Around (3): Luckily, the smelly poo did not leave awfully
nasty stains on my favorite shoes!
88
SMC 2011
![Page 9: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/9.jpg)
Framework (1)
• Lexicon-based sentence-level sentiment scoring by using SentiWordNet
• Optional support for sentiment negation
• Individual words are scored in the range [-1,1]
• Word scores are used to classify a sentence as positive (1) or negative (-1)
99
SMC 2011
![Page 10: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/10.jpg)
Framework (2)
• Score sentences in test corpus for their sentiment
• For an arbitrary sentence:– Retrieve all words (simple and compound)– Retrieve each words’ Part-Of-Speech (POS) and lemma– Disambiguate word senses (Lesk algorithm)– Retrieve words’ sentiment scores from lexicon– Negate sentiment scores of negated words, as determined
by means of one of the considered approaches, by multiplying the scores with an inversion factor (typically negative)
– Calculate sentence score as sum of words’ scores– Classify sentence as either positive (score ≥ 0) or negative
(score < 0)
1010
SMC 2011
![Page 11: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/11.jpg)
Evaluation (1)
• Implementation in C#, Microsoft SQL Server database, SharpNLP-based POS tagger, WordNet.Net API for lemmatization and word sense disambiguation, SentiWordNet sentiment lexicon
• Corpus of 930 positive and 1,355 negative manually classified English movie review sentences (60% training set, 40% test set)
1111
SMC 2011
![Page 12: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/12.jpg)
Evaluation (2)
• Baseline: sentiment without accounting for negation
• Alternatives: negation scoping with RoS, FSW, NNA, and FWL (window sizes ranging from 1 to 4)
• Optimized inversion factor for best alternative to a value in the range [-2, 0] (hill-climbing on training set)
1212
SMC 2011
![Page 13: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/13.jpg)
Evaluation (3)
Method Direction Window Inversion Accuracy Macro F1
Baseline - - -1.00 49.9% 49.4%
FSW Following - -1.00 52.3% 52.0%
FWL Following 1 -1.00 51.3% 51.0%
FWL Following 2 -1.00 52.7% 52.4%
FWL Following 3 -1.00 52.0% 51.7%
FWL Following 4 -1.00 52.2% 52.0%
FWL Following 2 -1.27 53.5% 53.3%
1313
SMC 2011
![Page 14: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/14.jpg)
Conclusions
• Recent sentiment analysis methods consider more and more aspects of content other than word frequencies
• Our corpus-based evaluation of several common negation scoping methods shows that only some perform significantly better than our baseline of not accounting for negation
• FWL with a window of 2 words following a negation keyword yields the highest increase in accuracy (5.5%) and macro-level F1 (6.2%) compared to the baseline
• An optimized inversion factor of -1.27 rather than -1 yields an accuracy increase of 7.0% and a macro-level F1 increase of 8.0% compared to the baseline
SMC 2011
14
![Page 15: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/15.jpg)
Future Work
• Let the negation scope detection method depend on the position of a negation keyword
• Deeper understanding of semantics in order to cope with, e.g., context-dependent interpretations
• Distinct sentiment inversion factors for negated positive and negative words
SMC 2011
15
![Page 16: Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649cff5503460f949d00b0/html5/thumbnails/16.jpg)
Questions?
Alexander HogenboomErasmus School of EconomicsErasmus University RotterdamP.O. Box 1738, NL-3000 DRRotterdam, the Netherlands
SMC 2011
16