1 emotion classification using massive examples extracted from the web ryoko tokuhisa, kentaro inui,...
TRANSCRIPT
![Page 1: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/1.jpg)
1
Emotion Classification Using Massive Examples Extracted from the Web
Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto
Toyota Central R&D Labs/Nara Institute of Science and Technology
Coling 2008
![Page 2: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/2.jpg)
2
Emotion Classification Given an input sentence, classify the
sentence into 10 emotion classes Need to construct an emotion-provoking
corpus (EP corpus) Example:
I was disappointed because the shop was closed and I’d traveled a long way to get there
Emotion: disappointment Event: the shop was closed and I’d traveled a
long way to get there
![Page 3: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/3.jpg)
3
Building an EP corpus (1/8) Ten emotions: happiness,
pleasantness, relief, fear, sadness, disappointment, unpleasantness, loneliness, anxiety, anger
Built a hand-crafted lexicon of emotion words from the Japanese Evaluation Expression Dictionary
349 emotion words
![Page 4: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/4.jpg)
4
Building an EP corpus (2/8)
![Page 5: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/5.jpg)
5
Building an EP corpus (3/8)
A subordinate clause was extracted as an emotion-provoking event if (a) it was subordinated to a matrix
clause headed by an emotion word (b) the relation between the
subordinate the matrix clauses is marked by eight connectives (Japanese)
![Page 6: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/6.jpg)
6
Building an EP corpus (4/8)
“It suddenly started raining” provokes disappointment
![Page 7: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/7.jpg)
7
Building an EP corpus (5/8) Applying the lexical patterns to the
Japanese Web corpus, which contains 500 million sentences, 1.3 million events were collected
![Page 8: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/8.jpg)
8
Building an EP corpus (6/8)
![Page 9: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/9.jpg)
9
Building an EP corpus (7/8) An annotator evaluated 2000 randoml
y chosen events Correct: correct example Context-dep: context-dependent Error: error example
![Page 10: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/10.jpg)
10
Building an EP corpus (8/8)
![Page 11: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/11.jpg)
11
Sentiment Polarity Classification (1/5)
Finding neutral events is difficult Collected 1000 sentences
randomly from the web and investigate their polarity
![Page 12: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/12.jpg)
12
Sentiment Polarity Classification (2/5) Two-step approach
First classify a given input into three sentiment polarity classes, either positive, negative or neutral
Then classify only those judged positive or negative into the 10 fine-grained emotion classes
![Page 13: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/13.jpg)
13
Sentiment Polarity Classification (3/5) The sentiment classification model is
trained with SVMs Test sentence is neutral if the output
of the classification model is near the decision boundary
Features are 1-gram, 2-gram and 3-gram extracted from word-polarity lattice
![Page 14: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/14.jpg)
14
Sentiment Polarity Classification (4/5)
Features: child, positive, child-of, positive-of, child-of-education, …
![Page 15: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/15.jpg)
15
Sentiment Polarity Classification (5/5) Polarity value of each word is
defined in a sentiment dictionary, which includes 1880 positive words and 2490 negative words One annotator identified positive and
negative words from the 50 thousand most frequent words samples from the Web
![Page 16: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/16.jpg)
16
Emotion Classification
Use k-nearest-neighbor approach (kNN)
![Page 17: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/17.jpg)
17
Experiments: Sentiment polarity classification (1/2)
TestSet1: 6 subject speakers produce 31 positive utterances, 34 negative utterances, and 25 neutral utterances
TestSet2: Used the 1140 samples that were labeled Correct before 491 positive samples, 649 negative
samples Add 501 neutral samples from the Web
![Page 18: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/18.jpg)
18
Experiments: Sentiment polarity classification (2/2)
![Page 19: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/19.jpg)
19
Experiments: Emotion classification (1/3) TestSet1 (2p, best): Two annotators annotated
each positive or negative sentence in TestSet1 with exactly one of the 10 emotion classes. A model’s answer is correct if it was identical with either of the two labels
TestSet1 (1p, acceptable): One annotator is asked to annotate each positive or negative sentence in TestSet1 with all the emotions involved in it. A model’s answer was considered correct if it was identical with one of the labeled classes
TestSet2: Use the results from the evaluation of EP corpus’ quality
![Page 20: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/20.jpg)
20
Experiments: Emotion classification (2/3) Baseline
Kozareva’s PMI unsupervised model Use Google search engine to obtain an e
motion and a word’s co-occurrence frequency
k-NN 1-NN, 3-NN, 10-NN
![Page 21: 1 Emotion Classification Using Massive Examples Extracted from the Web Ryoko Tokuhisa, Kentaro Inui, Yuji Matsumoto Toyota Central R&D Labs/Nara Institute](https://reader034.vdocuments.us/reader034/viewer/2022042718/56649eac5503460f94bb1e34/html5/thumbnails/21.jpg)
21
Experiments: Emotion classification (3/3)