![Page 1: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/1.jpg)
Using Sentiment Analysis to Fill in the Gaps in User
SurveysThat’s a really long title
![Page 2: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/2.jpg)
YOW! Data 2017My lack of expertise
Confusing setup
Flaunted my poor understanding of underlying tech
Audience understood less than when they went in
Dance number
![Page 3: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/3.jpg)
TodayLower expectations
Confusing setup
Poor explanation of the technology
Ambiguous results
Silent weeping
![Page 4: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/4.jpg)
Confusing Setup
![Page 5: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/5.jpg)
SuperannuationWe all love superannuation
Can we measure just how much we love it?
![Page 6: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/6.jpg)
Two SourcesPhone calls with consultants Web chat logs
![Page 7: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/7.jpg)
Phone callsSpeech-to-text
Poor quality, single-channel
Comical results
Much funnier presentation
![Page 8: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/8.jpg)
Transcript
“hope hi there John my name is Spider-Man advise of the strange episode advice can you please get some of that your separation under some fantastic now we all in 20 minutes according to that and so that I can include details Yu-Gi-Oh counterpart of a conversation”
![Page 9: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/9.jpg)
Web chat logsAgent: Hello, I am totally a human being. My name is Agent.
Customer: Yes, indeed. May I just say how much I love Super?
Agent: Me too, after all, it is called “super” for a reason.
Customer: How can I get more Super into me?
Agent: Would you be interested in our new “Super Super”?
Customer: Here is all my money, it is yours.
![Page 10: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/10.jpg)
Web chat logs13 months JSON data
100k conversations
1 million lines of dialogue
Metadata
![Page 11: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/11.jpg)
Web chat logsMeaningful Connection Score
Agent-supplied rating
-100 to +100
Almost every chat has one
Exit Survey
User-supplied rating
10% get asked
10% of those will do it
~1000 conversations have one
0 to 10
![Page 12: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/12.jpg)
Web chat logsNobody wants to fill in a survey
Can we use Sentiment Analysis as an indicator?
![Page 13: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/13.jpg)
Poor Technical Explanation
![Page 14: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/14.jpg)
Sentiment AnalysisClassification problem
![Page 15: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/15.jpg)
First ProblemHumans only agree on the sentiment of text about 80% of the time
![Page 16: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/16.jpg)
ComplicatedI like cheese.
I do not like cheese.
I like cheese, but I’d rather eat biscuits.
I used to like cheese.
![Page 17: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/17.jpg)
SarcasmWell, that’s great.
Vs.
Well, that’s great.
![Page 18: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/18.jpg)
ModelsLexical - word frequencies
Supervised - Naive Bayes, SVM
Unsupervised - Latent Dirichlet Allocation, word2vec
Neural Networks
![Page 19: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/19.jpg)
LevelsDocument
Paragraph
Sentence
Entity
![Page 20: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/20.jpg)
Google Natural Language APIParser + NN
Sentence and entity level
Sentiment score and magnitude
Cheap compared to writing your own
Other services are available
Build your own (can use SyntaxNet if you want)
![Page 21: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/21.jpg)
Ambiguous Results
![Page 22: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/22.jpg)
MCS vs Sentiment
![Page 23: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/23.jpg)
Exit Scores vs Sentiment
![Page 24: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/24.jpg)
Entity Sentiment
![Page 25: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/25.jpg)
Silent Weeping
![Page 26: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/26.jpg)
Please stop nowSentiment analysis is a reasonable indicator of customer satisfaction
Sentiment analysis is slightly more accurate than astrology
Probably best to train your own model for entity sentiment
Refunds available from the organisers
![Page 27: Using Sentiment Analysis to Fill in the Gaps in User Surveys€¦ · Build your own (can use SyntaxNet if you want) Ambiguous Results. MCS vs Sentiment. Exit Scores vs Sentiment](https://reader033.vdocuments.us/reader033/viewer/2022052021/6036400f79bb352d0479a620/html5/thumbnails/27.jpg)
Questions?Please don’t.
Nobody wants this to go on any longer than it has to.
@nomiddlename