technology frontiers: text, sentiment, and sense by seth grimes of alta plana corporation -...
DESCRIPTION
A basic definition: Text analytics transforms text-sourced information into data to help you generate insights that fuel better-informed business decision-making. Methods are applied to online and social information, as well as enterprise feedback, to complement and extend traditional and emerging research methods. Text analytics is the leading opinion mining technique, evolving to link emotion and intent signals to behaviors, profiles, and transactions. If text analytics isn’t part of your data toolkit, it should be; if you’re already exploiting text analytics, you’ll want to stay on top of developments. Seth Grimes, in this What’s Next talk, will tell you how.TRANSCRIPT
![Page 1: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/1.jpg)
Technology Frontiers: Text, Sentiment, and Sense
Seth Grimes@sethgrimes
![Page 2: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/2.jpg)
A Sensemaking Story
New York Times,September 30, 2012
![Page 3: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/3.jpg)
New York Times,September 8, 1957
Valium: A Chain of Connections
![Page 4: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/4.jpg)
Natural Language Processing
By H.P. Luhn, inIBM Journal,April, 1958
http://altaplana.com/ibm-luhn58-LiteratureAbstracts.pdf
![Page 5: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/5.jpg)
Modelling Text
“Statistical information derived from word frequency and distribution is used by the machine to compute a relative measure of significance, first for individual words and then for sentences. Sentences scoring highest in significance are extracted and printed out to become the auto-abstract.”
-- H.P. Luhn, The Automatic Creation of Literature Abstracts, IBM Journal, 1958.
Luhn’s analysis of Messengers of the Nervous System, a Scientific American article http://wordle.net,
applied to the NY Times article
![Page 6: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/6.jpg)
New York Times,September 8, 1957
Luhn’s Example
![Page 7: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/7.jpg)
Close Reading
![Page 8: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/8.jpg)
![Page 9: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/9.jpg)
Can Software Make the Connection?
Mark Lombardi, George W. Bush, Harken Energy and Jackson Stephens, c. 1979-90, Detail
![Page 10: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/10.jpg)
Insight from Connections
… via graphs, clusters, categories, and counts.
… by mining the full set of available data.
![Page 11: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/11.jpg)
http://techpresident.com/news/21618/politico-facebook-sentiment-analysis-bogus
Online & Social Change Everything
![Page 12: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/12.jpg)
(Accessible) Data Everywhere
![Page 13: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/13.jpg)
Lexical, syntactic, and semantic analysis discern features including relationships in source materials.
Features = entities, measure-value pairs, concepts, topics, events, sentiment, and more.
Text analytics may draw on:
• Lexicons & taxonomies.• Statistics.• Patterns.• Linguistics.• Machine learning.
Text Analytics
![Page 14: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/14.jpg)
How?
![Page 15: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/15.jpg)
From POS to Relationships
Understand parts of speech (POS), e.g. – <subject> <verb> <object> –to discern facts and relationships.
Semantic networks such as WordNet are a disambiguation asset.
![Page 16: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/16.jpg)
Clustered Clarity
Carrot2.(open source)
![Page 17: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/17.jpg)
Platforms and ecosystems.
APIs and services.
Text and content analytics --Discerns and extracts features including
relationships from source materials.
Features = entities, key-value pairs, concepts, topics, events, sentiment, etc.
Provide (for) BI on content-sourced data.
Data integration, record linkage, data fusion.
The Back End
![Page 18: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/18.jpg)
Content, Composites, Connections
![Page 19: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/19.jpg)
Content, Composites, Connections, 2
![Page 20: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/20.jpg)
Social Sources
![Page 21: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/21.jpg)
Sentiment Analysis
“Sentiment analysis is the task of identifying positive and negative opinions, emotions, and evaluations.”
-- Wilson, Wiebe & Hoffman, 2005, “Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis”
“Sentiment analysis or opinion mining is the computational study of opinions, sentiments and emotions expressed in text… An opinion on a feature f is a positive or negative view, attitude, emotion or appraisal on f from an opinion holder.”
-- Bing Liu, 2010, “Sentiment Analysis and Subjectivity,” in Handbook of Natural Language Processing
![Page 22: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/22.jpg)
Detection, Classification
![Page 23: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/23.jpg)
Beyond Polarity
![Page 24: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/24.jpg)
Intent Analysis
http://www.aiaioo.com/whitepapers/intention_analysis_use_cases.pdf
http://sentibet.com/
![Page 25: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/25.jpg)
Complications
Sentiment may be of interest at multiple levels.Corpus / data space, i.e., across multiple sources.Document.Statement / sentence.Entity / topic / concept.
Human language is noisy and chaotic!Jargon, slang, irony, ambiguity, anaphora, polysemy,
synonymy, etc.Context is key. Discourse analysis comes into play.
Must distinguish the sentiment holder from the object:“Geithner said the recession may worsen.”
![Page 26: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/26.jpg)
Audio including speech.Images.Video.
http://www.geekosystem.com/facebook-face-recognition/
http://www.sciencedirect.com/science/article/pii/S0167639312000118
http://flylib.com/books/en/2.495.1.54/1/
Beyond Text
![Page 27: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/27.jpg)
Sensemaking
“It is convenient to divide the entire information access process into two main components: information retrieval through searching and browsing, and analysis and synthesis of results. This broader process is often referred to in the literature as sensemaking. Sensemaking refers to an iterative process of formulating a conceptual representation from of a large volume of information. Search plays only one part in this process.”
-- Marti Hearst, 2009 http://searchuserinterfaces.com/
![Page 28: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/28.jpg)
Apply new tech to old needs, e.g., automated coding.
Select from and use all available data.
Marry social to profiles and surveys.
Factor in behaviors.
Interpret according to context and needs.
Understand intent to create situational predictive models.
Explore; experiment.
Suggestions
![Page 29: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/29.jpg)
Racing On
![Page 30: Technology Frontiers: Text, Sentiment, and Sense by Seth Grimes of Alta Plana Corporation - Presented at the Insight Innovation eXchange North America 2013](https://reader031.vdocuments.us/reader031/viewer/2022013100/54c66f264a795913618b4606/html5/thumbnails/30.jpg)
Technology Frontiers: Text, Sentiment, and Sense
Seth Grimes@sethgrimes