introduction to text analysis
DESCRIPTION
Slides from my text analysis workshop at the MLA, a part of "Getting Started in the Digital Humanities with Help from DHCommons."TRANSCRIPT
![Page 1: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/1.jpg)
Introduction to Text Analysis
MLA Annual ConventionGetting Started in the Digital Humanities
January 9, 2014
Lauren F. KleinGeorgia Institute of Technology
[email protected]@laurenfklein
![Page 2: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/2.jpg)
Introduction to Text Analysis
• What is text analysis?
![Page 3: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/3.jpg)
Introduction to Text Analysis
• What is text analysis?• Why should you use it?
![Page 4: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/4.jpg)
Introduction to Text Analysis
• What is text analysis?• Why should you use it?• How do you use it?– Examples– Tools
![Page 5: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/5.jpg)
What is Text Analysis?
![Page 6: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/6.jpg)
What is Text Analysis?According to Geoffrey Rockwell:
• “Text analysis systems can search large texts quickly. They do this by preparing electronic indexes to the text so that the computer does not have to read through the entire text. When finding words can be done so quickly that it is "interactive", it changes how you can work with the text - you can serendipitously explore without being frustrated by the slowness of the search process.
• “Text analysis systems can conduct complex searches. Text analysis systems will often allow you to search for lists of words or for complex patterns of words. For example you can search for the co-occurrence of two words.
• “Text analysis systems can present the results in ways that suit the study of texts. Text analysis systems can display the results in a number of ways; for example, a Keyword In Context display shows you all the occurrences of the found word with one line of context.”
http://tada.mcmaster.ca/Main/WhatTA
![Page 7: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/7.jpg)
![Page 8: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/8.jpg)
http://www.wordle.net
![Page 9: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/9.jpg)
![Page 10: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/10.jpg)
http://www.wordle.net
![Page 11: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/11.jpg)
Mark Hansen and Ben Rubin Movable Type
![Page 12: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/12.jpg)
Why Use Text Analysis?
![Page 13: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/13.jpg)
Why Use Text Analysis?Geoff Rockwell, again:
• “Text analysis tools aide the interpreter asking questions of electronic texts.”• “Text analysis practices encourage reflection on the questions asked and
formalization of queries.”• “Text analysis is a way of targeting rereading that tests intuitions.”
![Page 14: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/14.jpg)
Why Use Text Analysis?Geoff Rockwell, again:
• “Text analysis tools aide the interpreter asking questions of electronic texts.”• “Text analysis practices encourage reflection on the questions asked and
formalization of queries.”• “Text analysis is a way of targeting rereading that tests intuitions.”
![Page 15: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/15.jpg)
Why Use Text Analysis?Geoff Rockwell, again:
• “Text analysis tools aide the interpreter asking questions of electronic texts.”• “Text analysis practices encourage reflection on the questions asked and
formalization of queries.”• “Text analysis is a way of targeting rereading that tests intuitions.”
Ted Underwood:• “Proving a literary thesis with statistical analysis is often like cracking a nut with a
jackhammer. You can do it: but the results are not necessarily better than you would get by hand.”
![Page 16: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/16.jpg)
Why Use Text Analysis?Geoff Rockwell, again:
• “Text analysis tools aide the interpreter asking questions of electronic texts.”• “Text analysis practices encourage reflection on the questions asked and
formalization of queries.”• “Text analysis is a way of targeting rereading that tests intuitions.”
Ted Underwood:• “Proving a literary thesis with statistical analysis is often like cracking a nut with a
jackhammer. You can do it: but the results are not necessarily better than you would get by hand.”
What I think (in the spirit of Movable Type):• Text analysis as “a way to tell a new story.”
![Page 17: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/17.jpg)
How to Use Text Analysis?
![Page 18: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/18.jpg)
Ben Blatt, http://www.slate.com/articles/arts/culturebox/2013/11/hunger_games_catching_fire_a_textual_analysis_of_suzanne_collins_novels.html
![Page 19: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/19.jpg)
Sarah Lohman, http://www.fourpoundsflour.com/the-gallery-data-visualization-of-a-timeline-of-taste/
![Page 20: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/20.jpg)
Daniel, http://lkleincourses.lmc.gatech.edu/dh12/2012/02/22/the-role-of-senses-in-a-study-in-scarlet/
![Page 21: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/21.jpg)
Ted Underwood and Jordan Sellers, http://journalofdigitalhumanities.org/1-2/the-emergence-of-literary-diction-by-ted-underwood-and-jordan-sellers/
![Page 22: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/22.jpg)
Rob Nelson, http://dsl.richmond.edu/dispatch/
![Page 23: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/23.jpg)
Matt Jockers, http://www.nbcnews.com/technology/data-mining-classics-makes-beautiful-science-954577
![Page 24: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/24.jpg)
Matt Jockers, from Macroanalysis (Univ. of Illinois Press, 2013)
![Page 25: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/25.jpg)
Lauren Klein, from “The Image of Absence” (American Literature 85.4)
![Page 26: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/26.jpg)
Tools for Text Analysis
• Wordle • Google Ngram Viewer • IBM Many Eyes • Voyant • MONK (requires institutional access)• MALLET• Stanford’s Natural Language Processing Toolkit• R
![Page 27: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/27.jpg)
Google Ngram Viewer
Google Ngram Viewerhttps://books.google.com/ngrams
![Page 28: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/28.jpg)
IBM Many Eyes
Many Eyeshttp://www-958.ibm.com/software/analytics/manyeyes/
![Page 29: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/29.jpg)
Voyant Tools
Voyant Toolshttp://voyant-tools.org/
![Page 30: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/30.jpg)
MALLET
MALLEThttp://mallet.cs.umass.edu/
![Page 31: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/31.jpg)
Stanford NLP Toolkit
Stanford NLP Toolkithttp://nlp.stanford.edu/downloads/
![Page 32: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/32.jpg)
R Programming Language
R (programming language)http://www.r-project.org/
![Page 33: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/33.jpg)
TAPoR
TAPoR (Text Analysis PoRtal)http://tapor.ca/
![Page 34: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/34.jpg)
More Lists of Tools
• http://toolingup.stanford.edu/?page_id=367
• http://guides.library.upenn.edu/dhtextanalysis
• http://dirt.projectbamboo.org/categories/text-mining
![Page 35: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/35.jpg)
Many Eyes Demo
http://lkle.in/1bTr2eT
![Page 36: Introduction to Text Analysis](https://reader033.vdocuments.us/reader033/viewer/2022061202/547bae685806b512408b4643/html5/thumbnails/36.jpg)
Voyant Tools Demo
http://lkle.in/1e186zN