workshop: data visualization for corpus linguistics via shiny framework
TRANSCRIPT
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Data Visualization for Corpus Linguistics:Shiny Framework
Olga Scrivner
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
What You Will Learn
1. Introduction to Visual Analytics
2. Reactive Web Framework
3. Shiny Application
4. Practice
https://languagevariationsuite.wordpress.com/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Data Analytics
“Analytics is the critical technologyneeded to bring value out of data”
(Anonymous)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Visual Analytics
“The science of analytical reasoningfacilitated by visual interactive interfaces”
(Thomas and Cook, 2005)
“Visual analytics integrates new computational andtheory-based tools with innovative interactive techniquesand visual representations to enable human-information
discourse” (Thomas and Cook, 2005)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Visual Analytics and Linguistics
“ Language is a system of relatively arbitrary symbols”(Baker-Shenk and Cokely, 1980)
http://www.tableaufit.com/tableau-conference-linguistics-data-visualization/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Visual Analytics (and Corpus Linguistics)
1. Exploration (Queries)
2. Comprehension (Analysis)
3. Communication (Presentation)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Corpus Linguistics Visualization
http://scimaps.org/home.html
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Visualization Tools
Challenges:
I Many tools are built for non-linguistic research
I Limited preprocessing
I Visualization may not fit linguistic data
Solution: Reactive Framework
I Interactive and web-based
I Research-driven customization
I Build your own tool!
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Visualization Tools
Challenges:
I Many tools are built for non-linguistic research
I Limited preprocessing
I Visualization may not fit linguistic data
Solution: Reactive Framework
I Interactive and web-based
I Research-driven customization
I Build your own tool!
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Reactive Web Framework
“Reactive Systems are highly responsive, giving userseffective interactive feedback”
http://www.reactivemanifesto.org/
http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Reactive Architecture
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Reactive Framework and Data Science
“The impact of data scientists’ work depends on how wellothers can understand their insights to take further actions”
Benefit 1: Interactive display and manipulation of data
Benefit 2: No installation required
Benefit 3: Easy to develop and share with clients andproject teams
Benefit 4: Open source library
http://datascience.ibm.com/blog/shiny-a-data-scientist-best-friend/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Application
1. Shiny is an R package for building interactive webapplications
2. Open-Sourced by RStudio 11/2012 on CRAN
3. Uses web sockets (new HTTP):
I Interactive communication sessions between the user’sbrowser and a server without having to poll the serverfor a reply
4. Entirely extensible - custom input/output
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Library
http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Gallery - Get Inspired
https://www.rstudio.com/products/shiny/shiny-user-showcase/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Interactive Text Mining Suite (Scrivner et al.2016)
1. Web application for text processing and mining
2. Interactive natural language processing techniques
I Wordstops, stemming, text-preprocessing
3. High customization
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Stopwords
Stopwords (e.g. the, and):
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Manual Removal of Stopwords
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Stemming
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Word Cloud Representation
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Customization
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Format Flexibility
Example: Using Google Books API
Current limitation is 40 books
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Workshop Materials
1. Rstudio
2. R
3. Shiny library
4. Materials:
https:
//languagevariationsuite.wordpress.com/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
R software
R is a free software for data analysis, text mining andvisualization.
To install R on Window:
1. Download the binary file for Rhttps://cran.r-project.org/bin/windows/base/
R-3.3.1-win.exe
2. Open the downloaded .exe file and Install R
To install R on Mac:
1. Download the appropriate version of .pkg filehttps://cran.r-project.org/bin/macosx/
2. Open the downloaded .pkg file and Install R
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
R Studio
RStudio is a free user interface for R.
1. Install the appropriate RStudio version https:
//www.rstudio.com/products/rstudio/download/
2. Run it to install R-studio
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
R Studio Structure
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Installing Packages
In your bottom left window - go to Packages
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Selecting Packages - shiny
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Creating Scripts
Create R File: File → New File → R Script
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Execution - RUN
To execute - click run
In the script (top left window) type:
library(shiny)
Click Run (your cursor can be at the beginning or at the endof the line)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Demo
runExample(“01 hello”)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Demo
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Shiny Demo
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
UI.R
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Reactive Input/Output
I Input - things user can toggle
I Output - R objects that user can see, often depend oninputs
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Closing App
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Step 1 - Create New Web Shiny App
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Step 2 - Run App
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
UI
shinyUI( fluidPage(
titlePanel(“Old Faithful Geyser Data”),
sidebarLayout(
sidebarPanel(
sliderInput(“bins”,
“Number of bins:”,
min = 1,
max = 50,
value = 30)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
UI
shinyUI( fluidPage(
titlePanel(“Old Faithful Geyser Data”),
sidebarLayout(
sidebarPanel(
sliderInput(“bins”,
“Number of bins:”,
min = 1,
max = 50,
value = 30)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Modifying UI - Practice
I Change slider’s label:Number of bins → Choose a number
I Change slider’s values: max and value
I Save
I RunApp
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
HTML
The UI script simply creates HTML
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
HTML
HTML tags:
http:
//shiny.rstudio.com/articles/tag-glossary.html
I h1() = header1
I br() = line break
I p() = paragraph
I hr() = line
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Adding HTML Tags to UI.R - Practice
shinyUI( fluidPage(
titlePanel(“My Title”),
h3(“My subtitle”),
p(“This is my first app!”),
br(),
hr(),
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Layout
http:
//shiny.rstudio.com/articles/layout-guide.html
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
mainPanel
mainPanel( plotOutput(”distPlot”))
Let’s add 3 tab panels: Plot, Summary, Table
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
tabsetPanel
mainPanel(
tabsetPanel(
tabPanel(“Plot”, plotOutput(“distPlot”)),
tabPanel(“Summary”),
tabPanel(“Table”)
)
)
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Input Data
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Input Data
1. Blog https:
//languagevariationsuite.wordpress.com/
2. Download csv file - movie metadata.csv
3. Place this file into the directory myshiny
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
New Input in UI.R
I Let’s look at the fileInput() function from ShinyReference page
I Type in your browser: fileInput Shiny
I or go to the Shiny Reference page -https://shiny.rstudio.com/reference/shiny/
latest/fileInput.html
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
New Input in UI.R
I Scroll down to Examples
I We will add fileInput function inside sidebarPanel
I Copy fileInput function
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
New Input in UI.R
I Paste it after slider
NB: commas are important!
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Reactive Function in Sever.R
Reactive - it changes every time the user uploads new data
I Create a function that reads csv file
I myfile() is a function that will read and return csv files
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Render - Reactive Data
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Function summary in Server.R
output$summary <- renderPrint({
summary(myfile())
})
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
New Output Function table in Server.R
output$table <- renderDataTable({
myfile()
})
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Linking summary and table with UI.R
tabsetPanel(
tabPanel("Plot", plotOutput("distPlot")),
tabPanel("Summary", verbatimTextOutput("summary")),
tabPanel("Table", dataTableOutput("table"))
)
RunApp
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Linking summary and table with UI.R
tabsetPanel(
tabPanel("Plot", plotOutput("distPlot")),
tabPanel("Summary", verbatimTextOutput("summary")),
tabPanel("Table", dataTableOutput("table"))
)
RunApp
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Testing CSV Upload
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
One more IF-Statement - Server.R
We want to do a histogram for csv file:
if (condition) {do..} else {do...}
output$distPlot <- renderPlot({
if (is.null(input$file1)) {
....
hist(x, breaks = bins, col = ’darkgray’, border =
’white’)
}
else{
x <- myfile()$budget
bins <- seq(min(x), max(x), length.out = input$bins
+ 1)
hist(x, breaks = bins, col=’red’,
main = ’My New Histogram’)
}
})
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
One more IF-Statement - Server.R
We want to do a histogram for csv file:
if (condition) {do..} else {do...}output$distPlot <- renderPlot({
if (is.null(input$file1)) {
....
hist(x, breaks = bins, col = ’darkgray’, border =
’white’)
}
else{
x <- myfile()$budget
bins <- seq(min(x), max(x), length.out = input$bins
+ 1)
hist(x, breaks = bins, col=’red’,
main = ’My New Histogram’)
}
})
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
One more IF-Statement - Server.R
We want to do a histogram for csv file:
if (condition) {do..} else {do...}output$distPlot <- renderPlot({
if (is.null(input$file1)) {
....
hist(x, breaks = bins, col = ’darkgray’, border =
’white’)
}
else{
x <- myfile()$budget
bins <- seq(min(x), max(x), length.out = input$bins
+ 1)
hist(x, breaks = bins, col=’red’,
main = ’My New Histogram’)
}
})
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
RunApp
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Server - Recap
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Deployment Options
1. Share server.r and ui.r
2. Host on shinyapps.io
3. Host on Shiny server
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Deploy with shinyapps.io
I www.shinyapps.io
I sign up for an account.
I Publish Application button in RStudio and followinstructions
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Deploy with shinyapps.io
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
My shinyapps.io
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
My shinyapps.io
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Future of Visualization Tools: CNS
Cyberinfrastructure for Network Science Center at IndianaUniversity - http://cns.iu.edu/
Our current project on Shiny Framework (team OlgaScrivner and Jivitesh Poojary)
I Build Shiny templates for Data Visualization: stage I
https:
//github.com/Jivitesh-Poojary/CNS-Shiny-Apps
I Create user-friendly customizable Shiny dashboards:stage II
I Develop preprocessing plugins for Shiny templates:stage III
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Useful resources
1. Shiny official tutorial -http://shiny.rstudio.com/tutorial
2. Cheat sheet - http://shiny.rstudio.com/images/shiny-cheatsheet.pdf
3. Publish your app free - http://www.shinyapps.io
4. Examples -http://www.showmeshiny.com/
5. Tutorial by Dean Attali - http://deanattali.com/blog/building-shiny-apps-tutorial/
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Thank you!
My email: [email protected]
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
References
Borner, Katy. 2015. Atlas of Knowledge: Anyone Can Map. TheMIT PressCollins, Christopher. 2012. Bridging the Linguistic VisualizationDivide. LingVis/UNCLH WorkshopGrolemund, Garrett. 2017. How to start with Shiny, Part 1.Workshop
Data Visualizationfor CorpusLinguistics:
Shiny Framework
Olga Scrivner
Visual Analytics
ReactiveFramework
Shiny App
Practice
Credits
http:
//deanattali.com/blog/building-shiny-apps-tutorial/
http://scimaps.org/mapdetail/stream_of_scientific_128
https://github.com/IBMDataScience/dsx-shiny-apps
http://www.slideshare.net/SarahAerni/
data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-science-from-pivotal-open-source-hub-meetup
http://www.unixstickers.com/image/data/stickers/
react/badge/React-JS.sh.png
https://github.com/rstudio/shiny/issues/250
http://www.slideshare.net/ilio-catallo/
spring-mvc-the-basics