Visualizations, Mashups and Dashboards

University of Illinois at Urbana-Champaign

Outline

• Visualization

• SEASR as a service

• Text App: Airport Data

• Text App: Correlation & Ngram Viewer

• Text App: ProseVis

• Attendee Project Work

Visualization Components

• JavaScript – GIS: GoogleMaps– Temporal: Simile– InfoVis: Protovis – Parallel Coordinates, Link Node, Arcs– InfoVis: D3 – Line Graphs, Tag Cloud

• GWT – Dendogram– Table Viewer

• Flash – InfoVis: Flare

• Applets– Data Mining Results: Decision Tree, Naïve Bayes, Rule

Association

• HTML– Reports

Mashups & DashboardsHTML driven dashboard• Connect to DB to

make the query• 6 analytical

services (flows)– 4 different tag

cloud views– 2 entity

relationship views

Example: SEASR, Protovis

Usage of Service from HTML

• Simple

<iframe src=http://meandre-dev.leo.ncsa.edu:10000/service/post?query=myquery width="100%" height="250”/>

More Complex

<script>var servicePorts = [ 10000, 10001, 10002, 10003, 10004, 10005 ];var iframes = ["frame1", "frame2", "frame3", "frame4", "frame5", "frame6”];function searchService() {

var searchterm = document.getElementById("searchterm").value;

for (var i in servicePorts)window.frames[iframes[i]].location.href =

"http://meandre-dev.leo.ncsa.edu:" + servicePorts[i] + "/service/post?query=" + searchterm;}</script>…<iframe src="about:blank" name="frame1" id="frame1" width="100%" height="250”/>

Correlation-Ngram Viewer

Correlation-Ngram Viewer

• Pearson Correlation Algorithm

• 5 Flows run as ZigZag flows

Correlation-Ngram Viewer

• Words whose frequencies correlate with each other in a large dataset often turn out to have contextual associations as well

• Corrected version of the Google ngrams dataset (the English corpus)

• http://leovip029.ncsa.uiuc.edu/Correlation/

Correlation-Ngram Viewer

• New version of the Google ngrams viewer (for 1 grams)– addresses case-sensitivity

– period spellings

• Normalized to British practice, and period spellings have been normalized to modern practice (“honor” -> “honour,” “enow” -> “enough,” “publick” -> “public”)

– past-tense syncope (' d)

• based on 3-gram dataset to correct the treatment of syncopated past tenses (occurrences of “resolv’d” are now folded into the occurrences of “resolved”)

– f/s substitution as well as other OCR issues

• Searches within already stored correlation results (using Pearson) results for top 10K ngrams

• Computes correlation (using Pearson) results for given word against top 1K ngrams

Correlation-Ngram Viewer Flows

• Service Search Pearson for Top 10k Google Ngrams

– Uses the web service components to accept a query and executes a SQL query with this term against the table that stores the calculated Pearson results.

• Service Compute Pearson for Top 10k Google Ngrams

– Uses the web service components to accept a query and executes a SQL query to calculate the correlation of this search term against the top 10,000 ngrams.

• Service Query Pearson Results

– Shows the set of ngrams that have the highest Pearson score within the 10,000 result set.

• Service Google Ngrams Multi Line Chart

– Uses the web service components to accept a query, execute the SQL query with this term(s) to retrieve the frequency data and create a visualization showing the frequency graph for each ngram. The graph has a smoothing moving average that can be user adjusted.

• Service Google Ngrams List

– Uses the web service components to accept a query, execute the SQL query to identify the term(s) included in the graph and their total frequency counts. The list displays normalization for case, OCR errors from the transformation rules, and period spellings normalized to British.

Service Google Ngrams Multi Line Chart

Service Query

• http://leovip029.ncsa.uiuc.edu:10013/service/post?query=“dark”&query=“bright”&min_year=1700&max_year=1899

HTML code calling Meandre Service

<script>

var servicePorts = [10012,10013,10014,10015];

var iframes = [ "frame0","frame1","frame2","frame3" ];

function searchService() {

var searchterm = document.getElementById("searchterm").value;

searchterm = searchterm.replace(/,,/g, ","); //remove extra commas

searchterm = searchterm.replace(/ /g, "");

searchterm = searchterm.toLowerCase();

var gsearchterm = searchterm.replace(/,/g, "%22&query=%22");

searchterm = searchterm.replace(/,/g, "%22,%22");

window.frames[iframes[0]].location.href = "http://leovip029.ncsa.uiuc.edu:" + servicePorts[0] + "/service/post?query=%22" + searchterm + "%22&min_year=1700&max_year=1899";

window.frames[iframes[1]].location.href = "http://leovip029.ncsa.uiuc.edu:" + servicePorts[1] + "/service/post?query=%22" + gsearchterm + "%22&min_year=1700&max_year=1899";

}

</script>

ProseVis

ProseVis• Visualization tool

• Seeks to identify other features than the "word" to analyze texts

• Features comprise sound – parts-of-speech– accent– phoneme– stress– tone– break index

• http://tclement.ischool.utexas.edu/ProseVis/

ProseVis: Output Data

ProseVis: Data Processing Flow

ProseVis: Prosody Features

“Matisse” “Picasso”

Prosody Comparisons

ProseVis: Prosody Comparisons

Tender Buttons

ProseVis

• Calls Meandre service for data processing– Identify Prosody Features of Document

– Compare Prosody Features of Multiple Documents

Other Noteworthy Data Cleaning Tools• Data Wrangler

– http://vis.stanford.edu/wrangler/

– “Wrangler is an interactive tool for data cleaning and transformation.Spend less time formatting and more time analyzing your data.”

• Google Refine

– http://code.google.com/p/google-refine/

– “is a power tool for working with messy data, cleaning it up, transforming it from one format into another”

Demonstration

• Visualizations

• Text App: Correlation & Ngram Viewer

• Text App: ProseVis

Attendee Project Plan

• Study/Project Title

• Team Members and their Affiliation

• Procedural Outline of Study/Project– Research Question/Purpose of Study

– Data Sources

– Analysis Tools

• Activity Timeline or Milestones

• Report or Project Outcome(s)

• Ideas on what your team needs from SEASR staff to help you achieve your goal.

Learning Exercises

• Create a web page that accesses SEASR services

– Customize an HTML template that we have constructed to leverage 2-3 flows that you develop and execute

Discussion Questions

• What types of mashups or dashboards would you like to see?