hadoop meetup 2014
DESCRIPTION
Hadoop Users Group Pittsburgh discusses D3.jsTRANSCRIPT
D3, VISUALIZATIONS, N’ATPatrick M. Dudas
@pdudders
Who I am..
Data Visualization (Meetup)
Art
Data
Coding
UI/UX
Github: https://github.com/dudaspm/Pittsburgh-Data-Visualization
Data Mining Process
Edward Tufte
“With most visualizations and graphics, the main focus is to take multiple dimensions of information and project it on to a two-dimension plot”
Infographic
Infographic, Model, or Visualization
Model
http://fold.it/portal/info/about - University of Washington
Visualization
7
When Graphics Go Bad
Challenger January 28, 1986
8
Can Anyone Spot the Problem?
9
How About Now?
#1 Asked Question
I have “this data.” What visualization should I use?
http://www.datavis.ca/gallery/evil-pies.php
Data123443103523229
Data0.06383
0.180851 0.228723 0.053191 0.18617 0.12234
0.117021 0.047872
Normalize
Visualization?
Steamgraph, Sparklines, and Treemaps
Hive Plots, Star Plots, and Parallel Coordinates
Back to the #1 Asked Question I have “this data.”
What visualization should I use? Ask your users or experts
How to Get to D3
HTMLJavascript/JQuery
XML SVG D3
HTML
<html>
<head>
<title>I am metadata</title>
<script src="http://d3js.org/d3.v3.min.js"></script>
</head>
<body>
the body of HTML
</body>
</html>
http://boxnumbertwo.com/D3Simple/1.0/example1.html
DIV
<html><head>
<title>I am metadata</title><script src="../d3/d3.min.js"></script>
</head><body>
<div id="visualization_here"></div></body>
</html>
Not Well Formatted HTML
<html><head>
<title>I am a webpage</title><script src="../d3/d3.min.js"></script>
</header><ody>
<div id="visualization_here"></div></body
</html>
XML (Must Be Well Formatted)<?xml version="1.0"?><note> <to>Dave</to> <from>Jamie</from> <heading>Reminder</heading> <message>Don't forget me this
weekend!</message></note>
SVG<svg xmlns="http://www.w3.org/2000/svg" version="1.1" width="400px"
height="400px">
<polygon points="200,10 250,190 160,210" style="fill:lime;stroke-width:1"/>
</svg>
SVG vs. Canvas or Vector vs. Raster
Canvas = Pixels Raster
SVG Points Lines Polygons
Vector
D3 - http://d3js.org/
Data Driven Documents Enter, Update, Exit Interactions Transitions
Big List of Examples (1900) http://christopheviau.com/d3list/index.html
D3.js
SVG, CSS, and HTML update
update your data enter
enter svg objects using that data (example6)OR
exit remove svg using that data (example7)
mouse events (example8) transitions (example9)
smoothly change data example10 – adding affordances and a little fun
Simple Exam of D3
var circleData = [4, 8, 15, 16, 23, 42];
http://boxnumbertwo.com/D3Simple/1.0/example6.html
Bertin’s Retinal/Visual Variable
Adding Some Aesthetics
var color = d3.scale.category10();newCircles.enter().append("circle")
.attr("cx", function(d) { return d*10; })
.attr("cy", function(d) { return d*10; })
.attr("r", function(d) { return d; })
.style("fill", function(d) { return color(d); });
http://boxnumbertwo.com/D3Simple/1.0/example7.html https://github.com/mbostock/d3/wiki/Ordinal-Scales#wiki-category10http://boxnumbertwo.com/D3Simple/NetworkX/1.6/index.php?sizeOfGraph=60&ProbabilityOfEdge=.03
Enter, Update, Exit
http://boxnumbertwo.com/D3Simple/1.0/example7.html
Change Blindness
Minimizing changes in a scene to make visual changes more noticeable.
Sometimes We Miss the Change Change Blindness - phenomenon that
occurs when a person viewing a visual scene apparently fails to detect large changes in the scene (Wikipedia)
http://www.csc.ncsu.edu/faculty/healey/PP/movies/Dinner.gif
http://www.csc.ncsu.edu/faculty/healey/PP/movies/Airplane.gif
http://www.csc.ncsu.edu/faculty/healey/PP/movies/Chopper_Truck.gif
Smooooooth Transitions
newerCircles.exit().transition().duration(3000).attr("r", 0).remove();
http://boxnumbertwo.com/D3Simple/1.0/example9.hthttp://www.boxnumbertwo.com/PitterPatter/1.2/ml
32
Don Norman’s Design of Everyday Things
Builds on the notion of affordances Affordance – "refers to the perceived and
actual properties of the thing, primarily those fundamental properties that determine just how the thing could possibly be used … Affordances provide strong clues to the operations of things" (Norman, 1988) This can include objects or words/numbers!
Norman, D. A. (1988). The design of everyday things. New York, Doubleday
Affordance
34
Affordance Scenario 1
http://www.baddesigns.com
35
Trapped between the doors! She was walking from one building to the other
with a co-worker. They pulled the handles that opened the doors and went down the walkway. Upon reaching the other end they again pulled the handles, but the doors wouldn't budge. Assuming the doors were locked, they returned to the doors they originally opened to enter the walkway. But when they tried to pull open these doors, they wouldn't open either. They were trapped in the walkway between the two buildings!
My friend and her co-worker spent the next few minutes trying to signal to people though the windows in the buildings, but the people they signaled seemed strangely reluctant to come to the rescue. Finally, after trying the doors again, they discovered they needed to push the doors rather than pull them.
Let’s Add Some Affordances
newCircles.enter().append("circle").attr("cx", function(d) { return d*10; }).attr("cy", function(d) { return d*10; }).attr("r", function(d) { return d; }).style("fill", function(d) { return color(d); }).style("stroke-width", 0).style("stroke", "black").style("cursor", "pointer")// on mouse-over, change the border of the given circle to 2.on("mouseover", function() {d3.select(this).style("stroke-width", 2)})// on mouse-out, change the border back to the original (0).on("mouseout", function() {d3.select(this).style("stroke-width", 0)})
http://boxnumbertwo.com/D3Simple/1.0/example10.html
Heatmap
38
Heat Map
39
Heat Map
Matrix + Layers + Color = Heatmap
redM.push(0,0,0,0,0,1,1,1,1,1,0,0,0,0,0,0)redM.push(0,0,0,0,1,1,1,1,1,1,1,1,1,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,1,1,1,1,0,0,0,0,0,0)redM.push(0,0,0,0,0,1,0,1,1,0,1,0,0,0,0,0)redM.push(0,0,0,0,0,1,1,1,1,1,1,0,0,0,0,0)redM.push(0,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0)redM.push(0,0,0,0,1,1,1,0,0,1,1,1,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)redM.push(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0);http://boxnumbertwo.com/D3Simple/mario/standing.htm
l
Chaining Transitions
var t0 = vis.transition().ease('quad-in').selectAll("rect");t0.style("fill", function(d,i) { return marioColors[d[1][1]]; });var t1 = t0.transition();t1.style("fill", function(d,i) { return marioColors[d[2][1]]; });
http://boxnumbertwo.com/D3Simple/mario/marioMoving.html
Javascript Timers + D3 = Animationvar cloud_scale_1 = .8, cloud_translate_1 = [700,100];setInterval(function(){moveCloud_1();},25);
var boo_2 = 0;var cloud_scale_2 = .6, cloud_translate_2 = [0,300];setInterval(function(){moveCloud_2();},30);
var boo_3 = 0;var cloud_scale_3 = .7, cloud_translate_3 = [300,0];setInterval(function(){moveCloud_3();},40);
var boo_4 = 0;var cloud_scale_4 = .9, cloud_translate_4 = [500,200];setInterval(function(){moveCloud_4();},55);
http://boxnumbertwo.com/D3Simple/6.1/
43 Dynamic Twitter Network Analysis
44
Overall Platform
45
Ubiquitous Platform
Built upon HTML5 standards Does not require any additional software
to be downloaded All major browsers:
IE, Firefox, Chrome, Safari, Opera, etc. Supports all UTF-8 and Unicode
character types
46
Networks
James tweets: “Just saw @Dannie at the #Pittsburgh #Library. Checked out this book: http://bit.ly/U8AGsc” Username -> Username:
James -> Dannie Username -> Hashtag:
James -> Pittsburgh and James -> Library Hashtag -> Hashtag:
Pittsburgh -> Library Username -> URL:
James -> http://bit.ly/U8AGsc
47
DTNA Layout
48
Saliency and Network
limited to 100 nodes clustering algorithm
differentiated by color coding bias to inner-cluster
strength to inter-cluster strength
edge directionality bias inner-cluster vs.inter-cluster
log-based degree centrality highlight 1st degree
connections
Blondel, V. D., Guillaume, J. L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008.
49
Real-Time Network
50
Dynamic Query and Layout
51
Export Network
Export: UCINET Gephi Palantir Pajek
52
TweetViewer
53
TweetViewer Timeline
54
Sentiment Analysis
F. Å. Nielsen, "A new ANEW: Evaluation of a word list for sentiment analysis in microblogs," Arxiv preprint arXiv:1103.2903, 2011.
55
Sentiment Analysis
Negative Words:• Sad• Murder• Disgust• WTF• Kill•
Positive Words:• Praise• Love• Dear• kind• Hope•
http://www.youtube.com/watch?v=ao6V_5sK3-A
Thank you!
Contact: [email protected] Github:
https://github.com/dudaspm/Pittsburgh-Data-Visualization