visualising multi dimensional data @ fifth elephant 2015

86
Amit Kapoor @amitkaps Visualising Multi - Dimensional Data x w z y

Upload: amit-kapoor

Post on 14-Aug-2015

908 views

Category:

Data & Analytics


1 download

TRANSCRIPT

Page 1: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Amit Kapoor@amitkaps

Visualising Multi - Dimensional Data

x

wz

y

Page 2: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Flatland A Romance in Many Dimensions

by Edwin Abbot (1884)

Square

Page 3: Visualising Multi Dimensional Data @ Fifth Elephant 2015

A square is but a line in 2d

eye

The Square

Page 4: Visualising Multi Dimensional Data @ Fifth Elephant 2015

The disappearing circle

The Sphere visits the 2d

flatland

The Sphere rising out of

2d space

The Sphere on the point of

vanishing

eye

Page 5: Visualising Multi Dimensional Data @ Fifth Elephant 2015

The square is a cube in 3d!

eye

The Square sees the world in

a new way!

Page 6: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Show, not Tell

Page 7: Visualising Multi Dimensional Data @ Fifth Elephant 2015

70%of the sensory

receptors are in the eyes

50%of the brain

used for visual processing

100msto get a sense of the visual

scene

Visual Wired Brain

Page 8: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Symbolic Abstraction

Visual Abstraction

Phenomena

Source: Bret Victor

Page 9: Visualising Multi Dimensional Data @ Fifth Elephant 2015

“Visualisation is the transformation of the

symbolic into geometric”

Page 10: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Small DataLarge Data

Big DataWide Data

Page 11: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Visualise Small DataArea Sales (Rs.)

North 5

East 25

West 15

South 20

Central 10

Page 12: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Area Sales

North 5

East 25

West 15

South 20

Central 10

AcquireData

Page 13: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Area Sales

North 5

East 25

West 15

South 20

Central 10

x y

1 5

2 25

3 15

4 20

5 10

x (C) = Areay (Q) = Sales

Parse Variables

AcquireData

Page 14: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Area Sales

North 5

East 25

West 15

South 20

Central 10

x y

1 5

2 25

3 15

4 20

5 10

x (C) = Areay (Q) = Sales

x y

20

40

100

140

180

Encode Shape & Select Scales

Parse Variables

AcquireData

x - position, y - barscale - 200 x 200

Page 15: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Area Sales

North 5

East 25

West 15

South 20

Central 10

x y

1 5

2 25

3 15

4 20

5 10

x (C) = Areay (Q) = Sales

x - position, y - barscale - 200 x 200

x y

20

40

100

140

180

Parse Variables

AcquireData

cartesian

Render with Coordinates

Encode Shape & Select Scales

Page 16: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Points Line Bar

Bar - Stacked Bar - Stagger CoordinatesSystem

Create Visualisations

Page 17: Visualising Multi Dimensional Data @ Fifth Elephant 2015

CoordinatesCartesian

x

y

Dot Plot Line Chart Column Chart

WaterfallStacked Column

Page 18: Visualising Multi Dimensional Data @ Fifth Elephant 2015

CoordinatesCartesian - Flip

Dot Plot Line Chart Bar Chart

CascadeStacked Bar

y

x

Page 19: Visualising Multi Dimensional Data @ Fifth Elephant 2015

PolarCoordinate - X

x = θy = r

Marked Radar Line Radar CoxComb

Polar WaterfallBullseye

Page 20: Visualising Multi Dimensional Data @ Fifth Elephant 2015

PolarCoordinate - Y

x = ry = θ

Target Line Track Wind Rose

Polar CascadePie Chart

Page 21: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Viz Process (Small Data)Acquire Data

Encode Shape

Select Scales

Render Coordinates

Parse Variables

Page 22: Visualising Multi Dimensional Data @ Fifth Elephant 2015

gadfly

bokeh

ggplot2

matplotlib

graphics

Page 23: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Small DataLarge Data

Big DataWide Data

Page 24: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Visualise Large Data

Page 25: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Pincode Map

Scatter plot,play with alpha to

show density

But what if I want to show geographic

nature of pincode?

Page 26: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Pincode+ Map

Exploration of large data is iterative!

Refine Data (Filter, Transform)

Page 27: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Viz Process (Large Data)

Acquire Data

Encode Shape

Select Scales

Render Coordinates

Parse Variables

Refine Data

Page 28: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Small DataLarge Data

Big DataWide Data

Page 29: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Visualise Big Data

Comparable to theNumber of Pixels

on my MacBook Air

Data

Page 30: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Sample

Sampling can be effective (with overweighting

unusual values)

Require multiple plots or careful

tuning parameters

Page 31: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Sample

Model

Models are great as they scale nicely.

But, visualisation is required as

“I don’t know, what I don’t know.”

Page 32: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Sample

ModelBinning

Binning can solve a lot of these challenges

“Bin - Summarize - Smooth: A framework for visualising big data” - Hadley Wickam (2013)

“imMens: Real-time Visual Querying of Big Data” - Liu,

Jiang, Heer (2013)

Page 33: Visualising Multi Dimensional Data @ Fifth Elephant 2015
Page 34: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Tools Matter

Defaults Matter

Page 35: Visualising Multi Dimensional Data @ Fifth Elephant 2015

“We are calling 2015 the year of the histogram”

- Amanda Cox

Page 36: Visualising Multi Dimensional Data @ Fifth Elephant 2015

“Visualising big data is the process of creating generalized histograms”

Page 37: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Viz Process (Big Data)

Acquire Data

Encode Shape

Select Scales

Render Coordinates

Parse Variables

Filter Data

Aggregate Data

Page 38: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Small DataLarge Data

Big DataWide Data

Page 39: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Multi Dimensional Viz Standard 2d/3d

Pixel Based Approach

Glyph Approach

Geometric Transforms

Stacking Approach

Scatterplot

SPLOM

Trellis / Facets

Multiple View

Star plots

Stick Figure

Chernoff Faces

Color Icons

Parallel Coord

Table lens

Star Coords

Tours

Space Filling

Pixel Bar Chart

Spiral Technique

Treemaps

Dimensional Stacking

Hierarchical Axis

Page 40: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Multi Dimensional Viz Standard 2d/3d

Pixel Based Approach

Glyph Approach

Geometric Transforms

Stacking Approach

Scatterplot

SPLOM

Trellis / Facets

Multiple View

Star plots

Stick Figure

Chernoff Faces

Color Icons

Parallel Coord

Table lens

Star Coords

Tours

Space Filling

Pixel Bar Chart

Treemaps

Dimensional Stacking

Hierarchical Axis

Need for Interaction

Ease of Interpretation

Spiral Technique

Page 41: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Diamonds dataset

Page 42: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Diamonds dataset

Price of diamonds is

related to the 4C’s

Page 43: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Diamonds dataset

zdepth

table width

z

y

x

Page 44: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Diamonds dataset

Page 45: Visualising Multi Dimensional Data @ Fifth Elephant 2015

x

x

y

Page 46: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Chart Options Points Bars Lines Areas

1d Quantitative

1d Categorical

2d Quantitative + Categorical

2d Categorical + Categorical

2d Quantitative + Quantitative

Page 47: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Chart Options Points Bars Lines Areas

1d Quantitative Strip Plot Histogram Freq Poly Density Plots

1d Categorical Dot Plot Bar Chart Avoid Avoid

2d Quantitative + Categorical

Strip Plot Box Plot Freq Poly Density Plots

2d Categorical + Categorical

Avoid Bar Chart Avoid Mosaic Plot

2d Quantitative + Quantitative

Scatter Plot Table Lens Slopegraph Avoid

Page 48: Visualising Multi Dimensional Data @ Fifth Elephant 2015

2d Scatter Plot

Page 49: Visualising Multi Dimensional Data @ Fifth Elephant 2015

2d Scatter PlotInteraction: Annotation

Page 50: Visualising Multi Dimensional Data @ Fifth Elephant 2015

2d Scatter Plotlog transformation

Page 51: Visualising Multi Dimensional Data @ Fifth Elephant 2015

2d Scatter PlotSelect or Filter

Area of InterestCarat > 1, Price > 10,000

Page 52: Visualising Multi Dimensional Data @ Fifth Elephant 2015

2d Scatter PlotInteraction: Pan & Zoom

Page 53: Visualising Multi Dimensional Data @ Fifth Elephant 2015

z

x

y

Page 54: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Use aesthetic for 3d

Size Color Shape

Page 55: Visualising Multi Dimensional Data @ Fifth Elephant 2015

3d Scatter PlotSize for Quantitative Dim

Page 56: Visualising Multi Dimensional Data @ Fifth Elephant 2015

3d Scatter PlotColor for Categorical Dim

Page 57: Visualising Multi Dimensional Data @ Fifth Elephant 2015

3d Scatter PlotShapes don’t scale well

Page 58: Visualising Multi Dimensional Data @ Fifth Elephant 2015

3d Scatter Plotdepth persp not good

Page 59: Visualising Multi Dimensional Data @ Fifth Elephant 2015

3d Scatter PlotInteraction: Rotation

Page 60: Visualising Multi Dimensional Data @ Fifth Elephant 2015

zx

y

w

vu

Page 61: Visualising Multi Dimensional Data @ Fifth Elephant 2015

4d Bubble PlotColor and Size

Page 62: Visualising Multi Dimensional Data @ Fifth Elephant 2015

5d Bubble PlotColor, Size and Time

The Joy of Stat - Hans Rosling

Page 63: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Trellis / FacetsCreate Small Multiples

Page 64: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Trellis / Facet GridCreate Small Multiples

Page 65: Visualising Multi Dimensional Data @ Fifth Elephant 2015

SPLOMScatterplot Matrix

Price

Carat

Table

Depth

Page 66: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Multiple ViewCreate Many Small Charts

Page 67: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Multiple ViewInteraction: Brushing & Linking

Page 68: Visualising Multi Dimensional Data @ Fifth Elephant 2015
Page 69: Visualising Multi Dimensional Data @ Fifth Elephant 2015

zx

y

w

v

u

Page 70: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Star Stick Chernoff

Icon based Approach

Page 71: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Star PlotMatrix Layout

color

clarity

depth

cut

table

Page 72: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Star PlotPlot on X-Y location

color

clarity

depth

cut

table

Page 73: Visualising Multi Dimensional Data @ Fifth Elephant 2015

SubplotsBinned Plot Distribution

Page 74: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Orthogonal Parallel

Page 75: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Parallel CoordInteraction: Sorting

Page 76: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Parallel CoordInteraction: Selection

Page 77: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Table PlotInteraction: Bin & Sort

Page 78: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Table PlotInteraction: Zoom & Filter

Page 79: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Stacked Interaction: Brushing

Mosaic Plotcut, color and clarity

Other example - Treemaps

Page 82: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Viz Process (Wide Data)

Acquire Data

Encode Shape

Select Scales

Render Algorithm

Parse Variables

Filter Data

Aggregate Data

Make Views

Add Interactivity

Page 83: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Data Viz Process (Wide Data)

Acquire Data

Encode Shape

Select Scales

Render Algorithm

Parse Variables

Filter Data

Aggregate Data

Make Views

Add Interactivity

1. Encode wisely2. Use space and multiples 3. Add interactivity4. Reduce problem space

Page 84: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Code for these Slideshttps://github.com/amitkaps/multidim

R libraries

❖ ggplot2❖ GGally❖ ggsubplot❖ scales❖ iplots/Mondrian

❖ ggvis❖ tourr❖ rgl❖ scatterplot3d

❖ dplyr❖ tabplot❖ grid❖ gridExtra

Page 85: Visualising Multi Dimensional Data @ Fifth Elephant 2015

“The greatest value of a picture is when it forces

us to notice what we never expected to see”

John Tukey

Page 86: Visualising Multi Dimensional Data @ Fifth Elephant 2015

Amit Kapoor@amitkaps

amitkaps.comnarrativeviz.com

Data

Visual

Story

*