exploratory visual analysis in large high-resolution displays

Exploratory Visual Analysis in Large High-Resolution Display Environments

Khairi Reda

• Society is generating data at ever escalating rates

• The rise of the supercomputer

• Advances in data collection instruments and techniques (e.g., high-throughput genome sequencing)

• We can now afford to capture and store larger quantities of information, in the hope that we can learn something unexpected from it

!2background

background!3[Tukey and Wilk 66, Tukey 77, Tukey 80]

• Confirm / disconfirm a particular hypothesis

!• Typically involves statistical

tests and computation modeling

!• Perfect for computers

Confirmatory analysis

• Look for unexpected patterns and outliers

!• Ask new questions!

!• Not a fit task for computers

Exploratory analysis

Hypothesis-driven inquiry

Data-driven inquiry

New perceptual aids are needed to facilitate the exploration of “big data” sources

• Cost of digital displays is decreasing rapidly

• Large high-resolution displays are becoming more affordable

• Deployed in real-world scientific settings

• Collaboration between co-located teams

• Visual analysis and exploration of large-scale datasets

Nanoscale materials science

Earth sciences

Ecology and behavioral

biologybackground[Reda et al. 13b, Febretti et al. 13, Reda et al 13c] !4

Could these display improve the rate and quality of insight during exploratory visual analysis? And how/why??

information space

visual exploration

information space

visual exploration

information space

visual exploration

information space

+ + =?

visual exploration

!6visual exploration

temporal separation

temporal separation spatial separation (i.e., juxtaposition)

1. What is the effect of increasing the physical size and resolution of the display on user behavior, during visual exploration?

2. And on insight acquisition?

3. Are there new design patterns for exploratory multi-view visualizations on large high-resolution displays?

related work

Low-level visualization tasks (e.g., find the state with the highest population growth between 1960 and 1990)

Task completion

Data sizeStr

User performance on a

big display

[Yost and North 06, Yost et al. 07, Ball et al. 07]related work

!9related work [Andrews et al. 10, Andrews and North 13]

theory !10

display

theory !10

display

View4time

sequentialaccess to

information

theory !10

display

View1 View2

View3 View4

display

sequentialaccess to

information

theory !10

display

View1 View2

View3 View4

display

sequentialaccess to

information

embodied, non-sequential

access to information

theory !10

display

View1 View2

View3 View4

display

sequentialaccess to

information

Cost (Temporal-separation) Cost (Spatial-separation)

theory !10

display

View1 View2

View3 View4

display

sequentialaccess to

information

Cost (Temporal-separation) Cost (Spatial-separation)=>

!11[Norman 02]

seven stages of

action

System

Intention

Action

Execution Perception

Interpretation

Evaluation

Cost ~ Usability: the ease and learnability of a human-made object

theory

!12[Lam 08]

Cost (temporal-separation)

theory

Form goal!

Visualization tool

Form physical sequence!

Execute physical sequence

Form system operators!

Perceive state

framework of interaction

costs in visualizations

Evaluate interpretation

Interpret perception

!12[Lam 08]

theory

!12[Lam 08]

theory

Interpret perceptionTemporal-view association

!12[Lam 08]

theory

Interpret perceptionTemporal-view association

Mental map (re)-building[Purchase et al. 07]

Cognitive integration[Ratwani et al. 08, Plumlee & Ware 06 ]

!12[Lam 08]

theory

Form goal!

Visualization tool

Perceive state

!12[Lam 08]

theory “cognitive resistance”

Form goal!

Visualization tool

Perceive state

!13[Lam 08]

Perceive stateExecute physical sequence

Visualization tool

Form goal!

Cost (spatial-separation)

theory

!13[Lam 08]

Visualization tool

theory

!13[Lam 08]

Visualization tool

theory

Potentially strenuous physical navigation

(head turns, walking, leaning)

!13[Lam 08]

Visualization tool

theory

Managing attention in a more complex environment

Visually resolve a more cluttered visualization

Potentially strenuous physical navigation

(head turns, walking, leaning)

• Study I: exploratory case study

• Comparative visual analysis of ensemble datasets on large displays

• Real-world application + domain expert (behavioral ecology)

• Study II: comparative study

• Investigate the effects of increasing the size/resolution of the display on user behavior and insight acquisition

• Simulated analysis task on a real-world dataset

to nestto food

off-trail navigation?

Understanding the navigational strategies of Seed harvester ants

case study [Offord et al. 13]

to nestto food

off-trail navigation?

Understanding the navigational strategies of Seed harvester ants

case study [Offord et al. 13]

!16case study

On-trail ants West side ants East North south

!16case study

On-trail ants West side ants East North south

design pattern: query-by-example brush

Findings: user strategy

case study

• Think-aloud protocol

• Verbal protocol analysis

• Coding scheme: Observations, Hypotheses, and Decisions

• Interactions: Query by example, Workspace management

Workspacemanagement

Query by example

Hypothesis formulation

Decision making

Observing outliers

layout-preserving interactions

temporal separation of views

Study II: Effects of increasing the display size and resolution on user analytic behavior and insight acquisition

• Goals

• Measure effects of increasing the display size and resolution on quantity and quality of discoveries made during visual exploration

• Understand variations in user behavior and analytic strategy induced as a result of using a larger display with more pixels

• Study design

• Two display conditions (Small, Large)

• Between-subject design

• Scenario

• Open-ended visual exploration task: analysis of Chicago crime patterns between 2006 - 2012 (approximately 2.8 million crime incidents)

experiment

experiment !19

Visualization interface

[Cockburn et al. 08]

201220092006

TheftBurglaryNarcotics

overview map

experiment !19

design pattern: “seed and grow”

201220092006

overview map

experiment !19

201220092006

overview map

participantexperimenter

wide-angle video camera

keyboard & mouse

papernotepad

experiment [Reda et al. 13b]

Experimental conditions

small !

3 x 4 panels 12 Megapixels 40 degree FOV

large !

13 x 4 panels 54 Megapixels

190 degree FOV

experiment

Experimental conditions

small !

3 x 4 panels 12 Megapixels 40 degree FOV

large !

13 x 4 panels 54 Megapixels

190 degree FOV

experiment

!22experiment

• Participants

• 10 unpaid volunteers (4 female) recruited from EVL

• Distributed evenly under the two experimental conditions

• Procedure

• 15-minutes training

• 2.5 hours of open-ended exploration

• Semi-structured debriefing interview

• Verbal protocol analysis

• Coding scheme: Observation, hypothesis, question, goal, comment.

• 5-points quality score: lower-score refer to isolated insights. Higher scores imply broader, more integrative insights

Results: exploration time

experiment

0!20!40!60!80!

100!120!140!

large! small!

Average length of exploratory activity!

experiment

0!20!40!60!80!

100!120!140!

large! small!

Average length of exploratory activity!

t(8) = 33.5, p < .01

Results: observations

experiment

O1! O2! O3! O4! O5! All!

Average number of observations!

large!small!

* * * *

experiment

O1! O2! O3! O4! O5! All!

large!small!

* * * *

𝝌2(4, N=1327) = 263.3, p < .001

experiment

O1! O2! O3! O4! O5! All!

large!small!

* * * *0!

O1! O2! O3! O4! O5! All!

Observation rate!

large!small!

𝝌2(4, N=1327) = 263.3, p < .001

Results: hypotheses

experiment

Results: hypotheses

experiment

H1! H2! H3! H4! H5! All!

Average number of hypotheses!

large!small!

Results: hypotheses

experiment

H1! H2! H3! H4! H5! All!

large!small!

𝝌2(4, N=145) = 67.3, p < .001

Results: hypotheses

experiment

H1! H2! H3! H4! H5! All!

large!small!

H1! H2! H3! H4! H5! All!

Hypothesis formulation rate!

large!small!

𝝌2(4, N=145) = 67.3, p < .001

Results: quantity and rate of insights

experiment

• Comparable rate of insight between the two display conditions

• Participants chose to spend 35 minutes extra time on average exploring the dataset with the large display

• This may have caused participants to make more observations with the large display

Results: quantity and rate of insights

experiment

• Comparable rate of insight between the two display conditions

• Participants chose to spend 35 minutes extra time on average exploring the dataset with the large display

• This may have caused participants to make more observations with the large display

minutes into activity

!27experiment

Results: quality of insights

!27experiment

insight “breadth”

abilit

• The large display is more likely to elicit broader, more integrative insights.

Results: user behavior

experiment

participant S1 (small)

experiment

participant S1 (small)

participant L5 (large)

!29experiment

p(large) - p(small) Results: user behavior

!29experiment

p(large) - p(small)

• Significant tendency to “integrate information / continue to make observations” with the large display

• More frequent transitions to “form goal” (36% increase, non-significant)

• Suggests the formulation and pursuit of more exploratory goals with the large display

Discussion: human limits

experiment

• Some complaints suggest information overload situations with the large display • “It’s hard to look over so much! It’s so hard to compare so many things.”

• Difficulty in integrating information across spatially disparate views • “By the time I finished turning my head to the other side I would forget [the contents of

the previous view]”

• Participants on the large display seem to limit themselves to 6-7 columns (87-102 degrees of visual angle)

Discussion: human limits

experiment

• Some complaints suggest information overload situations with the large display • “It’s hard to look over so much! It’s so hard to compare so many things.”

• Difficulty in integrating information across spatially disparate views • “By the time I finished turning my head to the other side I would forget [the contents of

the previous view]”

• Participants on the large display seem to limit themselves to 6-7 columns (87-102 degrees of visual angle)

Discussion: summary• RQ1 - What is the effect of increasing the display size/resolution on user behavior?

• Significant increase in the length of the exploratory activity.

• Suggests a tendency to pursue more ambitious exploratory goals with the large display

• Small display participants seem to resist exploration when they did not have a priori intuition: “I only bothered to look at the years when I knew something about an area– like Cabrini Green and the Taylor area” !

• Significant tendency to integrate observations / continue to derive new observations!

!• RQ2 - What is the effect of increasing the display size/resolution on insight acquisition?

• Significant increase in the number of observations reported

• Slight increase in the number of hypotheses formulated (not significant)

• Comparable rates of insight (for observations and hypotheses)

• Significant tendency to generate higher-level, more integrative insights.

!• RQ3 - Are there new design patterns for multi-view based visualization on large displays?

• “Seed and grow” design patter

• Loose view coordination model to support multiple exploratory threads

• Lens metaphor to facilitate the exploration of disparate parts of the information space

Scientific reasoning through dual search

outlook [Klahr and Dunbar 88]

Scientific reasoning through dual search: experimenters

outlook

Hypothesis space

experiment / Information

[Klahr and Dunbar 88]

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

observations

outlook

Hypothesis space

observations

outlook

Hypothesis space

generalize

Scientific reasoning through dual search: theorists

outlook

Hypothesis space

outlook

Hypothesis space

outlook

Hypothesis space

predict

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

outlook

Hypothesis space

predict

collect evidence

evidence

outlook

Hypothesis space

predict

collect evidence

evidence

outlook

Hypothesis space

predict

collect evidence

evaluate evidence

evidence

outlook

Hypothesis space

predict

collect evidence

evaluate evidence

re-frame

!35contributions

theory[c2]

!35contributions

theory[c2]

!35contributions

theory[c2]

design patterns

!35contributions

theory[c2]

design patterns

!35contributions

theory[c2]

design patterns

case study + experiment

[c4-5]

!35contributions

apply instantiate

theory[c2]

design patterns

[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c4-5]

!35contributions

apply instantiate

validate

theory[c2]

design patterns

[c4-5]

scientific reasoning

hypothesis

observations

!35contributions

apply instantiate

validate reflect

theory[c2]

design patterns

[c4-5]

scientific reasoning

hypothesis

observations

!36future work

• New design patterns?

• Desktop has “overview, zoom, detail-on-demand” [Shneiderman, 96]

• “Spread, eye-ball, coalesce”?

• Design space for multi-view-based visualizations on large displays

• Collaborative visual analytics

• This is where the cost investment makes more sense.

• Applications and real-world case studies

thank you!

Extras

background

Why visualize data?

visualization statistics

I II III IVx mean 9 9 9 9y mean 7.50 7.50 7.50 7.50

x variance 10 10 10 10

y variance 3.75 3.75 3.75 3.75

x/y correlation 0.81 0.81 0.81 0.81

[Anscombe’s quartet]

background

Why visualize data?

visualization statistics

I II III IVx mean 9 9 9 9y mean 7.50 7.50 7.50 7.50

x variance 10 10 10 10

y variance 3.75 3.75 3.75 3.75

x/y correlation 0.81 0.81 0.81 0.81

[Anscombe’s quartet]

Contained within the data of any investigation is information that can yield conclusions to questions not even originally asked. That is, there can be surprises in the data… To regularly miss surprises by failing to probe thoroughly with visualization tools is terribly inefficient because the cost of intensive data analysis is typically very small compared with the cost of data collection.

-William Cleveland, the Elements of Graphing Data

intuition, prior hypotheses,

experience

Visualization

spontaneous observations,

new hypotheses

observations: unit of knowledge acquired from the visualization !hypothesis: conjecture that cannot be directly inferred from the visualization

visual exploration

{insight

form exploratory goals

interpret visual patterns

[Liu and Stasko 10, Treisman 86]

Alternate projections of the same information

Multiple visualization states (i.e., exploratory

threads)

Different subsets of information

(aka, small-multiples)

visual exploration

http://flowingdata.com/tag/small-multiples/

[Wang Baldonado et al. 2000]

Vis!workspace 1

Vis!workspace 2

Vis!workspace 3

Visualization

• User performance kept pace with increasing data and screen size [Yost et al. 07] • Increased physical navigation over virtual navigation [Ball and North 05, Ball et al. 07] • Mixed results in map-related tasks; some interfaces are poorly-suited to large displays (e.g. context + focus lenses) [Jakobsen and Hornbæk 11]

Intelligence analysis

(text)

• Memory externalization, semantic spatial arrangement (users attach meaning to space), Schematization [Andrews et al. 10] • Automatic inferring of semantics from interactions [Endert et al. 12b]

Office environment

• Improved spatial cognition + embodied memory [Tan 04] • Reduced gender performance disparities in virtual navigation [Tan 03b] • Reduced window switching, and reduced user frustration [Ball and North 05] • Increased productivity in cognitively demanding office work [Czerwinski et al. 03, Bi et al. 09]

related work

High-res / reduced FOV

[Ball and North 08]related work

Low-res / wide FOV

• High-resolution + physical navigation is more important to improving user performance

High-res / reduced FOV

[Ball and North 08]related work

Low-res / wide FOV

• High-resolution + physical navigation is more important to improving user performance

context + focus displays [Baudisch et al. 02]

action

perception

visual exploration

action

perception

data mining, filtering,

transformation, re-projection

induction, deduction, analogical thinking

information space insights

visual exploration [Sedig et al. 12]

action

perception

data mining, filtering,

transformation, re-projection

induction, deduction, analogical thinking

information space insights

Temporal separation

visual exploration [Sedig et al. 12]

!47theory

View1 V2 V3 V5V4

[Maxcey-Richard and Hollingworth 13]

unexpected discoveries

Cost (temporal-separation): bottom-up costs

!47theory

View1 V2 V3 V5V4

[Maxcey-Richard and Hollingworth 13]

unexpected discoveries

Cost (temporal-separation): bottom-up costs

timeP(View1)

!48theory

• Top-down costs

• Discourage the formation of new exploratory goals

• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets in the information space

• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest

• Reduction in the probability of making spontaneous inferences relating to patterns deposited in temporally-separated views

Cost (temporal-separation): summary

!48theory

• Top-down costs

• Discourage the formation of new exploratory goals

• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets in the information space

• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest

• Reduction in the probability of making spontaneous inferences relating to patterns deposited in temporally-separated views

Cost (temporal-separation): summary

!49case study

Screen surface

2D movement

Trajec

case study

experiment !50

201220092006

overview map

experiment !50

201220092006

overview map

experiment !50

Density of crime incidence

Yearly, weekly, daily crime trends

201220092006

overview map

experiment !50

Density of crime incidence

Yearly, weekly, daily crime trends

201220092006

overview map

!51experiment

structure layoutfreeform layout

!52experiment

• 3 “mental states”

• Make observation

• Form goal

• Formulate hypothesis

• 2 types of interaction

• Layout-preserving (brush-and-link / pan map)

• Layout-changing (create new view, changing view contents)

Results: user behavior / strategy

experiment

average (small)

average (large)

large!

small!

usability utility

usability / utility

experiment

!55experiment

Do higher-level insights occur at a later time in the activity?

!55experiment

minutes into activity

l of i

Do higher-level insights occur at a later time in the activity?

Form goal!

Visualization tool

Execute physical sequence Perceive state

Form goal!

Visualization tool

Form goal!

Visualization tool

Form goal!

Visualization tool

!57contributions

1. Theory of how interaction costs affect user behavior in exploratory visual analysis

• `Tunnel vision’ phenomenon in situations involving large amounts of data

• Costs are elevated due to excessive temporal-separation of views

2. Effects of increasing the size and resolution of the visualization interface on user behavior and insights acquisition

• Increased user investment / exploration

• More observations

• Formation of higher-level, more integrative insights

3. Design patterns for large high-resolution displays

• Query-by-example brush for the analysis of ensemble data

• “Seed and grow” pattern: loose view coordination model with lens metaphor

exploratory visual analysis in large high-resolution displays

exploratory visual analysis

exploration of large

object theory

nonsequential access

big display yost

conrmatory analysis

exploratoryvisual analysis

cost of digital displays

Science

ftx openlc - orbx-user-guides.storage.googleapis.com ·...

windows 7 for seniors for...

jtouch interactive displays - full compass systems · 2019....

image super-resolution the techniques, …...therefore: are...

engaging displays for...

tutorial – “how to:” 1) change the resolution on your...

using high-resolution displays for high-resolution cardiac...

skyline high resolution metabolomicsfiles... ·...

face hallucination revisited: an exploratory study …face...

piper 2014 - piper aircraft · safety up front. primary...

clustering solutions final exploratory run full 10’...

adopting a game engine for large, high-resolution displays

user manual - planar€¦ · introduction 020 page | 7 ep...

statistics: more graphs and displays. graphing quantitative...

a multiscale interaction technique for large,...

grating light valves for high resolution displays

1 objectives objectives explain how video displays work...

the effects of curving large, high-resolution displays on

high resolution displays at pppl mike miller...

high resolution displays at pppl