display techniques in information-rich virtual environments
DESCRIPTION
Display Techniques in Information-Rich Virtual Environments. Nicholas F. Polys PhD Research Proposal August 18, 2004 Committee: Dr. Doug Bowman, VT Dr. Chris North, VT Dr. Scott McCrickard, VT Dr. Ken Livingston, Vassar College Dr. Don Brutzman, Naval Postgraduate School. - PowerPoint PPT PresentationTRANSCRIPT
Display Techniques in Information-Rich Virtual Environments
Display Techniques in Information-Rich Virtual
Environments Nicholas F. Polys
PhD Research ProposalAugust 18, 2004
Committee:
Dr. Doug Bowman, VT
Dr. Chris North, VT
Dr. Scott McCrickard, VT
Dr. Ken Livingston, Vassar College
Dr. Don Brutzman, Naval Postgraduate School
Display Techniques in Information-Rich Virtual Environments
Proposal Outline
• Problem scope & Statement
• Background
• Research Questions & Goals
• Information Design Dimensions
• Approach, Method, Measures
• Experimental Program
• Significance
Display Techniques in Information-Rich Virtual Environments
• Complex systems typically span multiple scales and involve heterogeneous data types (objects, spatial relations, attributes)
• Engineers, researchers, and analysts need to access, manage, and understand a wide variety of information and inter-relationships
General Problem: Integrated Information Spaces
Display Techniques in Information-Rich Virtual Environments
Fundamental Data Types
• Spatial / perceptual data:
geometry, colors, textures, lighting
• Abstract data / world & object attributes: nominal, ordinal, quantitative
• Temporal data / behaviors:
states, dynamics
Display Techniques in Information-Rich Virtual Environments
Information-Rich Domains
Simulation and design applications require spatial/perceptual fidelity and information enhancement:
• Engineering / CAD• Construction / architecture • Medicine / biology • Science / research• Education / training• …
Display Techniques in Information-Rich Virtual Environments
Display Techniques in Information-Rich Virtual Environments
Display Techniques in Information-Rich Virtual Environments
User Problem: Integrated Information Spaces
• Multiple applications and fragmented views make it difficult to understand the relationships between information types
• Next generation information interfaces must unify display and interaction spaces for:– Exploration– Search– Comparison and Pattern recognition
Display Techniques in Information-Rich Virtual Environments
Problem Statement:We lack precise definitions, development tools, and
systematic research as to how perception and cognition operate in information-rich interfaces and environments:
• Combining Virtual Environments and Information Visualizations is currently ad hoc and application-specific
• There are competing models of Vision and Working Memory that may apply
• An experimental methodology and theory is required to assess, design, and deliver ‘appropriate’ displays
Display Techniques in Information-Rich Virtual Environments
Background:Information Psychophysics
• Edward Tufte, Envisioning Information
(1983, 1990)• Jaques Bertin, Semiology of Graphics
(1983)• Donald Norman, Cognitive Engineering
(1986)• Joseph Goguen, Semiotic Morphisms
(2000)• Colin Ware, Perception for Design
(2003)
Display Techniques in Information-Rich Virtual Environments
Background:Information Visualization
Visual display of abstract information• Visual Markers
(Cleveland & McGill, 1984; Mackinlay, 1986; Card et al, 1999)
• Multiple Views (North, 2001; North et al, 2002; Convertino et al, 2003)
• Zoom-able Interfaces (Bederson et al, 1996, 2000; Woodruff et al, 1998a-c)
Display Techniques in Information-Rich Virtual Environments
Background:Virtual Environments
Visual display of spatial and
perceptual data• Immersive and Desktop Platforms• Conceptual Learning (Salzman et al, 1999)
• Navigating space (Darken et al, 1996, 2002)
• Naturalism & Performance (Bowman, 2002; Bowman et al, 2004)
• Image Plane (Pierce et al, 1997)
Display Techniques in Information-Rich Virtual Environments
Background:Augmented Reality
Enhancing perceptual scenes with additional abstract information
• Feiner et al. Windows on the World (1993)• Bell et al. Dynamic Space Management,
View Management (2000, 2001)
Display Techniques in Information-Rich Virtual Environments
Background:Multimedia & Comprehension
• Co-references between text and images(Chandler & Sweller, 1990; Faraday & Sutcliffe, 1997, 1998)
• Task Knowledge Structure (Sutcliffe & Faraday, 1994; Sutcliffe 2003)
• Meaningful Learning: troubleshooting, redesigning, deriving principles (Mayer, 2002)
Display Techniques in Information-Rich Virtual Environments
Background:Architecture of WM
• Componentized WM (Baddeley, 2003) – phonological loop, visuospatial sketchpad,
episodic buffer, central executive
• Individual differences in WM Capacity (Just & Carpenter: 1996)
• Short and Long term WM (Ericsson & Kintsch: 1995)
Display Techniques in Information-Rich Virtual Environments
Working Memory (Baddeley, 2003)
Central Executive
PhonologicalLoop
Episodic bufferVisuospatial sketchpad
Visual semantics Episodic LTM Language
Fluid system
Crystallized system
Display Techniques in Information-Rich Virtual Environments
Background: Visuospatial WM
• Capacity 3-5 ‘items’ • Functional units & chunking
– Objects & features (Vogel et al, 2001)
– Visual indices & dynamic feature binding (Saiki, 2003)
• Subsystems: form & color, space & movement (Logie 1995)
• Relation to Central Executive(Miyake et al. 2001)
Display Techniques in Information-Rich Virtual Environments
Background: Interpreting Linegraphs
• Computational model: Understanding Cognitive Information Engineering [UCIE] (Lohse, 1991) – similar to GOMS (Card, Moran, & Newell, 1983), and ACT*
(Anderson, 1983)
• Processing – encoding visual patterns– inferring and retrieving functional relations via
graph schema– associating / labeling referents
Display Techniques in Information-Rich Virtual Environments
Proposal: Information-Rich Virtual Environments (IRVEs)
We need to understand:
• How spatial/perceptual information and abstract information can be combined and displayed
• What makes the combinations effective• What makes them usable and • How users think and act when using them
Display Techniques in Information-Rich Virtual Environments
Research Question for IRVE Information Design:
Where and How should enhancing abstract information be displayed
relative to its perceptual referent
so that
the respective information can be understood together and separately?
Display Techniques in Information-Rich Virtual Environments
Research Goals:
• To understand how the respective design techniques of Virtual Environments and Information Visualization can be combined and balanced
• To enable stronger mental associations between spatial and abstract information while preserving the models of each type of information.
Display Techniques in Information-Rich Virtual Environments
Research Goal 1:
• Define a theoretical framework for Information-Rich Virtual Environments (IRVEs) as the solution to the problem of integrated information spaces
• Results: – Terminology– Definitions– Bowman et al, 2003
Display Techniques in Information-Rich Virtual Environments
Research Goal 2:
• Enumerate the design space for IRVE tasks and display techniques through Usability Engineering process and literature review
• Results:– IRVE Task Space, Information Design Space,
Interaction Design Space– Polys & Bowman, 2004; Polys et al, 2004c
Display Techniques in Information-Rich Virtual Environments
Research Goal 3:
• Prototype information-rich application interfaces to identify problems and generate hypotheses regarding optimal IRVE information design per task
• Results:– IRVE Display Prototypes– Claims Analysis 1 (heuristic evaluation, informal user
studies, pluralistic walkthroughs)– Polys, 2003; Polys et al, 2004a, 2004b
Display Techniques in Information-Rich Virtual Environments
Research Goal 4:
• Describe IRVE display configurations in a concise XML DTD and Schema and use this display description to generate runtime components
• Results [pending]:– IRVE Testbed
Display Techniques in Information-Rich Virtual Environments
Research Goal 5:
• Provide a quantitative basis by which to characterize the density and distribution of information in an IRVE
• Results [pending]:– Assessment method for IRVE information sets
Display Techniques in Information-Rich Virtual Environments
Research Goal 6:
• Identify tradeoffs and guidelines for the IRVE display design space using
– Empirical usability evaluations and
– Metrics for individual cognitive differences
• Results [pending]:– Empirical data relating design techniques,
information sets, and user performance
Display Techniques in Information-Rich Virtual Environments
Research Question re-phrased:
For the combination of abstract and perceptual visualizations in IRVEs, association should be maximized and interference (such as occlusion and crowding by layouts) should be minimized…
What are the best ways to manage layout space and association cues so that perceptual and
abstract information can be understood together and separately?
Display Techniques in Information-Rich Virtual Environments
Goals & Results Summary
• Survey IRVE information design space
• Prototype IRVE display techniques and applications
• Understand and quantify how IRVE information sets vary
• Enumerate information design heuristics and guidelines for IRVEs
Display Techniques in Information-Rich Virtual Environments
IRVE Information Design Challenges
• Visibility
• Legibility
• Association
• Occlusion
• Aggregation
Display Techniques in Information-Rich Virtual Environments
Abstract information design parameter
Psychological process
Usability impact
Visual attributes:- color- fonts- size- background- transparency
Perception - Legibility - Readability
- Occlusion
Layout attributes:- location- association - density
Interpretation - Relating abstract and perceptual information- Conceptual categories & abstractions- Occlusion
Aggregation:- level of information encoding - type of visualization
Making Sense - Comparison & Pattern Recognition- Effectiveness - Satisfaction
IRVE Information Design Dimensions
Display Techniques in Information-Rich Virtual Environments
Layout Space (Locations)
The layout space of abstract information in IRVEs is described by the coordinate system it is resident in:
• Object• World• User• Viewport• Display
Display Techniques in Information-Rich Virtual Environments
Object Space
Object space is relative to an object’s location in
the environment (e.g. Semantic Objects).
Display Techniques in Information-Rich Virtual Environments
World SpaceWorld space is
relative to an area, region, or location in the
environment.
Display Techniques in Information-Rich Virtual Environments
User Space
User space is relative to the user’s location but not their viewing angle.
Display Techniques in Information-Rich Virtual Environments
Viewport Space
Viewport space-is the image plane where HUDs or overlays may be located.
Display Techniques in Information-Rich Virtual Environments
Display Space
Display layout space where abstract visualizations are located outside the rendered view in some additional screen area.
Display Techniques in Information-Rich Virtual Environments
Association
The association dimension of IRVE information design is delineated by the Gestalt principles:
Display Techniques in Information-Rich Virtual Environments
Approach (Goals 4-6)
• Develop methods to describe and generate IRVE display components
• Develop quantitative methods to characterize IRVE datasets
• Use the above to control testbed environment and stimuli and run empirical usability evaluations
Display Techniques in Information-Rich Virtual Environments
IRVE Testbed Configuration Syntax
• The composition of an IRVE display will be defined under an XML DTD and Schema. The DTD and Schema will provide syntactic and semantic production rules for IRVE display spaces.
• In order to instantiate an IRVE display, the testbed will read the information mapping configuration from the XML and generate the X3D code for that
IRVE.
Display Techniques in Information-Rich Virtual Environments
IRVE Stimuli & Metrics
Exocentric vs. Egocentric:
Exocentric metrics have an advantage in that they may be employed on a data set independent of the user or rendering; egocentric metrics may be more advantageous to find dynamic, perspective-specific layouts.
This research proposal will investigate exocentric metrics for IRVE data characteristics such as the quantity, density, and distribution of abstract information in the VE.
Display Techniques in Information-Rich Virtual Environments
IRVE Layout & Association Dimensions
Common Region
Proximity Connected ness
Similarity Common Fate
Object x x x x x
World x x x x x
User x x x x x
Viewport x x x x x
Display x x x x x
Experiment 1 Experiments 2 & 3
Display Techniques in Information-Rich Virtual Environments
Evaluation Method
• The usability experiments will have a factorial, within-subjects, counterbalanced design
• Cognitive battery test will be used control between-subject variance and may provide insight into how display techniques work with different visuospatial abilities.
These experiments will allow us to explore tradeoffs in IRVE information design and identify guidelines and design patterns for integrated information spaces.
Display Techniques in Information-Rich Virtual Environments
Procedure
• Subjects from VT population
• Demographic questionnaire &
Cognitive battery tests
• Training session for spatial navigations
• Experimental Protocol &
dependent measures
Display Techniques in Information-Rich Virtual Environments
Measures
• Time to Completion
• Correctness
• Satisfaction / Ease of Use
• User strategy
• Cognitive Battery
Display Techniques in Information-Rich Virtual Environments
Perceptual Speed / Closure Flexibility
Spatial Visualization
Spatial Orientation Cognitive
Battery
Display Techniques in Information-Rich Virtual Environments
IRVE TasksIRVE Search Tasks require subjects to:• find a piece of abstract information based on some
perceptual criteria• find a piece of perceptual information based on
some abstract criteria. IRVE Comparison Tasks require subjects to:• compare perceptual attributes of two items with a
given abstract criteria• compare abstract attributes of two items with a
given perceptual criteria
Display Techniques in Information-Rich Virtual Environments
Experiment 1:
Object space layouts can be drawn at a fixed orientation or billboarded to always face the user. Is one technique better than the other for common tasks and navigation modes?
Here we will investigate Search and Comparison tasks combined with flying or terrain-following navigation.
Display Techniques in Information-Rich Virtual Environments
Object space:Billboard vs. Fixed
• Search & Comparison Tasks• Terrain following and Flying navigation• Fixed orientations provide additional
wayfinding cues• Billboarded orientations require less spatial
navigation for legibility• In these circumstances, is one display
technique better that the other?
Display Techniques in Information-Rich Virtual Environments
Experimental Unit Human Subject (n = 8)
Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation,
Factor 1 Object space layout
Level 1 Billboarded
Level 2 Fixed
Factor 2 Navigation type
Level 1 Walk
Level 2 Fly
Factor 3 Task: Search
Level 1 Spatial -> Abstract
Level 2 Abstract -> Spatial
Factor 4 Task: Comparison
Level 1 2 Spatial items
Level 2 2 Abstract items
Response Variables Performance: Time to Completion, CorrectnessSatisfaction: Ease of Use
16 conditions
Display Techniques in Information-Rich Virtual Environments
Experiment 2:
Abstract information is associated to perceptual information through different cues in Object and Viewport spaces. How do the Gestalt principles rank in IRVEs?
We will examine connectedness, proximity, and common region for Search tasks between these two layout spaces.
Display Techniques in Information-Rich Virtual Environments
Association in Object vs. Viewport spaces
• Search Task
• Relating abstract and perceptual information between layout spaces and association parameters
• Do depth cues of object space enable a tighter coupling of information types or is the occlusion unneccessary?
Display Techniques in Information-Rich Virtual Environments
Experimental Unit Human Subject (n = 8)
Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation,
Factor 1 Layout space
Level 1 Object
Level 2 Viewport
Factor 2 Gestalt principle
Level 1 Connectedness
Level 2 Proximity
Level 3 Common Region
Factor 3 Task: Search
Level 1 Spatial -> Abstract
Level 2 Abstract -> Spatial
Response Variables Performance: Time to Completion, CorrectnessSatisfaction: Ease of Use
12 conditions
Display Techniques in Information-Rich Virtual Environments
Experiment 3:
The theory behind IRVEs must account for a wide range of content. How does content determine an appropriate layout space if at all? We will examine Object and Viewport spaces for Comparison tasks in different environments.
We will factorially compare environments of high and low density and distribution.
Display Techniques in Information-Rich Virtual Environments
Density and Distribution thresholds for
Object vs. Viewport spaces
• Comparison Task
• Can environment characteristics predict advantageous display techniques and ground design guidelines?
Display Techniques in Information-Rich Virtual Environments
Experimental Unit Human Subject (n = 16)
Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation, Digit Span
Factor 1 Layout Space
Level 1 Object
Level 2 Viewport
Factor 2 Task: Comparison
Level 1 2 Spatial items
Level 2 2 Abstract items
Factor 3 Information Density
Level 1 Low
Level 2 High
Factor 4 Information Distribution
Level 1 Homogenous
Level 2 Clustered
Response Variables Performance: Time to Completion, CorrectnessSatisfaction: Ease of Use
12 conditions
Display Techniques in Information-Rich Virtual Environments
Significance
• Establishes a research area combining Information Visualization and Virtual Environments
• Investigates how search and comparison tasks can be supported with IRVE display techniques
• Relates display techniques to models of perception and working memory
Display Techniques in Information-Rich Virtual Environments
Significance (cont’d)
• Supports and extends international standards technology (e.g. X3D, XML)
• Provides tools for IRVE developers
• Provides guidelines for IRVE designers
• Environment characteristics and display guidelines may also transfer to Augmented Reality
Display Techniques in Information-Rich Virtual Environments
Timeline & MilestonesSeptember 2004 IRVE Task Space
EnumerationIRVE Data Metrics
October 2004 IRVE Display LanguageTestbed completed
November 2004 Experiment 1
January 2004 Experiment 2
March 2005 Experiment 3
Fall 2005 Research Defense
Spring 2006 Dissertation Defense
Display Techniques in Information-Rich Virtual Environments
Thank YouQuestions & Discussion
http://www.virtuworlds.com/X3D/npolys_PhD_Proposal.doc
http://www.virtuworlds.com/media_02/index.html#wares