![Page 1: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/1.jpg)
Information Visualization
and
Immersive Interfaces
CSCI 6174: Open Problems in CS
Fall 2013
Richard Fowler
![Page 2: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/2.jpg)
An example
• Hans Rosling's 200 Countries, 200 Years, 4 Minutes • http://www.youtube.com/watch?v=jbkSRLYSojo
![Page 3: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/3.jpg)
To visualize is …
• I see what you mean …
– so, visualization can be considered not just a visual process, but a cognitive (thought) process as well
• …. and a very large part of human brain taken up with visual system
– and that part of brain is useful beyond “simply” getting an
image of the world
![Page 4: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/4.jpg)
Visualization is …
• Visualize:– “To form a mental image or vision of …”– “To imagine or remember as if actually seeing …”– Firmly embedded in language, if you see what I mean
• (Computer-based) Visualization:– “The use of computer-supported, interactive, visual
representations of data to amplify cognition”• Cognition is the acquisition or use of knowledge• Card, Mackinlay Shneiderman ’98
– Scientific Visualization: physical
– Information Visualization: abstract
![Page 5: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/5.jpg)
Visualization is not New
• Cave guys, prehistory, hunting
• Directions and maps
• Science and graphs– e.g, Boyle: p = vt
• … but, computer based visualization is new– Allows interactive exploration and can be embedded
in the work flow– … and the systematic delineation of the design space
of (especially information) visualization systems is growing nonlinearly
![Page 6: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/6.jpg)
Visualization and Insight
• “Computing is about insight, not numbers”– Richard Hamming, 1969– And a lot of people knew that already
• Likewise, purpose of visualization is insight, not pictures– “An information visualization is a visual user
interface to information with the goal of providing insight.”, (Spence, in North)
• Goals of insight– Discovery– Explanation– Decision making
![Page 7: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/7.jpg)
“... insight, not numbers”
• Numbers – states, %college, income:State % college degree income State % college degree income
![Page 8: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/8.jpg)
“... insight, not numbers”
• Insights:– What state has highest income?
– What is relation between education and income?
– Any outliers?
State % college degree income State % college degree income
![Page 9: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/9.jpg)
“… insight, not numbers”
• Insights:– What state has highest income?,
– What is relation between education and income?,
– Any outliers?
![Page 10: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/10.jpg)
Not about Useless Visual Stuff - Clutter
• Here, “3d” adds nothing– (at best)
![Page 11: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/11.jpg)
Detrimental, Useless Stuff
• USA Today• So, what information is this
trying to convey?
![Page 12: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/12.jpg)
An Example, Challenger Shuttle
• Presented to decision makers prior to launch– To launch or not
– Temp in 30’s
• “Chart junk”
• Finding form of visual representation is important– cf. “Many Eyes”
![Page 13: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/13.jpg)
An Example, Challenger Shuttle
• With right visualization, insight (pattern) is obvious– Plot o-ring damage vs. temperature
![Page 14: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/14.jpg)
Insight, through Visualization …
• Some examples ….
![Page 15: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/15.jpg)
Insight, through Visualization …
• What accounts for US deficit?
– Economic recovery measures– TARP, Fannie, and Freddie
“bailouts”– Wars in Iraq and Afghanistan– Bush-era tax cuts– Economic downturn– Other (longer standing) things
![Page 16: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/16.jpg)
For what it’s worth …
![Page 17: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/17.jpg)
A “Taxonomy” of Visualization TopicsCard, Mackinlay, and Schneiderman
SpacePhysical Data1D, 2D, 3DMultiple Dimensions, >3TreesNetworks
InteractionDynamic QueriesInteractive AnalysisOverview + Detail
Focus + ContextFisheye ViewsBifocal LensDistorted ViewsAlternate Geometry
Data Mapping: TextText in 1DText in 2DText in 3DText in 3D + Time
Higher-Level VisualizationInfoSphereWorkspacesVisual Objects
![Page 18: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/18.jpg)
1D - Linear DataSoftware Visualization
![Page 19: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/19.jpg)
2D - Map DataPresidential votes 2008 on Map
![Page 20: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/20.jpg)
3D World Data – Scientific Visualizatione.g., Molecule Modeling – Augmented Physical Representation
![Page 21: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/21.jpg)
Multiple Dimensions > 3What does visual representation for > 3 dimensions look like?
• “Straightforward” 1, 2, 3 dimensional representations– E.g., time and
concrete
• Can extend to more challenging n-dimensional representations– Which is at core of
visualization challenges
• E.g., Feiner et al., “Worlds within Worlds”
![Page 22: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/22.jpg)
Temporal DataBaby names across years
![Page 23: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/23.jpg)
Trees, Networks, and Graphs
• Connections between /among individual entities
• Most generally, a graph is a set edges connected by a set of vertices– G = V(e)– “Most general” data
structure
• Graph layout and display an area of iv
• Trees, as data structure, occur … a lot– E.g., Cone trees
![Page 24: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/24.jpg)
Routes of the Internet, 1/15/05
• Earlier snapshot in permanent collection of NY Museum of Modern Art
![Page 25: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/25.jpg)
Abstract, Non-physical, Network
• E.g., concept map– Graph of
“conceptual” information
• What’s this one?
• From Tim Berners-Lee’s proposal to CERN for what is now called w w w, March 1989
• Manual “graph drawing”
http://www.nic.funet.fi/index/FUNET/history/internet/w3c/proposal.html
![Page 26: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/26.jpg)
Text and Document Collection Structure
• Derivation of relationships upon which display is to be based a challenge
• E.g., Wise et al.
![Page 27: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/27.jpg)
Overview Strategies
• Typically useful, or critical, to have “feel” for all data– Then, system allows closer inspection in “context” of all data– Overview + detail, focus + context
• Known from the outset of visualization– Bifocal Lens
• Database navigation: An Office Environment for the Professional by R. Spence and M. Apperley
• Shneiderman mantra– “overview first, zoom and filter, details on demand”
![Page 28: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/28.jpg)
Focus+Context: Fisheye Views, 1
• Detail + Overview – Keep focus, while remaining aware of context
• Fisheye views– Physical, of course– Also, distance function (based on relevance)– Given a target item (focus)– Less relevant items dropped from display– Classic cover
• New Yorker’s idea of the world
![Page 29: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/29.jpg)
Focus+Context: Fisheye Views, 2• Detail + Overview
– Keep focus while remaining aware of context
• Fisheye views– Physical, of course– Also, a distance function (based on relevance)– Given a target item (focus)– Less relevant items dropped from display – Or, are just physically smaller – distortion
![Page 30: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/30.jpg)
Focus + Context – Spatial Distortion
• Selectively reduce complexity as:
f(user’s viewpoint)
• Spatial distortion– Project network on
distorted space
• Viewing “lens”
![Page 31: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/31.jpg)
Focus + Context – Spatial Distortion
• Selectively reduce complexity as:
f(user’s viewpoint)
• Spatial distortion– Project network on
distorted space
• Viewing “lens”
• Seamless transition
![Page 32: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/32.jpg)
Focus + Context – Hyperbolic View
• Again, selectively reduce complexity as f(user’s viewpoint)• As with distorted lens, space is non-linear, here, hyperbolic
• Smooth change during interaction
![Page 33: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/33.jpg)
3-d Hyperbolic Tree of Web Sites
• Using Prefuse
![Page 34: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/34.jpg)
Visualization Pipeline:Mapping Data to Visual Form, cf. North paper
• Visualizations: – “adjustable mappings from data to visual form to human perceiver”
• Series of data transformations– Multiple chained transformations– Human adjust the transformation
• Entire pipeline comprises an information visualization
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
![Page 35: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/35.jpg)
Visualization Stages
• Data transformations:– Map raw data (idiosynchratic form) into data tables (relational descriptions
including metatags)
• Visual Mappings:– Transform data tables into visual structures that combine spatial substrates,
marks, and graphical properties
• View Transformations:– Create views of the Visual Structures by specifying graphical parameters
such as position, scaling, and clipping
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
![Page 36: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/36.jpg)
Information Structure
• Visual mapping is starting point for visualization design
• Includes identifying underlying structure in data, and for display– Tabular structure– Spatial and temporal structure– Trees, networks, and graphs– Text and document collection structure– Combining multiple strategies
• Impacts how user thinks about problem - Mental model
RawInformation
VisualFormDataset Views
User - Task
DataTransformations
VisualMappings
ViewTransformations
F F -1
Interaction
VisualPerception
![Page 37: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/37.jpg)
A Recent (2012) System
![Page 38: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/38.jpg)
A Recent (2012) System
• Information visualization for task of document retrieval
• Document collection is organized as clusters (visually represented)– User views and interacts with the cluster representations
• IV Interaction:
– Overview– Flock-based clustering
– Zoom, spatially
– Filter– Text match, query– Markers
– Details on demand– Document itself
– Tunable
![Page 39: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/39.jpg)
Swarm (Flock) based Clusteringfor Information Visualization and Retrieval
• Reynold’s technique for animation of flocks of birds– Computational model
of group motion
• Particle system– Standard part of many
toolkits
• http://www.youtube.com/watch?v=-8KZBGAAM30
• Agent-based approach– Individual elements each apply rules for motion, as particle systems, etc.
Reynolds (1987). Flocks, herds, and schools: A distributed behavioral model,Computer Graphics, 21(4), 25-34.
![Page 40: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/40.jpg)
Flock-based Movement
• Individual agents utilize only three rules– Agents adjust their directions of movement and velocities - steer – Form single group
• Movement perceptually similar to flock of birds, other social biological group
– Alignment: Steer towards average direction of movement of nearby agents– Separation: Steer to avoid being too close to nearby agents– Cohesion: Steer to move toward the center position of nearby agents
• Rules are added for obstacles, …, edge of screen, …
![Page 41: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/41.jpg)
Flock-based Clustering: Multiple Groups – Bird Species, Document Clusters, …
• Rules described so far form a single group– As, e.g., with a single species of birds
• Can add additional rules that consider similarity or a priori categorization among agents to form multiple groups
• “Multi-species flocking”– Birds of same species form groups and moved together– Agents (birds) described by feature vector, e.g., length of beak, color, size
• Flock-based document clustering– Vector Space Model– Similarity of documents determined by weighted number of terms in common– Essentially, a biologically (or animation) inspired “Iterative clustering technique”
![Page 42: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/42.jpg)
System
• Information visualization for task of document retrieval
– Overview– Flock-based clustering
– Zoom, spatially
– Filter– Text match, query– Markers
– Details on demand– Document itself
– Tunable
![Page 43: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/43.jpg)
System’s Flock-based Clustering
• E.g., 500 documents– Clusters apparent
• Uses standard metrics of document similarity with velocity vector change based on similarity
• Also considers thresholds of similarity and dissimilarity – Provides tunable parameters for arranging spatial display (Cui et al., 2006)
– Now,
![Page 44: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/44.jpg)
Identifying Clusters of InterestTerm Match
• Term match to mark documents to guide exploration– Matching documents’
color is changed
– Multiple term matches
![Page 45: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/45.jpg)
Identifying Clusters of InterestTerm Match
• Term match to mark documents to guide exploration– Matching documents’
color is changed
– Multiple term matches
![Page 46: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/46.jpg)
Identifying Clusters of InterestTerm Match
• Term match to mark documents to guide exploration– Matching documents’
color is changed
– Multiple term matches
![Page 47: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/47.jpg)
Identifying Clusters of InterestTerm Match
• Term match to mark documents to guide exploration– Matching documents’
color is changed
– Multiple term matches
![Page 48: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/48.jpg)
Identifying Clusters of InterestQuery Match
• Term match to mark documents to guide exploration– Matching documents’
color is changed
– Multiple term matches
• Query match to mark documents
![Page 49: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/49.jpg)
Viewing Individual Documents
• Zoom, then select individual element– Title is displayed
![Page 50: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/50.jpg)
Viewing Individual Documents
• Select title to display individual document text– Also, can select from list retrieved in query
![Page 51: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/51.jpg)
Interactively Adjusting Clusters
• In practice, many clustering techniques make use of “tuning”
• System allows user to change parameters interactively
– Weights for “basic” rules and other motion parameters
• Avoidance, cohesion, alignment• Velocity minimum, maximum• Limit of steering
– Range for consideration– Thresholds for similarity and dissimilarity
![Page 52: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/52.jpg)
Stereoscopic Display
• Perception / extraction of structure facilitated by stereoscopic display– Reveals z component of
3d arrangement
– Head-tracked stereo.
• Structure from motion
• Challenge of practical stereoscopy in context of task performance Two monitor desktop
![Page 53: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/53.jpg)
Stereoscopic Display
Multi-screen display
• Perception / extraction of structure facilitated by stereoscopic display– Reveals z component of
3d arrangement
– Head-tracked stereo.
• Structure from motion
• Challenge of practical stereoscopy in context of task performance
![Page 54: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/54.jpg)
Immersive Interfaces, Prescence, …
![Page 55: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/55.jpg)
Introduction
• The “best” interfaces, and all systems, typically find their task utility through engagement appropriate for the task
– This idea is at the core of arguments for the use of direct manipulation interfaces
• All of the following are interrelated:– Immersion, engagement, presence, virtual reality– 3D display and interaction devices
• In field of CS and HCI: – “spatial interfaces”, “immersive interfaces”
![Page 56: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/56.jpg)
Examples of Immersive Interfaces
Tiled display wall
Head mounted display
Surround screen projection
Spherical projection
![Page 57: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/57.jpg)
Immersion, “Virtual Reality”, and Virtual Environments
• Immersive interfaces– High sensory immersion – visual, auditory, haptic, proprioceptive
• “Virtual reality”, or, virtual environments– “Virtual reality is a technology that is used to generate a simulated environment in
digital form… users are immersed in a totally virtual world.”– Working definition – an immersive interactive system
• In context of “virtual reality”, immersion = spatial immersion– Different literatures use word in different ways
• Note: “Immersion” (and engagement and presence) is a continuum
– Text ... Visual and 3d .. Stereo ... HMD… “jacked in”– Cyberspace
• Term coined by Gibson in Neuromancer, 1984• … and in the 21st century, the Matrix
![Page 58: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/58.jpg)
Immersion and Virtual Reality
• “The mind has a strong desire to believe that the world it perceives is real” – Jaron Lanier, VR “pioneer”, … among others
• For example, “illusion” (perception) of depth (for spatial immersion)• Stereo parallax• Head motion parallax• Object motion parallax• Texture scale
• Interaction: grab and move an object
• Proprioceptive cues: – When you reach out and see a hand where you believe your hand
should be, you accept the hand as your own
• Often user will accept what see asn “real” even if graphics poor
• Constellation of cues
![Page 59: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/59.jpg)
optional
Components for Perception of 3D for Spatial Immersion, Cutting, 1996
• Perception of 3 (vs. 2) dimensions strongest element of spatial immersion
– Perception of 3d from depth cues
• See figure
• Other elements– Integration important– Visual display types– Stereoscopic display – Head position sensing – Hand-position sensing – Force feedback – Sound input and
output – Other sensations
![Page 60: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/60.jpg)
Presence “The Aesthetic Impression of 3D Space”
• Sense of presence – Actually present in the world– Vividly 3d– Sense of being there– Holodeck …– In game literature, often called “immersion”, which is fine, but confusing
for us
• Presence has to do with engagement, as well as visual information– E.g., one can be “in the world”, when reading– Here, one sees, or visualizes, the world
![Page 61: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/61.jpg)
Immersive and 3D Interfaces
• Degree of immersion– conventional desktop screen
• Some 3D displays:– Data walls– Workbenches– Hemispherical display– Head-mounted displays– Arm-mounted displays– Surround screen displays - CAVE– Virtual retinal display– Autostereoscopic displays
• But first, … about stereoscopic display– A “compelling” 3D depth cue
![Page 62: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/62.jpg)
3D Interfaces, Stereopsis“Discovery” of Stereopsis, 1838
• Charles Wheatstone– Prolific scientist, Wheatstone bridge
• “… the mind perceives an object of three-dimensions by means of the two dissimilar pictures projected by it on the two retinae…”, 1838
• Contributions to the Physiology of Vision.—Part the First. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision.
• "Philosophical Transactions" of the Royal Society of London, Vol. 128, pp. 371 – 394, 1838.
• http://www.stereoscopy.com/library/wheatstone-paper1838.html
What the left and right eye see (retinal image)
![Page 63: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/63.jpg)
Getting Different Images to Each Eye Wheatstone’s Stereoscope
• “The stereoscope is represented by figs. 8. and 9; the former being a front view, and the latter a plan of the instrument. A A' are two plane mirrors, about four inches square, inserted in frames, …”
• Below, modern mirror stereoscope using computer monitors
![Page 64: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/64.jpg)
Getting Different Images to Each Eyethese days
• Commodity televisions– 120 hz display rate– Alternately, left then right
eye image– Active lcd glasses,
alternately block left eye, then right eye image
• lcd lens is alternately transparent, then opague
• 60 hz left eye, • 60 hz right eye
• Theaters– Polarized glasses
![Page 65: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/65.jpg)
Getting Different Images to Each Eye Sutherland’s 1960’s equipment
• “Ultimate display”, 1965
• Sword of Damocles – 1st HMD– Actual camera-like shutters– Actual camera-like metal shutters
![Page 66: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/66.jpg)
Immersive 3D Displays(head mounted displays)
• Head position is tracked, and display updated for user view– As if “in a virtual (cg) world”
Head mounted display, NASA Ames, Fisher et al., mid-1980’s
![Page 67: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/67.jpg)
Also, Virtual and Augmented Reality
• Augmented reality shows real world with an overlay of additional overlay
• Knowlton (1975)
• Partially-silvered mirror over keyboard
• Programmable labels
• Tactile feedback
![Page 68: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/68.jpg)
Also, Virtual and Augmented Reality, 2
• Enables users to see real world with an overlay of additional interaction– Situational awareness
• See through glasses
• Typically, add text+images to real world
• Very sensitive to tracking
• These days, “mobile apps” are appearing
![Page 69: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/69.jpg)
Immersive 3D DisplaysSurround Screen Systems
• Essentially same multi-surface display and interaction paradigm used today as 1992
• Orders of magnitude less hardware and software cost
![Page 70: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/70.jpg)
Immersive 3D Displays Surround Screen Systems
• Space with walls and/or floor formed by rear projection screens
– Stereoscopic – Head and position tracking
• So can change 3D view (projection) as user moves, or moves head
• Projects 3D scenes for viewer’s point of view on walls
– Walls “vanish”, • User perceives full 3D scene
– View only correct for one viewer
• Visual immersion– Field of view is 100% possible
![Page 71: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/71.jpg)
Immersive 3D Displays Surround Screen Systems
• Typical size: 10’ x 10’ x 10’
• 2 or 3 walls are rear projection screens
– To save space, mirrors often used to “fold” projection
– Floor projected from above
![Page 72: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/72.jpg)
UTPA Immersive Systems Lab~Summer, 2013 … or so
Proj.
Proj. Proj.
27’13’
Security area
21’
6’
TV
stor
age
~8’
CAVE
Ph
ysio
log
ica
lM
easu
rem
ent
E
qu
ipm
ent
Computers
Front Projection Screen
Proj.
Develo
pmen
t
![Page 73: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/73.jpg)
Current Immersive Display Projects
• Information visualization in immersive environments
• Role of stereoscopy, display size, and narrative on presence
• Role of personality factors in presence
• Gallery installation
• Getting things to work …– Clusters, projectors, software
![Page 74: Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649e615503460f94b5c52a/html5/thumbnails/74.jpg)
Questions?