2004 Stefan Seipel, IT, Uppsala University
VR based HCITechniques & Application
May 14, [email protected]
2004 Stefan Seipel, IT, Uppsala University
What is Virtual Reality ?
Coates (1992):Virtual Reality is electronic simulations of environments experienced via headmounted eye goggles and wired clothing enabling the end user to interact in realistic three-dimensional situations.
Greenbaum (1992):Virtual Reality is an alternate world filled with computer-generated images that respondto human movements. These simulated environments are usually visited with the aid ofan expensive data suit which features stereophonic video goggles and fiber-optic gloves.
Krueger (1991):….The term (virtual worlds) typically refers to three-dimensional realities implemented with stereo viewing goggles and reality gloves.
2004 Stefan Seipel, IT, Uppsala University
Variables to define Virtual Reality
Vividness(richness of an environments representation)
• breadth (visibility, audibility, touch, smell)• depth (quality, fidelity)
Interactivity (extend to which a user can modify form and content of a mediated environment)
• speed (update rates, time lag)• mapping (text, speech, gestures, gaze, complex behavior patterns)
2004 Stefan Seipel, IT, Uppsala University
Classification of Virtual Reality and other Media
high
high
low Interactivity
Vividness
book
35mm film
3D IMAX
TV pay-TV
VR
photo-graph
tama-gotchi
Jonathan Steuer
phone
videoconferencing
Sensorama
2004 Stefan Seipel, IT, Uppsala University
History of Virtual Reality(technological milestones)
1956 Sensorama (Morton Heilig)3D visuals, vibration, stereo sound, wind, smell, little interaction
1961 Headsight System (Philco Corp.)HMD, head tracking, remote video camera, telepresence
1965 The Ultimate Display (Ivan Sutherland)Stereoscopic HMD, computer generated images, tracking, visually coupled system
1967 Grope (University of North Carolina)6 degree of freedom force feedback
1977 The Sayre Glove (Sandin, Sayre, DeFanti Univ. Illinois)Gesture recognition
1987 Virtual Cockpit (British Aerospace)head and hand tracking, eye tracking, 3d visuals, 3D audio, speech recognitionvibro tactile feedback
2004 Stefan Seipel, IT, Uppsala University
VR VR -- (visually) coupled systems(visually) coupled systems
computersimulation
multimodalinput
multimediafeedback
real world
replication
real-time
2004 Stefan Seipel, IT, Uppsala University
How Do We Perceive 3D ?
2004 Stefan Seipel, IT, Uppsala University
How Do We Perceive 3D ?
Visual depth cues:
a) monoscopic cuesrelative sizeinterposition and occlusionperspective distortionlighting and shadowstexture gradientmotion parallax
b) binocular (stereoscopic) cuesstereodisparityconvergence
2004 Stefan Seipel, IT, Uppsala University
Binocular VisionBinocular Vision
Binocular focal zonePeriphery field of view
Horizontal disparityConvergence
2004 Stefan Seipel, IT, Uppsala University
How can we recreate 3D sensation ?
Providing visual cues:
a) monoscopic cuesrealistic rendering / lighting simulation
b) stereoscopic cues -> stereodisparitypresentation of appropriate view to each eye - time multiplexing of images- multiplexing with chromatic filters (anaglyph)- multiplexing with polarizer filters- providing two views simultaneously
2004 Stefan Seipel, IT, Uppsala University
Realistic 3D rendering
http://www.agh.edu.pl/htdocs/gifs/computer_art/raytracing/final18.jpg
2004 Stefan Seipel, IT, Uppsala University
Stereoscopic rendering
2004 Stefan Seipel, IT, Uppsala University
Time Multiplexed Stereo Image Pair
V-Sync at 60 Hz
Addidional V-Sync at 120 Hz(enforced with sync. doubler)
open
clos
e
clos
e
open
Active Shutter Glasses(LCD-Shutters)
2004 Stefan Seipel, IT, Uppsala University
Dual Channel Head Mounted Display (HMD)(© nVision)
Datavisor 80
Datavisor
2004 Stefan Seipel, IT, Uppsala University
What Renders VR Applications Specific ?
1. Visual and Acoustic Realism of Objects• 3D effect• level of detail• specularity• color and texture
2. Real-time Response (approx. >15 Hz)
3. Natural Like Interaction Metaphors(many degrees of freedom input)
4. Peripheral Visual Stimuli
2004 Stefan Seipel, IT, Uppsala University
System Architecture in Virtual Environments
DisplayDevices
SensorDevices
Simulation ManagerScene GeneratorSimulation Loop
RenderEngine
SensorHandler
User
Scene DB
2004 Stefan Seipel, IT, Uppsala University
Types of Virtual Reality Environments
1. Immersive Virtual Environments• subjects are visually isolated from the real environment• virtual scene is responding to the subjects actions• subjects are unable to perform in the real environment
2. Semi-Immersive Virtual Environments• subjects can perform both in the real and virtual environment• subjects perceive a strong involvement into the VE• subjects may perform less in the real environment
3. Non-Immersive Virtual Environments• the three-dimensional scene is considered as a part of
the physical environment• subjects do fully respond in the real environment• relatively little involvement into the VE
(4. Augmented Reality Interfaces)
Deg
ree
of I
mm
ersi
onC
onsciousness in Physical Environment
2004 Stefan Seipel, IT, Uppsala University
Display Devices
Visual Displays (3D imagery)• Head Mounted Displays (HMD)• Projection Displays (CAVE, Virtual Plane)
Acoustic Displays (spatial sound)• Multi-Channel Sound Systems• Specialized Convolution Processors (e.g. Convolvotron)
Haptic Displays (force feedback)• Robot Arms (e.g. Grope, Phantom)• Active Joystics (e.g. Microsoft Sidewinder)• Vibrotactile Devices (e.g. Logitec Cyberman)
2004 Stefan Seipel, IT, Uppsala University
An Immersive Car Simulator Using HMD(© British Aerospace)
2004 Stefan Seipel, IT, Uppsala University
A BOOM Display Application in Aerodynamics(© NASA Ames Research Center)
2004 Stefan Seipel, IT, Uppsala University
CAVE - An Immersive VR Environment(EVL, University of Illinois at Chicago)
2004 Stefan Seipel, IT, Uppsala UniversityPlanung - Behandlung - Ausbildung
Six-sided CAVE (KTH, Stockholm)
2004 Stefan Seipel, IT, Uppsala University
The ImmersaDesk - A Semi-Immersive Device(University of Illinois at Chicago)
2004 Stefan Seipel, IT, Uppsala University
Non-Immersive Desktop VR : 3D Implant Planning(© 1995 CMD, Uppsala University)
2004 Stefan Seipel, IT, Uppsala University
The Haptic Display Grope III(© University of North Carolina)
2004 Stefan Seipel, IT, Uppsala University
Examples of Haptic Devices
Low Cost Force Feedback Device
High Fidelity Force Feedback Devices
PHANToM ©SensableDevices
2004 Stefan Seipel, IT, Uppsala University
The Virtual Workbench(© 1998 Kent Ridge Digital Labs (KRDL), Singapore)
2004 Stefan Seipel, IT, Uppsala University
Sensor Devices
1. Spatial Position/Orientation Sensors• 2DOF (Mouse)• 3DOF (Microscribe, FreeD Joystick)• 6DOF (Polhemus Fastrack)
2. Directional Force Sensors• 5 DOF (Spacemouse)• 2 DOF (Joystick)
3. Gesture Recognition• Data Gloves
4. Eye Tracking
5. Speech Recognition Systems
2004 Stefan Seipel, IT, Uppsala University
Example : Spatial Position/Orientation Sensors
Polhemus InsideTrack
FreeD Joystick(UltraSonic Tracking)
MicroScribe(Mechanical Tracking)
(Magnetic Tracking)
2004 Stefan Seipel, IT, Uppsala University
Example : Directional Force Sensors
SpaceMaster
SpaceBall 2003
SpaceBall 3003
2004 Stefan Seipel, IT, Uppsala University
Example : Gesture Recognition
Cyberglove, 5th Dimension SUPERGLOVE, Nissho
Dextrous Hand Master, Exos
2004 Stefan Seipel, IT, Uppsala University
Example : Eye Tracking Systems
http://psych.utoronto.ca/~reingold/eyelink/eyelink.htm
2004 Stefan Seipel, IT, Uppsala University
The Role of Dynamic Perspective
2004 Stefan Seipel, IT, Uppsala University
The Role of Dynamic Perspective
The alternate-ego view The primary-ego view
2004 Stefan Seipel, IT, Uppsala University
Examples of DesktopVR Applications(from the Medical Field)
Typical single user applicationsSimulation based trainingSimulation based planning
2004 Stefan Seipel, IT, Uppsala University
Planning and Rapid Prototyping Planning and Rapid Prototyping TheThe SurgicaseSurgicase SystemSystem
1999-2000 http://www.materialise.be/surgicase/
• Manual segmentation• Surface Rendering• Rapid prototyping
2004 Stefan Seipel, IT, Uppsala University
1999 S. Haßfeld et al.Mund-, Kiefer- und Gesichtschirurgie, Universität Heidelberg
Automatic SegmentationAutomatic Segmentation• Automatic segmentation
of mandibular nerve• Hybrid rendering
2004 Stefan Seipel, IT, Uppsala University
Registration and Visualizationof Prosthetic Design
• CT Scanning of jaws• CT Scanning of dentures• Correlationof datasets• Planning with regard toprosthetic restoration
1998 K. Verstreken et al., Laboratory for Medical ImagingResearch (ESAT & Radiology), K.U.Leuven/Belgium
2004 Stefan Seipel, IT, Uppsala University
HapticHaptic Simulators for Training of Simulators for Training of Tooth BrushingTooth Brushing
Quelle: http://wwwipr.ira.uka.de/~salb
2004 Stefan Seipel, IT, Uppsala UniversityQuelle: http://wwwipr.ira.uka.de/~salb
Haptic ManipulatorVisual Simulation
HapticHaptic Simulators for Training of Simulators for Training of Tooth BrushingTooth Brushing
2004 Stefan Seipel, IT, Uppsala University
Computer manufactured mechanical templates
• Stereo-lithography• Machine milling
Planning CAM Template Surgery
http://www.materialise.be/surgicase/
2004 Stefan Seipel, IT, Uppsala UniversityQuelle : http://cs.millersv.edu/haptics/lumbar/lumbar.html
A PrototypeA Prototype HapticHaptic LumbarLumbarPuncture SimulatorPuncture Simulator
A Prototype Haptic Lumbar Puncture Simulator, Paul Gorman1, Thomas Krummel1, Roger Webster2, Monica Smith2, David Hutchens2 1Department of Surgery, School of Medicine Stanford University2Department of Computer Science School of Science and Mathematics Millersville University of Pennsylvania
2004 Stefan Seipel, IT, Uppsala University
A PrototypeA Prototype Haptic Haptic Suturing SimulatorSuturing Simulator
A Prototype Haptic Suturing Simulator Randy Haluck, M.D.1, Roger Webster, Ph.D.2,Will Wang, M.D.1, Dean Zimmerman2, Betty Mohler2, Alan Synder, Ph.D.1, Mike Melkonian, M.D.1 1Department of Minimally Invasive Surgery Penn State University College of Medicine 2Department of Computer Science School of Science and Mathematics Millersville University of Pennsylvania
Quelle : http://cs.millersv.edu/haptics/suture/suture.html
2004 Stefan Seipel, IT, Uppsala University
Dynamic Tissue SimulationDynamic Tissue Simulation
Quelle : http://cs.millersv.edu/haptics/suture/suture.html
2004 Stefan Seipel, IT, Uppsala University
Examples of Augmented Reality Interfaces
Typical single user applicationsRequires user and data registration
2004 Stefan Seipel, IT, Uppsala University
ArtmaArtma
Interactive Consultation (ref. University of Vienna,Clinic for Oral and maxillofacial Surgery
Augmented Instrument Navigation(Artma, Vienna) www.artma.com
2004 Stefan Seipel, IT, Uppsala University
• Registration of view on real world
• Correlation with computer data
• Fusion of real world view withcomputer generated information
Augmented Vision Guidance
Department of Computer Science, University of North Carolina at Chapel Hill
Department of Computer Science, University of North Carolina at Chapel Hill
2004 Stefan Seipel, IT, Uppsala University
Augmented reality to support navigation
Control of High-Speed Boats
Navigational data is presented in different systems
Visual Integration of navigation data is desirable
2004 Stefan Seipel, IT, Uppsala University
Examples of Collaborative 3D Interfaces
Multiple Simultaneous Usersa) Spatially distributedb) Spatially co-located
2004 Stefan Seipel, IT, Uppsala University
Applied CG&VR Research
Multi-User Collaborative Shared Virtual Environments
Virtual Teaching Settings for Learning
low-bandwidth protocols for network VR
intelligent clients rather than full state replication
transformation driven state propagation
highly aggregated objects for net VR
2004 Stefan Seipel, IT, Uppsala University
Virtual Teaching in Computer Graphics Education
2004 Stefan Seipel, IT, Uppsala University
Virtual Teaching in the Dental Curriculum
2004 Stefan Seipel, IT, Uppsala University
The Stereoscopic Powerwall(© 1999 CMD, Uppsala University)
2004 Stefan Seipel, IT, Uppsala University
3D Jaw Occlusion Analysis
Question: Where and when occur interferences in jaw occlusion?
Problem : The real world is not accessible to the naked eye
2004 Stefan Seipel, IT, Uppsala University
Operation Planning(FMV/FHS)
The AQUARIUMA highly parallel visual environment
2004 Stefan Seipel, IT, Uppsala University
Visualizations in the AQURARIUM
2004 Stefan Seipel, IT, Uppsala University
Visualizations in the AQUARIUM
2004 Stefan Seipel, IT, Uppsala University
Using 3D Interfaces To Enhance Screen-Space Utilization
2004 Stefan Seipel, IT, Uppsala University
3D GUI for mobile computing
Physical client area = virtual client area : 640x480 pixels
2004 Stefan Seipel, IT, Uppsala University
Display tilt at 15 degrees
Virtual client area : 640x526 pixels
2004 Stefan Seipel, IT, Uppsala University
Display tilt at 30 degrees
Virtual client area : 640x626 pixels
2004 Stefan Seipel, IT, Uppsala University
Display tilt at 45 degrees
Virtual client area : 640x844 pixels
2004 Stefan Seipel, IT, Uppsala University
Decision Support in Process Industry(Scanraff)
Visualizing process variables in time and context3D tilted process graphs