Download - Week14 VR AR
-
7/29/2019 Week14 VR AR
1/26
Combining real imagery withcomputer generated imagery
Virtual reality;
Augmented reality;Teleorobotics
-
7/29/2019 Week14 VR AR
2/26
Combining real imagery with
computer generated imagery Robot-assisted surgery
Virtual real estate tours
Virtual medical tours
Urban planning
Map-assisted navigation Computer games
-
7/29/2019 Week14 VR AR
3/26
Virtual image of real data3D sensed data can bestudied for surgical paths tobe followed by a surgeon ora robot.
In the future, real-timesensing and registration canbe used for feedback in theprocess.
-
7/29/2019 Week14 VR AR
4/26
Human operating in a real
environment: brain surgery.
All objects are real; we cook food, chopwood, do brain surgery
-
7/29/2019 Week14 VR AR
5/26
Most computer games / videos
are entirely virtualIMMERSION, or engagement, canbe very high, however, with
Quality spatial resolutionStereo
Smooth motion
Little time delay between userinteractions and visual effects
Synchronized audio and forcefeedback are important
Courtesy of University ofWashington HIT Lab
-
7/29/2019 Week14 VR AR
6/26
Virtual immersive environments
-
7/29/2019 Week14 VR AR
7/26
Virtual environment schematic
Example: nurse gets training on giving injections usinga system with stereo imagery and haptic feedback
-
7/29/2019 Week14 VR AR
8/26
Virtual dextrous work
Medical personnel practice surgery or injection, etc. Artist can carvea virtual 3D object. Haptic system pushes back on tool appropriateto its penetration (intersection) of the model space. Users free hand
grabs a physical arm model under the table in injection training.
http://www.sensable.com/products-haptic-devices.htm
-
7/29/2019 Week14 VR AR
9/26
Augmented reality: views of
real objects + augmentation
-
7/29/2019 Week14 VR AR
10/26
AR in teleconferencing person works at real desk
remote collaboratorrepresented by picture or
video or talking head
objects of discussion; e.g.a patients brain image,might also be fused intovisual field
HOW IS THIS ACHIEVED?
From University of Washington HIT Lab
-
7/29/2019 Week14 VR AR
11/26
Imagine the virtual book Real book with empty identifiable pages
AR headset
Pay and download a story
System presents new stereo imageswhen the pages are turned
Is this better than a .pdf file?
Is this better than stereo .pdf?
-
7/29/2019 Week14 VR AR
12/26
Human operating with AR
Think of a heads up display on your auto windshield, or on theinstrument panel. What could be there to help you navigate?
(Vectors to nearby eating places? Blinking objects we mightcollide with? Congestion of nearby intersections? Web pages?)
-
7/29/2019 Week14 VR AR
13/26
Special devices needed to fuse/register real and
generated images
Human sees real environment optics design problem
Human sees graphicsgenerated from 3D/2D models computer graphics problem
Graphics system needs toknow how the human isviewing the 3D environmentdifficult pose sensing problem From University of Washington HIT Lab.
-
7/29/2019 Week14 VR AR
14/26
Devices that support AR
Need to fuse imagery;
Need to compute pose of userrelative to the real world
-
7/29/2019 Week14 VR AR
15/26
Fusing CAD models with real env.
Plumber marks the wall where the CAD blueprint shows thepipe to be.
-
7/29/2019 Week14 VR AR
16/26
Two types of HMD
-
7/29/2019 Week14 VR AR
17/26
Difficult augmentation problem
How does computer system know where toplace the graphic overlay?
Human very sensitive to misregistration Some applications OK such as circuit board
inspection.
Can use trackers on HMD to give approximate
head pose Tough calibration procedures for individuals
(see Charles Owens work)
-
7/29/2019 Week14 VR AR
18/26
Teleoperation
remotely guided police robot moves a suspected bomb
teleoperated robot cleans up nuclear reactor problem
surgeon in US performs surgery on a patient in France
Dr in Lansing does breast exam on woman in Escanaba (work
of Mutka, Xi, Mukergee, et al.)
-
7/29/2019 Week14 VR AR
19/26
Teleoperation on power lines
-
7/29/2019 Week14 VR AR
20/26
Face2face mobiletelecommunication
Concept HMD at left; actual images from our prototype HMD at right.
Problem is to communicate the face to a remote communicator.
-
7/29/2019 Week14 VR AR
21/26
Reddy/Stockman used geometrictransformation and mosaicking
Which 2 are real
video frames andwhich arecomposed of 2transformed andmosaicked views?
-
7/29/2019 Week14 VR AR
22/26
Miguel Figueroas system
Face image is fit as a blendof basis faces from trainingimages
c1F1+c2F2+ cnFn
Coefficients [c1, c2, , cn]sent to receiver embeddedin the voice encoding.
Receiver already has thebasis vectors F1, F2, , Fnand a mapping from sideview to frontal view and canreconstruct the currentframe.
-
7/29/2019 Week14 VR AR
23/26
Actual prototype in operation
Mirror size is exaggerated in these images by perspective;however they are larger than desired. Consider using theMotorola headsets that football coaches use with tinycamera on the microphone boom.
-
7/29/2019 Week14 VR AR
24/26
Captured side view projectedonto basis of training samples
-
7/29/2019 Week14 VR AR
25/26
Frontal views contructed bymapping from side views
This approach avoids geometrical reconstruction of distorted left andright face parts by using AAM methods -- training and mapping.
-
7/29/2019 Week14 VR AR
26/26
Summary of issues
All systems (VR,AR,TO) require sensingof human actions or robot actions
All systems need models of objects orthe environment
Difficult registration accuracy problem
for AR, especially for see-throughdisplays, where the fusion is done inthe humans visual system