seminario marco agus, 4-10-2012
Post on 18-May-2015
595 Views
Preview:
DESCRIPTION
TRANSCRIPT
www.crs4.it/vic/
Technologies for improving real-time exploration of massive ((volumetricvolumetric))
models
Marco Agus[magus@crs4.it]
CRS4Visual Computing
Cagliari, October 2012
M. Agus, Technologies for improving massive models understanding, October 2012
We focus on…….
• Advanced technologies for massive models rendering on light field displays
– Joint work with J. A. Iglesias Guitian, E. Gobbetti, F. Marton, F. Bettio, G. Pintore, R. Pintus, A. Zorcolo
M. Agus, Technologies for improving massive models understanding, October 2012
Technology pillars
• Software: Scalable techniques for interactive rendering of massive models– State of the art methods able
to handle potentially infinite static surface and volume to handle potentially infinite static surface and volume models
• Hardware: Autostereoscopic light field displays– They are now able to render
the perceptual aura of real 3D objects
– They provide real compelling 3D experience without glasses
M. Agus, Technologies for improving massive models understanding, October 2012
Research challenges• Light field display driving
– How to project geometry on such kind of systems?
• Image representation
– How to create effective light fields from 3D massive data?
– How to deal with volumes, surfaces, video streams?
• Evaluation
– Do light field displays improve perceptive cues and immersiveness?– Do light field displays improve perceptive cues and immersiveness?
– How to improve visual confort and depth discrimination?
• Interaction
– How to exploit light field displays to create natural interactionsystems?
– How to exploit display characteristics to provide meaningfulinformation?
• Applications
– How these kind of displays can be useful?
M. Agus, Technologies for improving massive models understanding, October 2012
Main Contributions
José Antonio Iglesias Guitián, Enrico Gobbetti, and Fabio Marton. View-dependent Exploration of Massive Volumetric Mo dels on Large Scale Light Field Displays . The Visual Computer, 26(6--8): 1037-1047, 2010
Marco Agus, Fabio Bettio, Andrea Giachetti, Enrico Gobbetti, José Guitián, Fabio Marton, Jonas Nilsson, and Giovanni Pintore.An interactive 3D medical visualization system base d on a light field display. The Visual Computer, 25(9): 883-893, 2009
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, Fabio Marton, and Giovanni Pintore. GPU Accelerated Direct Volume Rendering on an Inter active Light Field Display. Computer Graphics Forum, 27(3): 231-240, 2008. Proc. Eurographics 2008.
Fabio Marton, Marco Agus, Enrico Gobbetti, Giovanni Pintore, and Marcos Balsa. Natural exploration of 3D massive models on large-s cale light field displays using the FOX proximal na vigation techniqueComputers & Graphics, 2012.
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, and Fabio Marton. Evaluating layout discrimination capabilities of co ntinuous and discrete automultiscopic displays.In Proc. Fourth International Symposium on 3D Data Processing, Visualization and Transmission, 2010.
Fabio Marton, Enrico Gobbetti, Fabio Bettio, José Antonio Iglesias Guitián, and Ruggero Pintus. A Real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field displa y. In Proc. 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2010
Marco Agus, Giovanni Pintore, Fabio Marton, Enrico Gobbetti, and Antonio Zorcolo. Visual enhancements for improved interactive render ing on light field displays.In Eurographics Italian Chapter Conference, November 2011.
M. Agus, Technologies for improving massive models understanding, October 2012
Main Contributions
José Antonio Iglesias Guitián, Enrico Gobbetti, and Fabio Marton. View-dependent Exploration of Massive Volumetric Mo dels on Large Scale Light Field Displays . The Visual Computer, 26(6--8): 1037-1047, 2010
Marco Agus, Fabio Bettio, Andrea Giachetti, Enrico Gobbetti, José Guitián, Fabio Marton, Jonas Nilsson, and Giovanni Pintore.An interactive 3D medical visualization system base d on a light field display. The Visual Computer, 25(9): 883-893, 2009
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, Fabio Marton, and Giovanni Pintore. GPU Accelerated Direct Volume Rendering on an Inter active Light Field Display. Computer Graphics Forum, 27(3): 231-240, 2008. Proc. Eurographics 2008.
VOLUMES
DATA
Fabio Marton, Marco Agus, Enrico Gobbetti, Giovanni Pintore, and Marcos Balsa. Natural exploration of 3D massive models on large-s cale light field displays using the FOX proximal na vigation techniqueComputers & Graphics, 2012.
Fabio Marton, Enrico Gobbetti, Fabio Bettio, José Antonio Iglesias Guitián, and Ruggero Pintus. A Real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field displa y. In Proc. 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2010
Marco Agus, Giovanni Pintore, Fabio Marton, Enrico Gobbetti, and Antonio Zorcolo. Visual enhancements for improved interactive render ing on light field displays.In Eurographics Italian Chapter Conference, November 2011.
VIDEO STREAMS
SURFACES
M. Agus, Technologies for improving massive models understanding, October 2012
Main Contributions
José Antonio Iglesias Guitián, Enrico Gobbetti, and Fabio Marton. View-dependent Exploration of Massive Volumetric Mo dels on Large Scale Light Field Displays . The Visual Computer, 26(6--8): 1037-1047, 2010
Marco Agus, Fabio Bettio, Andrea Giachetti, Enrico Gobbetti, José Guitián, Fabio Marton, Jonas Nilsson, and Giovanni Pintore.An interactive 3D medical visualization system base d on a light field display. The Visual Computer, 25(9): 883-893, 2009
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, Fabio Marton, and Giovanni Pintore. GPU Accelerated Direct Volume Rendering on an Inter active Light Field Display. Computer Graphics Forum, 27(3): 231-240, 2008. Proc. Eurographics 2008.
MEDICAL
APPLICATIONS
Fabio Marton, Marco Agus, Enrico Gobbetti, Giovanni Pintore, and Marcos Balsa. Natural exploration of 3D massive models on large-s cale light field displays using the FOX proximal na vigation techniqueComputers & Graphics, 2012.
Fabio Marton, Enrico Gobbetti, Fabio Bettio, José Antonio Iglesias Guitián, and Ruggero Pintus. A Real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field displa y. In Proc. 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2010
Marco Agus, Giovanni Pintore, Fabio Marton, Enrico Gobbetti, and Antonio Zorcolo. Visual enhancements for improved interactive render ing on light field displays.In Eurographics Italian Chapter Conference, November 2011.
3D - TV
VIRTUAL
MUSEUMS
M. Agus, Technologies for improving massive models understanding, October 2012
Main Contributions
Marco Agus, Fabio Bettio, Andrea Giachetti, Enrico Gobbetti, José Guitián, Fabio Marton, Jonas Nilsson, and Giovanni Pintore.An interactive 3D medical visualization system base d on a light field display. The Visual Computer, 25(9): 883-893, 2009
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, Fabio Marton, and Giovanni Pintore. GPU Accelerated Direct Volume Rendering on an Inter active Light Field Display. Computer Graphics Forum, 27(3): 231-240, 2008. Proc. Eurographics 2008.
USER PERFORMANCE
+
HUMAN FACTORS
Fabio Marton, Marco Agus, Enrico Gobbetti, Giovanni Pintore, and Marcos Balsa. Natural exploration of 3D massive models on large-s cale light field displays using the FOX proximal na vigation techniqueComputers & Graphics, 2012.
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, and Fabio Marton. Evaluating layout discrimination capabilities of co ntinuous and discrete automultiscopic displays.In Proc. Fourth International Symposium on 3D Data Processing, Visualization and Transmission, 2010.
Marco Agus, Giovanni Pintore, Fabio Marton, Enrico Gobbetti, and Antonio Zorcolo. Visual enhancements for improved interactive render ing on light field displays.In Eurographics Italian Chapter Conference, November 2011.
PERCEPTION
INTERACTION
M. Agus, Technologies for improving massive models understanding, October 2012
Light-field display: overview
• The key feature characterizing 3D light field displays is direction-selective light emission
• 3D display hardware based on commercially available commercially available technology developed by Holografikahttp://www.holografika.com– Specially arranged projector
array and a holographic screen
– Side mirrors increase the available light beams count
– Each projector emits light beams toward a subset of the points of the holographic screen
M. Agus, Technologies for improving massive models understanding, October 2012
Physical behavior
• In the horizontal direction, selective light transmission
• Vertically, the screen scatters widely (diffuse behavior)
Projector
• Results in homogeneous light distribution and continuous 3D view
Screen
Light field
M. Agus, Technologies for improving massive models understanding, October 2012
• In practice, at any moment in time, a given screen pixel has the same color when viewed from all vertical viewing angles
• In order to provide a full perspective effect, the vertical viewing angle must thus be shown.
Projection technique
viewing angle must thus be shown. We thus introduce a “virtual observer”, fixing the viewer height and distance from screen
• The resulting MCOP technique is exact for all viewers at the same distance from the screen and height as the virtual observer. It proves in practice to be a good approximation for all viewing positions in the display workspace.
M. Agus, Technologies for improving massive models understanding, October 2012
Depth-dependent resolution
• The display design has consequences not only on the projection equation but also imposes limits on the spatial resolution that depends on depth
M. Agus, E. Gobbetti, J.A. Iglesias Guitián, F. Marton, and G. Pintore. GPU Accelerated Direct Volume Rendering on an Inter active Light Field Display. Computer Graphics Forum, 27(3), 2008. Proc. Eurographics 2008.
• In general, the size of the smallest feature that can be reproduced depends on the distance of its center from the screen and from the beam angular size
M. Agus, Technologies for improving massive models understanding, October 2012
Rendering System overview
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
Parallel rendering
• We employ a GPU cluster for rendering.
• Sort first parallel rendering approach
– Adaptive out-of-core GPU rendering vs Replicating data
– Static assignment: rendering process images.
• Good load balancing. Caused by the geometry of the display, • Good load balancing. Caused by the geometry of the display, with all projectors looking at the same portion of the model.
M. Agus, Technologies for improving massive models understanding, October 2012
Light field displays considered
• Two light field displays are considered:– Large-scale model capable of visualizing
35Mpixels by composing images generated by 72 SVGA LED projectors.
• The screen has 160x90cm, 50◦ horiz. field-of view with 0.8◦ angular accuracy.
• Pixel size on the screen surface is 1.5mm.
• Rendering cluster: 18 Athlon64 3300 + Linux PCs equipped with two NVidia 8800GTS 640MB.PCs equipped with two NVidia 8800GTS 640MB.
• GPUs are several generations older. Slowdown of 2x−3x
– Small-scale model capable of visualizing 7Mpixels by composing images generated by 96 320x240 small CCDs
• The screen is 26 inches, 50 degrees horiz. field-of view with 0.8◦ angular accuracy.
• The display is driven by one linux PC equipped with one NVidia 8800GTS 640MB.
M. Agus, Technologies for improving massive models understanding, October 2012
Massive volume rendering
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
Massive volume rendering
• Full-system architecture overview
APPLICATION: APPLICATION: Massive volume
rendering
José Antonio Iglesias Guitián, Enrico Gobbetti, and Fabio Marton. View-dependent Exploration of Massive Volumetric Mo dels on Large Scale Light Field Displays . The Visual Computer, 26(6--8): 1037-1047, 2010
M. Agus, Technologies for improving massive models understanding, October 2012
• Use CPU for …
– Creation & loading
– Octree refinement
– Encode current cut using an spatial index
• Use GPU for …
Technique overview
• Use GPU for …
– Stackless octree traversal
– Rendering
Enrico Gobbetti, Fabio Marton, and José Antonio Iglesias Guitián. A single-pass GPU ray casting framework for interactive out-of-core rendering of massive volumetric datasets.The Visual Computer, 24, 2008. Proc. CGI 2008.
M. Agus, Technologies for improving massive models understanding, October 2012
Taking advantage of GPGPU
• Improved multi-resolution CUDA ray caster:
– Flexible ray traversal and compositing strategies
– Improved visibility feedback (e.g. scatter writes)scatter writes)
– Pre-integrated transfer-functions
– Integrate an adaptive frame reconstruction scheme
M. Agus, Technologies for improving massive models understanding, October 2012
adaptive loaderpreprocessing
visibility
feedbackoctree refinement
[ creation and maintainance ] [ rendering ]
offli
ne
Method overview
volume
render
storage
octree node
database
has current working set enough accuracy?
yes
prepare to render
no
GPUCPU
offli
ne
M. Agus, Technologies for improving massive models understanding, October 2012
Preview: massive volumes rendering
M. Agus, Technologies for improving massive models understanding, October 2012
Massive surface rendering
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
Massive surface rendering
• Full-system architecture overview
APPLICATION: APPLICATION: Virtual exploration
of massive surface models
Fabio Marton, Marco Agus, Enrico Gobbetti, Giovanni Pintore, and Marcos Balsa Rodriguez. Natural exploration of 3D massive models on large-s cale light field displays using the FOX proximal navigation te chnique. Computers & Graphics, 2012.
M. Agus, Technologies for improving massive models understanding, October 2012
Handling massive models: overview
• Multiresolution structure based on a modification of Adaptive Tetra Puzzles
– Few Ktri Patches
– Optimized GPU/CPU communication
– Per-patch spatial index to organize patches in triangle strippatches in triangle strip
Giovanni Pintore, Enrico Gobbetti, Fabio Marton, Russell Turner, and Roberto Combet. An Application of Multiresolution Massive Surface Representations to the Simulation of Asteroid Missi ons . In Eurographics Italian Chapter Conference. Pages 9-16. Eurographics Association, November 2010.
Paolo Cignoni, Fabio Ganovelli, Enrico Gobbetti, Fabio Marton, Federico Ponchio, and Roberto Scopigno. Adaptive TetraPuzzles - Efficient Out-of-core Construc tion and Visualization of Gigantic Polygonal Models. ACM Transactions on Graphics, 23(3): 796-803, August 2004. Proc. SIGGRAPH 2004.
M. Agus, Technologies for improving massive models understanding, October 2012
Handling massive models
• Offline construction
– spatial partition by longest edge bisection of tetrahedra
– fine-to-coarse parallel out-of-core simplification of the surface contained in diamondsdiamonds
• Run-time rendering
– selective refinement queries based on projected space error estimation on tetrahedron hierarchy
– adaptive loader rapidly produces LOD by combining precomputed patches
M. Agus, Technologies for improving massive models understanding, October 2012
Results: massive models rendering
M. Agus, Technologies for improving massive models understanding, October 2012
Multiview capture and rendering
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
Multiview capture and rendering
• Full-system architecture overview
APPLICATION: Multiview capture Multiview capture
and rendering system
Fabio Marton, Enrico Gobbetti, Fabio Bettio, José Guitián, and Ruggero Pintus. A Real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field display. In Proc. 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). 2011
M. Agus, Technologies for improving massive models understanding, October 2012
Capture
• Simple front-end: a single PC acquires M-PEG video stream from low cost USB camera array.
• Each node need to receive all camera images since each projectors sees a images since each projectors sees a large portion of the display workspace
• Solution: front-end sends single-frame JPEG images through UDP multicast protocol to display cluster nodes
29
M. Agus, Technologies for improving massive models understanding, October 2012
Upload to GPU
• Minimization of CPU work
– Moreover, old PCs not fast enough for JPG decoding.
• Pipelined CPU-GPU parallel decoding, interleaving:
– CPU:entropy decoding using libjpeg-turbo
– GPU: CUDA – GPU: CUDA
• dequantization,
• inverse-DCT,
• YCbCr to RGB conversion
– 6X speed-up wrt standard CPU solution
– Fast but still dominates the whole processingtime using old G80 GPUs!
30
CPUImg N-1
CPUImg 1
CPUImg 0
GPUImg 0
GPUImg N-2
GPUImg N-1
M. Agus, Technologies for improving massive models understanding, October 2012
Depth estimation (CUDA)
• For each pixel of each projector
– Find depth and color
– identify a ray using equations which consider MCOP geometry of the display.
– identify the closest cameras to the ray using the camera viewing transforms.using the camera viewing transforms.
• Plane sweeping to find best depth
– Evaluate pixel similarities at different depths and keep the best one
31
M. Agus, Technologies for improving massive models understanding, October 2012
Coarse to fine depth estimation & color sample
• Start at half depth with coarser resolution
• Upsample and refine
• Filter each level
• Up to final resolution
• Final color gathering• Final color gathering
32
M. Agus, Technologies for improving massive models understanding, October 2012
Results: Multiview Capture System
M. Agus, Technologies for improving massive models understanding, October 2012
Results: “First teleport experiment”
M. Agus, Technologies for improving massive models understanding, October 2012
Illustrative methods
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
Illustrative methods
• Full-system architecture overview
FOCUS: FOCUS: Improve volume understanding
José Antonio Iglesias Guitián, Enrico Gobbetti, and Fabio Marton. View-dependent Exploration of Massive Volumetric Mo dels on Large Scale Light Field Displays . The Visual Computer, 26(6--8): 1037-1047, 2010
M. Agus, Technologies for improving massive models understanding, October 2012
Novel view-dependent illustrative tools
• The view-dependent characteristics of the display can be exploited to develop specialized interactive illustrative techniques designed to improve spatial understanding
• Simple head motions can reveal new aspects of the inspected data
M. Agus, Technologies for improving massive models understanding, October 2012
Clip-plane with view-dependent context
• Traditional cut away visualization, when our view direction is orthogonal to the clip plane, while offering more helpful contextual information in other situations
M. Agus, Technologies for improving massive models understanding, October 2012
Clip-plane with view-dependent context
• Compute distance from plane
• If distance is positive
• modify the opacity of samples by multiplying it by a view-dependent correction factor:
• Otherwise
• vary the opacity of the plane and shading parameters from the original ones at to full opacity and ambient plus emission shading at
M. Agus, Technologies for improving massive models understanding, October 2012
Clip-plane with view-dependent context
M. Agus, Technologies for improving massive models understanding, October 2012
Other view-dependent illustrative tools
• Context-preserving probe
• Band-picker• Band-picker
• More info in...
José Iglesias Guitián, Enrico Gobbetti and Fabio Marton.View-dependent exploration of massive volumetric models on large-scale light field displays.The Visual Computer, 26, 2010. Proc. CGI 2010.
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
FOCUS: FOCUS: Perceptual and performance
evaluation
Marco Agus, Enrico Gobbetti, José Antonio Iglesias Guitián, and Fabio Marton. Evaluating layout discrimination capabilities of continuous and discrete automultiscopic displays . In Proc. Fourth International Symposium on 3D Data Processing, Visualization and Transmission, 2010.
M. Agus, Technologies for improving massive models understanding, October 2012
• The main goal of tests performed in the evaluation process is …
– … to elucidate if light field displays
Evaluation
– … to elucidate if light field displays could provide visual information not available with traditional volume rendering systems
• The main focus will be set on psychophysical tests.
M. Agus, Technologies for improving massive models understanding, October 2012
• Stereopsis evaluation– Random dot spiral ramp– Depth discrimination
(overlapping disks)
• Spatial understanding
Evaluation
• Spatial understanding evaluation– Path tracing
performance evaluation
M. Agus, Technologies for improving massive models understanding, October 2012
Evaluation results
Users rapidly recover all depth cues to instantaneously recognize complex structures
– Very useful for analysis of angiography datasets
• More details about the evaluation tests can be found in …M. Agus, F. Bettio, A. Giachetti, E. Gobbetti, J. A. Iglesias Guitián, F. Marton, J. Nilsson, and G. Pintore. An interactive 3D medical visualization system based on a light field display. The Visual Computer, Vol. 25, No. 9, pp. 883–893, 2009.
M. Agus, E. Gobbetti, J. A. Iglesias Guitián, F. Marton. Evaluating layout discrimination capabilities ofcontinuous and discrete automultiscopic displays. 3DPVT, 2010
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
FOCUS: Display adaptation
Marco Agus, Giovanni Pintore, Fabio Marton, Enrico Gobbetti, and Antonio Zorcolo. Visual enhancements for improved interactive render ing on light field displays.In Eurographics Italian Chapter Conference. Pages 1-7. Eurographics Association, November 2011.
M. Agus, Technologies for improving massive models understanding, October 2012
Problem: visual discomfort
• Aliasing artifacts occur when objects are too far with respect to the screen
• Causes
– Geometry errors due to the discrete characteristics of projectors and displayprojectors and display
– Image-based calibration error
• subpixel errors on the display screen
• bigger errors as the distance from screen increases
M. Agus, Technologies for improving massive models understanding, October 2012
Visual discomfort: our solution
• Non-linear depth remapping method
– constrains most of the scene to stay inside CVR
– drives users to focus on parts inside CVR
Depth-of-field simulation • Depth-of-field simulation method
– blurs parts of the scene in background
• Frame fade-out method
– reduces clipping artifacts due to the borders of the light field display
M. Agus, Technologies for improving massive models understanding, October 2012
Non-linear depth remapping method
• Implemented on a vertex shader according to the equation
• B, F are the comfort thresholds
• DF, DB are the asymptotic output ranges
M. Agus, Technologies for improving massive models understanding, October 2012
Depth of field simulation
• Depth-dependent blur to adapt the frequency content to display resolution
– Focused objects are sharp within CVR, around focal plane
– Background objects are instead blurred
– Objects close to the viewer cannot be blurred
• Implemented as image-based two-pass DOF simulation [Nguyen 2007]
– CoC proportional to display spatial resolution
– Post-processing pixel shader blending between hi-res and low-res images according to CoC and blurriness by stochastic Poisson sampling
M. Agus, Technologies for improving massive models understanding, October 2012
Frame fade-out
• Limited angular workspace
– Objects can abruptly disappear while viewer moves horizontally
• Simple but effective solution
– Color blending which fades to background in boundary areas
– Objects smoothly fade out– Objects smoothly fade out
– Implemented in a fragment shader
M. Agus, Technologies for improving massive models understanding, October 2012
Qualitative results
• Non-linear depth-mapping (front positions)
Without adaptation
Depth mapping DF = 1500 mm
Depth mapping DF = 500 mm
M. Agus, Technologies for improving massive models understanding, October 2012
Qualitative results
• Non-linear depth-mapping + DOF blur
Without adaptation
Depth mapping DB = 1000 mm
Depth mapping DB = 1000 mm + DOF
M. Agus, Technologies for improving massive models understanding, October 2012
Qualitative results
Without adaptation
Depth mapping DF = 1000 mm
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
M. Agus, Technologies for improving massive models understanding, October 2012
System overview
• Full-system architecture overview
FOCUS: FOCUS: natural interaction
Fabio Marton, Marco Agus, Giovanni Pintore, and Enrico Gobbetti.FOX: The Focus Sliding Surface Metaphor for Natural Exploration of Massive Models on Large-scale Light FieldDisplays . In Proc. VRCAI. Pages 83-90, December 2011.
M. Agus, Technologies for improving massive models understanding, October 2012
Natural interaction: requirements
• Light field display constraints
– Depth-dependent spatial resolution, calibration errors, angular boundsangular bounds
• Interaction metaphor should be simple
– Museum context
– Reduced number of DOFs
– Short learning time
M. Agus, Technologies for improving massive models understanding, October 2012
FOX interface: key ideas
• Guided motion
– Scene is kept centered wrt display workspace
• DOF reduction
– Interaction consists of changing a contact point
Focus+
Visual confort
– Translation, rotation and zoom are coupled together
• Natural and intuitive
– Simple to learn: only one button
– It can be implemented with various devices
Ease of use+
Efficiency
M. Agus, Technologies for improving massive models understanding, October 2012
FOX interface: components
• Translation and rotation
• Automatic zooming
Automatic hotspot • Automatic hotspot placement
M. Agus, Technologies for improving massive models understanding, October 2012
Translation and rotation
• Surface slides on display hotspot
– (dx,dy) defines a displacement on current plane
– Closest point and normal is forced to display hotspot
– Up direction is kept
– Smoothing filters
M. Agus, Technologies for improving massive models understanding, October 2012
Automatic zooming
• Speed-dependent automatic zooming
– Vector length dS = |dx,dy| defines rate of motion
– Pan-speed function with three states
– Steady (FOCUS) vs fast motion (EXPLORATION)
M. Agus, Technologies for improving massive models understanding, October 2012
Hot-spot automatic placement
• Coarse sampling for computing average plane
• Hotspot depth corrected to put scene in focus
M. Agus, Technologies for improving massive models understanding, October 2012
Resulting interface
• From device (2D pos + button state) to displacement vector dS
• Model matrix M function of dS and current state (model surface)
• Easy to implement with all pointing devices (2D mouse, 3D mouse, free-hand motion-tracking, touch-screens)
• Cursor glyphs indicating dS and pan-zoom state• Cursor glyphs indicating dS and pan-zoom state
M. Agus, Technologies for improving massive models understanding, October 2012
Kinect implementation
• Hand tracking with gesture recognition (open/close)
– Covariance analysis in order to recognize singular points
M. Agus, Technologies for improving massive models understanding, October 2012
Results
• Real-time exploration of David 0.25 mm model, composed of 970M triangles
• User evaluation:
– Metaphor comparison ( 5DOF ObjectInHand vs FOX)
– Device comparison (IS 9000 vs Kinect) – Device comparison (IS 9000 vs Kinect)
M. Agus, Technologies for improving massive models understanding, October 2012
Interaction with IS device
M. Agus, Technologies for improving massive models understanding, October 2012
Free-hand interaction (Kinect)
M. Agus, Technologies for improving massive models understanding, October 2012
User evaluation
• 33 participants (26 novices, 7 experts)
• Quantitative evaluation
– FOX vs 5DOF, IS vs KINECT
– Guided and explorative tasks
– Task completion time and image quality– Task completion time and image quality
• Qualitative evaluation
– Metaphor comparison (ease of learning, ease of reaching positions, perceived 3D image quality, preferred one)
– Device comparison (ease of learning, ease of reaching positions, preferred one)
M. Agus, Technologies for improving massive models understanding, October 2012
User evaluation: samples
M. Agus, Technologies for improving massive models understanding, October 2012
Results: 3D image quality
M. Agus, Technologies for improving massive models understanding, October 2012
Results: task completion time
M. Agus, Technologies for improving massive models understanding, October 2012
Results: qualitative evaluation
M. Agus, Technologies for improving massive models understanding, October 2012
Results: user preferences
M. Agus, Technologies for improving massive models understanding, October 2012
Discussion
• Tasks are easier with FOX (especially for novice users)
• Overall image quality is sensibly better with FOX
• With respect to devices IS 900 performs better • With respect to devices IS 900 performs better (especially for novice users)
• FOX provides better comfort (even difference in ease of use is not significant)
M. Agus, Technologies for improving massive models understanding, October 2012
Summary of contributions
• Interactive system for virtual exploration of massive models on light field displays
• Light field display adaptation techniques for improving visual perception
• Natural interaction metaphor for light field • Natural interaction metaphor for light field displays
• Perceptual and performance evaluation of light field displays for interactive exploration
M. Agus, Technologies for improving massive models understanding, October 2012
• But there is still a lot of work to do ...– Improve light field representations
• Complex scenes (hybrid scenes, point-based representations, image-based representations)
• Model retargeting within the range of 3d displays
Current and future work
• Model retargeting within the range of 3d displays– Improve the interaction techniques
• e.g. natural interfaces for the manipulation of large models (gesture interfaces)
• Exploration of more complex virtual environments (jumps, fly-through, change of metaphors)
M. Agus, Technologies for improving massive models understanding, October 2012
Take-home messages
• Data explosion ���� Impossible to explore all data
you acquire
• Displays are evolving ���� Many ways to improve
exploration of data and provide useful informationinformation
• Complexity reduction is needed
– Interface � Constrained but natural exploration
– Visual representation � Extraction of meaningful information+ Hints on exploratio + Integration and fusion
M. Agus, Technologies for improving massive models understanding, October 2012
Take-home message
• Simple scene ���� simple
exploration
– Interface can be simple
– User can explore all the scene to reach her target
M. Agus, Technologies for improving massive models understanding, October 2012
Take-home message
• Huge scene ���� Simple or complex interface?
– Where is the Pac-man food?
– How much time would Pac-Man need to complete the maze?
M. Agus, Technologies for improving massive models understanding, October 2012
Marco Agus[magus@crs4.it]
That’s all, folks…Thank you
CRS4CRS4Visual Computing
http://www.crs4.it/vicvic@crs4.it
top related