virtual reality features of nvidia gpus › siggraph › 2016 › ... · 2016-07-28 ·...
TRANSCRIPT
Mark Kilgard, July 27, 2016
Virtual Reality Features ofNVIDIA GPUs
2
Mark Kilgard
Principal System Software Engineer
OpenGL driver implementation & API evolution
Other interests
Programmable shading languages, original Cg GPU shading language
Web, vector graphics & path rendering, GPU rendering for 2D graphics
Also known for creating OpenGL Utility Toolkit (GLUT)
My Background
3
GPU Challenges for Virtual Reality Rendering
1. Must render unique view for each eye
Two scene renders per frame
2. Wide field-of-view rendered
More pixels, distributed towards periphery
3. HMD lens magnifies rendered scene
Non-linear inverse lens warp
Allows the HMD lens magnification to provide convincing sense of presence
4
Magnification Effect of HMD Lens
Graph paper viewed & magnified through HMD lens
5
Head Mounted Display Lens Effect
Barrel Distorted Imageson HMD Screen
Magnified Overlapping Wide Field-of-viewBinocular View
6
Focus on Virtual Reality Features of Pascal GPUs
Virtual Reality rendering features in Pascal GPUs
GeForce 1080/1070/1060
NEW Quadro P6000 & P5000
As exposed in standard 3D APIs
7
Background: NVIDA GPU Architecture Road Map
Our interest NVIDIA GPU architectures of interest: Pascal
What is Pascal mentioned on last slide?
8
Conventional 3D Single-eye Scene
9
Conventional 3D Single-eye Scene
10
Wide Stereo Views of Same Scene
Left Right
11Left Right
12Right Left
13
Left & Right Views AreSimilar but Different
Left and Right eye views have different occlusion
While left & right eye views look similar, must be draw independently
Really only different in a small translation in X
NVIDIA Single Pass Stereo supports generating left & right eye positions
GPU automatically makes left & right eye versions of each primitive
Then renders each view to a different framebuffer slice
Exposed in OpenGL by NV_stereo_view_rendering extension
Exposed for Direct3D 11 by NVAPI SinglePassStereoMode feature
14
Image Difference of Two Views
− + 0.5 =
Left eye view Right eye view
Clamped difference image
15
Left & Right Views Rendered Efficiently in One Pass
16
Lens Matched Shading
Left Right
17
Visualizing Lens Matched Shading
18
Lens Matched Shading
Left Right
19
Visualization of Lens Matched Shading Rendering
20
Still Needs Non-linear Warp
Lens Matched Shadingrendering
Warped image suitablefor HMD display
Non-linearimage
21
Still Needs Non-linear Warp
Lens Matched Shadingrendering
Warped image suitablefor HMD display
Non-linearimage
BUT substantiallyless pixel motion
and distortionthan conventional
VR re-warping
22
Example HMD Post-rendering Warp
23
Lens Matched Shadingwith Window Rectangle Testing
24
Lens Matched Shadingwith Window Rectangle Testing
Nothing in black corners is shaded or even rasterized
Yellow lines show overlaid 8 inclusive window rectanglesSame 8 window rectangles “shared” by each view’s texture array layer
25
Lens Matched Shadingwith Window Rectangle Testing
26
What Didn’t Have to be Rendered
17% of left view pixels 17% of right view pixels
27
Window Rectangle TestingDuring Rasterization
New multi-vendor OpenGL extension
EXT_window_rectangles
Provides a set of window-space rectangles, 8 for NVIDIA GPUs
Rasterizer either includes or excludes rasterization from rectangles
Discards at speed-of-light
As if free to rasterize to irregular rectangular regions
WindowRectangles
Test
NEWstage
OpenGL’s fragment processing pipeline
28
Warped Lens Matched Scene
Warped version of Lens Matched Shading to match HMD lens
29
Warped Lens Matched Shading
30
Visualizing Warp Window Rectangles
31
What Didn’t Have to be Warped
17% of left view pixels 17% of right view pixels
32
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
LMS = Lens Matched Shading
33
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
Displayed within HMD
Single Rendering PassSingle Pass Stereo +
Lens Matched Shading +Window Rectangle Testing
Drawn with Single TriangleFragment Shader WarpingWindow Rectangle Testing
Perception to user is linear rendering
HMD lens “undoes” warping to providea perceived wide field-of-view
Pascal does all thisefficiently in a singlerendering pass!
8 viewports, 1 pass
34
Traditional 3D Rendering Pipeline
Transform3D View
DownsampleDisplayFrame
AppProcessing
if multisampled
uses pinhole camera projection,rasterizing to uniform rectilinear pixel grid
monitor directlydisplays uniform rectilinear pixel grid
Rasterize3D View
35
Virtual Reality Rendering PipelineRendering Left & Right Eye Views
TransformLeft View
Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
TransformRight View
DownsampleRasterize
Right View
HMDWarp
HMDWarp
warps uniform rectilinear grid based on HMD’s device profile
TransformRight View
RasterizeRight View
TransformRight View
DownsampleRasterize
Right ViewTransformRight View
HMDWarp
DownsampleRasterize
Right ViewTransformRight View
second view rendering
36
Virtual Reality Rendering PipelineWith Lens Matched Shading
TransformLeft View
Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
TransformRight View
DownsampleRasterize
Right View
HMDWarp
HMDWarp
LensMatchedShading
LensMatchedShading
replicates primitiveto 4 projected quadrants
less pixel resamplingeffort & error operating onLMS rasterized images
37
Virtual Reality Rendering PipelineWith Single Pass Stereo
TransformLeft & Right
View
Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
DownsampleRasterize
Right View
HMDWarp
HMDWarp
one transform instance,outputs left & right position
38
Virtual Reality Rendering PipelineSingle Pass Stereo + Lens Matched Shading
Downsample
DisplayFrame to HMD
RasterizeLeft View
DownsampleRasterize
Right View
HMDWarp
HMDWarp
LensMatchedShading
LensMatchedShading
TransformLeft & Right
View
AppProcessing
+Advantages1. Single transform for both views2. Closer match between rendering & lens images3. Fewer overall pixels for similar quality
use window rectangle testingto bound lens region, duringboth rasterization & warp
39
Pascal GPU Features for VRUnique to NVIDIA
Pascal renders left & right eye
In single rendering pass
Halves CPU load for rendering
Better concurrent texture locality
OpenGL: NV_stereo_view_rendering
Direct3D: NVAPI SinglePassStereoMode
Better matches rendering to HMD lens
Single pass
Four separately projected quadrants
Less rendering, less resampling error
OpenGL: NV_clip_space_w_scaling
Direct3D: NVAPI ModifiedWSupport
40
Other NVIDIA Virtual Reality Initiatives
41
More Information
VRWORKS 2.0 Software Development Kit
Available NOW
Pascal OpenGL extensions for Virtual Reality
Specification now public
In official OpenGL.org extension registry
OpenGL Extension Wrangler 2.0 (GLEW) support
Example code in VRWORKS 2.0 SDK
VRWORKS also documents NVAPI usage
For Direct3D 11
Includes Examples code