nvidia cloudxr - on-demand.gputechconf.comdecode hevc frame sensor & input data rgb frame audio...
TRANSCRIPT
David Weinstein, 19 December 2019
NVIDIA CLOUDXR™
2
Gaming
Movies
Sports
Concerts
Travel Retail
PERSONAL ENTERTAINMENT
VR
3NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
QUADRO PROFESSIONAL VR/AR
MEDICAL ARCHITECTURE
DESIGN MANUFACTURING COLLABORATION
DIGITAL CONTENT CREATION
4
5
6
7
FUNDAMENTAL VR GRAPHICS CHALLENGES
TRADITIONAL = 60 MP/S(1920 X 1080 @ 30 FPS)
1920
1080
VIRTUAL REALITY = 450 MP/S(3024 X 1680* @ 90 FPS)
1512
1680
1512 Motion to Photon: ≤ 20 ms
Very High Pixel Rendering Throughput Must Maintain Very Low Latency
Pixel Throughput and Perceived Latency
HMD updates pose
CPU triggers redraw
GPU computes new image
HMD flashes display
8
WANTED: THE BEST OF BOTH WORLDS
Highest Quality Graphics + Mobile Freedom
9
CLOUDXR VISION
RTX ServersContent Network
Clients
XR Streaming to Any Device
10
MOBILE WORLD CONGRESS (LA), OCT 2019
CloudXR Announcement
11
ARCHITECTURE OVERVIEWServer-Side Driver, Client-Side Application and SDK
OpenVR Runtime (Valve)
VR App
Virtual HMD Driver
Cloud
Late & Lens Warp
VR Runtime
Video Decoder
Cloud
HMD
Video Encoder
Virtual Audio Driver
Audio Encoder Audio Decoder
InternetSERVER driver (Nvidia) CLIENT app (Nvidia)
audio, video, haptics
pose, control inputs, video
12
ARCHITECTUREWIN SERVER CLIENT
INST
ALL
ER
Audio/Video
Data (sensors, inputs, QoS)
Encode
RGB Frame
Display
VR App
Decode
Vid
eo P
ara
ms
(e.g
. RO
I, o
pti
on
al)
HEVC Frame HEVC Frame
VR Runtime
Controller & inputs
Sensor,Pose &Controller Data
RGB Frame
Sensor & input data
Sensor & input data
Audio
(e.g., OpenVR)
Ap
p
Clie
nt
app
Serv
er d
rive
r
Time Warp
Lens Warp
RGB Frame
Driver API
VR Runtime
Runtime API
OpenVR Interfaces
HMD
Au
dio
Pla
ybac
k
Audio Capture / Encode
13
QOS
Relevant Internet problems:
• Jitter
• Missed packets
• Bandwidth variation
Reliability & Resilience
14
QOS
Reliability & Resilience
Challenges Solutions Control parameters
Latency Custom HEVC profile NVENC knobs
Jitter De-jitter buffer Buffer size
Missed packets Forward error correction Correction strength
Bandwidth variation Video compression Compression rate
Server has no display Frame pacing to match
client display rate
Server render rate
15
WIN SERVER
VR App
Vid
eo P
ara
ms
(e.g
. RO
I, o
pti
on
al)
Ap
p
Driver API
VR Runtime
Runtime API
ARCHITECTURE
INST
ALL
ER
Audio / Video / Data (sensors, inputs, QoS)
Encode
RGB Frame
Decode
HEVC Frame
Sensor,Pose &Controller data
Sensor & input data
Audio
Clie
nt
app
Serv
er d
rive
r
Audio Capture / Encode
Au
dio
Pla
ybac
k
Frame Pacing
Jitter FIFO
CLIENT
Display
VR Runtime
Controller & inputs
(e.g., OpenVR)Time Warp
Lens Warp
HMD
OpenVR Interfaces
16
LATENCY
Using:
• HelloVR sample modified to change color on button press
• Latency meter (instrumented switch + photodiode)
• Local Cloud with Client on WiFi (average ping time 4 ms)
Button to Photons
17
LATENCY
Results:
Server Driver + Client App latency = 12.1 ms.
The Jitter FIFO can be downsized at the cost of occasional missed frames.
Button to Photons
Latency [ms] Stdev
Local VR 38.3 3.4
CloudXR 81.6 3.3
Diff 43.3 -0.1
Jitter FIFO + Ping 31.2
Diff – Above 12.1
*Average of 40 clicks
18
WIN SERVER
VR App
Vid
eo P
ara
ms
(e.g
. RO
I, o
pti
on
al)
Ap
p
Driver API
VR Runtime
Runtime API
LATENCY
INST
ALL
ER
Audio / Video / Data (sensors, inputs, QoS)
Encode
RGB Frame
Decode
HEVC Frame
RGB FrameSensor & input data
Audio
Clie
nt
app
Serv
er d
rive
r
Audio Capture / Encode
Au
dio
Pla
ybac
k
Frame Pacing
Jitter FIFO
CLIENT
Display
VR Runtime
Controller & inputs
(e.g., OpenVR)Time Warp
Lens Warp
HMD
3-10 ms
10-50 ms(depending on network)
1-3 ms
3-10 ms
OpenVR Interfaces < 0.5 ms
19
20
Total Flexibility
CLOUDXR CONFIGURATIONS
5G
Ethernet
WiFi
21
EARLY ACCESS PROGRAMTest CLOUDXR for VR
@ https://developer.nvidia.com/nvidia-cloudxr-sdk-early-access-program
22
NVIDIA 5G MEC XR DEVKIT
23
THE OPPORTUNITY: VR/AR/VGPU OVER 5GA Convergence of Visual Compute + 5G and a New Edge Topology
Powerful GPUsIncredible compute capabilities redefining
graphics
Bandwidth and Low LatencyBoth are required for VR/AR
Compute is at the EdgeVery few hops – Low Latency
24
ENTERPRISE MARKETSSuccess Across Verticals
ManufacturingEngineering, Design, Marketing/Sales
AECEntire Design Process,
Sales, Moving to Construction
M&EVirtual Production
TrainingPossibly the Largest
Market
Enscape
Nurulize IC.IDO
Autodesk
Location-Based EntertainmentEnterprise/Consumer
ESI
25
ENTERPRISE AR
DIGITAL MODELS ANNOTATEDINSTRUCTION/COLLABORATION
Driving Value Propositions
26
THE FUTURE OF MOBILE XR
• Ultra Low Latency
• High Bandwidth
• Cutting-edge Compute
• Cost Effective Delivery
• Valuable Use Cases
Requirements:
5G + VR/AR + vGPU at the Edge
27
RTX Performance in a Server
• Powered by Quadro RTX 8000/6000
• Ray Traced Global Illumination up to 96 GB Scenes
• Remoting and Multi-GPU Virtualization with Quadro virtual Datacenter Work Station (vDWS)
• All the technical advantages of Turing
NVIDIA RTX SERVER
28
XR/VGPU 5G EDGE STACKSoftware/Hardware Stack Demands a Diverse Effort
Cloud Orchestration
Containerized Edge Applications
Low Latency Streaming
Visual Computing and AI
5G transmission
5G Network
Virtualization
Enabling SDKs
Fundamental GPUDrivers
29
COME TRY THE DEMO ®ISTER ONLINE FOR EARLY ACCESS
https://developer.nvidia.com/nvidia-cloudxr-sdk-early-access-program
Visit the VR Village on the Expo Floor