rtc building the metaverse apr21 - khronos group · 2021. 4. 28. · the metaverse? 3d application...
TRANSCRIPT
© The Khronos® Group Inc. 2021 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License
Building the Metaverse One Standard at a TimeNeil TrevettPresident, KhronosVice President Developer Ecosystems, NVIDIA
© The Khronos® Group Inc. 2021 - Page 2This work is licensed under a Creative Commons Attribution 4.0 International License
Topics
1. Just what do we mean by ‘the metaverse’?
2. How will it come into existence?
3. What role will standards play – and NOT play!
4. Which Khronos standards can help build the metaverse?
© The Khronos® Group Inc. 2021 - Page 3This work is licensed under a Creative Commons Attribution 4.0 International License
Discussions on connecting ecosystems with interoperable runtimes and economies for portable assets and avatars
Fortnight, PUBG, Valheim
Virtual spaces where gamers create their own content
Roblox, Minecraft, Manticore
Digital twins eventually encompassing the world
GE, Siemens, Bentley, NVIDIA
1992Humans, as avatars, interacting with each other and software agents, in a
three-dimensional space that uses the metaphor of the real world
What is the Metaverse?
3D application interoperability for real-time immersive collaborative simulation and design
NVIDIA Omniverse
Spatially indexing and streaming the digitized world
OGC, Cesium
A spatial evolution of the Web
Augmenting Reality with persistent geo-locking, linking, occlusion and realistic scene illumination
PTC, Open AR Cloud, Niantic
VR environments where users can import their own authored assets
Microsoft Flight Simulator
© The Khronos® Group Inc. 2021 - Page 4This work is licensed under a Creative Commons Attribution 4.0 International License
How Will the Metaverse Evolve?
Evolution will be organicDarwinian mix of ideas, innovations, commercial successes and failuresOverlapping, messy, synergistic,
awesome experimentation
Proven solutions will emerge and be
widely adoptedProviding the foundation for the next round of innovation
What role can Open Standards play in this process?
Not as a single platformNo one design or roadmap can serve all
needs and solve all problems …Synthetic visual reality, practical XR optics,
effective remote social interactions, server scaling to millions of users, online personas and social
connections, realistic avatars, security, privacy, sharable assets, interoperable run-times, online economies and currencies, real-world XR geo-
anchoring with persistence, universal digital twins, IOT sensor networks, pervasive low-latency
wireless connectivity, pervasive 3D commerce, real-time environment scanning and semantic
understanding… and many more …Significant opportunities for all companies
contributing to metaverse solutions!
© The Khronos® Group Inc. 2021 - Page 5This work is licensed under a Creative Commons Attribution 4.0 International License
What is an Open Standard?INTEROPERABILITY standards define precise COMMUNICATION
Through a specification that enables multiple conformant implementations
E.g., Software <-> Hardware Communication through an Application Programming Interface (API)
True OPEN standards are not controlled by any single companyStandards Developing Organizations (SDOs) enable industry cooperation
Inclusive participation, well-defined governance and intellectual property frameworksStronger industry consensus and participation increases potential Adoption
Growing MarketsBy reducing consumer confusion and increasing
capabilities and usability
Reducing CostsBy sharing development between many
companies and driving volume
Speeding Time to MarketWith well-proven functionality, testing
and interoperability
Enabling InnovationCompanies compete on implementationquality, performance, power etc. etc.
Successful Open Standards Share Core Goals
© The Khronos® Group Inc. 2021 - Page 6This work is licensed under a Creative Commons Attribution 4.0 International License
Don’t Need Standards?Making the metaverse
ubiquitous will involve a constellation of standards!
IEC 60038 Standard voltagesIEC 60228 Conductors of insulated cables
IEC 60269 Low-voltage power fusesIEC 60320 C13 Connectors and C14 Inlets
IEC 60884 Household Plugs And Socket-OutletsIEC 61970 APIs for energy management systems
Widely adopted platforms require multiple standards
Standards are the basis for ubiquitous infrastructure
© The Khronos® Group Inc. 2021 - Page 7This work is licensed under a Creative Commons Attribution 4.0 International License
Proprietary Products and Technologies
Rapid InnovationDarwinian testing ground
Smart Innovators can retain long-term advantage
Complex and Interdependent
Relationship
Technology becomes ProvenBusiness interests are better served
by cooperation than competition
Areas of Emerging Consensus Beachhead standardization opportunities
Proprietary Technology and Open Standards
Multi-company GovernanceOpen Standards
Do not R&D!!Satisfy wider need for technologies
Thread of continuity for industry forward progress
Standards Move the Industry Forward Frees competitive energy to find new ways
to innovate value
© The Khronos® Group Inc. 2021 - Page 8This work is licensed under a Creative Commons Attribution 4.0 International License
Open Standards and Open Source
Open Standard = Shared Specification
Implementation
Implementation Implementation
Implementation
Implementation
Best When …Competitive advantage in implementation innovation
Industry needs multiple implementationsNeed a stable design target
But..Can take time to generate consensus on a new version
Conformance testing is vital
Not the same thing! Both are powerful tools in the right situationChoose the right tool to make best forward progress
Often positive synergy between the two e.g.,Open-source sample implementations of open standards
Tools, samples, conformance tests for open standards in open source
Contributor
Contributor
Contributor
Contributor
Contributor
Open Source = Shared Implementation
Best When …No competitive advantage in implementation
Industry consensus to share implementation resourcingNeed rapid updates
But..Can fork and fragment
Need governance model clarity
Many open-source projects have concentrated control – not multi-company governance
© The Khronos® Group Inc. 2021 - Page 9This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Standards as Metaverse Components
High-performance, low-latency 3D Graphics
Pervasive, universal 3D assetsEnabling reliable deployment of 3D in e-commerce at industrial scale
Vision and sensor processing Accelerated machine
learning and inferencing
Portable interaction with VR/AR sensor, haptic and
display devices
© The Khronos® Group Inc. 2021 - Page 10This work is licensed under a Creative Commons Attribution 4.0 International License
XR Portability
OpenXR provides cross-platform, high-performance access directly into XR device runtimes across multiple platforms
XR App/Experience
GameEngine
Application Interface
WebXR
XR App/Experience
XR App/Experience
AndMore!
Virtual Reality
V
AAugmented Reality
© The Khronos® Group Inc. 2021 - Page 11This work is licensed under a Creative Commons Attribution 4.0 International License
Advanced UI Cross-Vendor OpenXR ExtensionsDevelopers can build cross-platform applications that useadvanced UI solutions from different technology vendors
OpenXR API layers can be used implement extensions
Eye Tracking
Eye gaze interaction for intuitive interfaces 2-Step Interaction
Hand-eye coordinationNatural aiming
Shipping on HoloLens 2
Hand Tracking
26 unique joints per hand for fully articulated hands visible to the user
Shipping on HoloLens 2and Microsoft Hand Mesh Extension
for HoloLens 2 layers over it
Ultraleap developer preview available
© The Khronos® Group Inc. 2021 - Page 12This work is licensed under a Creative Commons Attribution 4.0 International License
OpenXR Widespread Industry AdoptionConformant Implementations
Microsoft HoloLens 2Windows Mixed Reality Headsets
Oculus Rift SQuest and Quest 2 HTC Vive Cosmos
Developer Preview Implementations
Valve SteamVRTransitioning from OpenVR to OpenXR
VarjoPreliminary support shipping now
CollaboraOpen-source Implementation
Engine Support
Unreal EngineSupport in 4.24. Optimizations in 4.25
Unity EngineOpenXR Plugin Preview in 2020.2+
Chromium 81 for Chrome and Edge BrowsersOpenXR default backend for WebXR
© The Khronos® Group Inc. 2021 - Page 13This work is licensed under a Creative Commons Attribution 4.0 International License
The Web Will be Part of the Metaverse
https://xkcd.com/1367/
© The Khronos® Group Inc. 2021 - Page 14This work is licensed under a Creative Commons Attribution 4.0 International License
Bringing XR to the Web
Web XR AppsNative XR Apps
Native 3D Engines
Web 3D Engines
Khronos provides the foundation for native and Web-based 3D/XR
Close cooperation between WebXR
and OpenXR
Lifting OpenXR functionality into
the Web stack
© The Khronos® Group Inc. 2021 - Page 15This work is licensed under a Creative Commons Attribution 4.0 International License
glTF – The JPEG of 3D!
Efficient, reliable transmission
Bring 3D assets into 100s of apps and engines
glTF 2.0 – June 2017Native AND Web APIsPhysically Based Rendering
Metallic-Roughness and Specular-Glossiness
glTF 1.0 – December 2015Primarily for WebGL
Uses GLSL for materials
glTF spec development on open GitHub – get involved!
https://github.com/KhronosGroup/glTF
Compact to TransmitSimple and Fast to LoadDescribes Full Scenes
Runtime NeutralOpen and Extensible
Audio Video Images 3D
MP3 H.264 JPEG
New market opportunities for 3D content creation
and deployment!
© The Khronos® Group Inc. 2021 - Page 16This work is licensed under a Creative Commons Attribution 4.0 International License
https://github.khronos.org/glTF-Project-Explorer/
© The Khronos® Group Inc. 2021 - Page 17This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Universal GPU Textures & PBR Roadmap
KTX 2.0 textures supportBasis Universal supercompression
20th April 2021glTF assets with compact, high-quality textures
that can be efficiently converted to GPU compressed textures on-the-fly on diverse target platforms to reduce both asset file download sizes AND GPU memory usage
https://www.khronos.org/ktx/
Future Waves of glTF PBR Extensions
Subsurface Scattering, Attenuation,
Index of Refraction (IOR), Thickness, Specular Color, Anisotropy, Translucency,
Thin Film (iridescence) and more…
glTF PBR Extensions
December 2020Transmission
ClearcoatSheen
© The Khronos® Group Inc. 2021 - Page 18This work is licensed under a Creative Commons Attribution 4.0 International License
Using Open Standards to Build the Metaverse• Seek friction points in your industry caused by lack of agreed communication protocols- Between people, companies, products, subsystems, components- Is the potential solution to better communication well understood?
• Find or create a safe space for industry cooperation to define interoperability standards- With well defined, consensus-based governance model and effective IP framework- Standardize when the need is wide – not when doing R&D
• Neil Trevett- [email protected] | @neilt3d
Gather industryrequirements for future
open standards
Draft SpecificationsConfidential to Khronos
members
Publicly Release Specifications and Conformance Tests
Gain early insights into industry trends and directions
Influence the design and direction of key open standards that will drive your business
Accelerate your time-to-market with early access to specification drafts
Network with domain experts from diverse companies in your industry
State-of-the-art IP Framework protects your Intellectual Property
Enhance your company reputation as an industry leader through Khronos participation
Benefits of Khronos membership www.khronos.org