rising trends in 3d user interfaces trends in 3d user interfaces introduction to qt 3d & qt...
TRANSCRIPT
Rising trends in 3D User InterfacesIntroduction to Qt 3D & Qt Canvas3D
Nils Christian Roscher-NielsenProduct Manager, The Qt Company
Content
• Trends in 3D
• Qt 3D Value Proposition
• Highlights of Qt 3D features
• The road ahead
© 20152
Trends in 3D
3
© 20154
Industries embracing 3D user interfaces
Industrial Automation
Geological Modeling
Car IVI Systems Medical / Learning
3D Modeling / Animations Product Visualization
Virtual Reality
GOOGLE CARDBOARD
STEAM VR / HTC VIVE SONY MORPHEUSOCULUS RIFT
SAMSUNG GEAR VR
© 20155
Augmented / Mixed Reality
© Microsoft © Magic Leap
© 20156
360º Imaging & Video
Google CARDBOARD & JUMPwith GoPro Odyssey
RICOHTHETA
NOKIA OZO
© 20157
3D scanning
© 20158
9
• Games• Virtual Tours - Google Street View • 360 videos - YouTube and Facebook• 3D Design and engineering – SpaceX, Autodesk• Real Estate - Matterport Inc• Teaching - Case Western Reserve University • Automotive – Mini Goggles
Virtual & Augmented Reality Examples of Use Cases
© Mini
© Matterport Inc
© Microsoft
© 2015
10
• Head movement• Hand gestures• Torso tracking• Controllers
• Mouse• Game controllers (with
movement tracking)
• Speech
Interaction with VR & AR
© Oculus Rift
© 2015
3D Printing
© Microsoft © Formlabs
© 201511
© 201513
Pioneer in Modern User Experiences
• For the past 20 years, Qt has delivered all the tools and libraries for creating the best possible UX for your end users• Classic desktop look-and-feel• Fluent, modern touch-based UX
• Today, the future of UIs is going towards 3D• Qt is pioneering there with you!
• It has been possible to integrate 3D content with Qt UIs easily• Qt is now providing means to create 3D content easily
• Qt Canvas3D• Qt 3D
New 3D features in Qt
Qt Canvas3D module• About WebGL• About Canvas3D• three.js
Qt3D Module• Key Features• Building Blocks• Future of Qt 3D
© 201514
Building Blocks for Qt and 3D
Summary
2015
Building Blocks for Qt and 3D
Qt Scenegraph / OpenGL
Tools&
Integration to 3rd Party
Tools
Data Visualization
Data VisualizationCanvas3DCanvas3DQt 3DQt 3D
Integration to 3rd Party 3D
Engines
Integration to 3rd Party 3D
Engines
ServicesServices
Focus of this presentation
© 201516
New 3D features in Qt 5.5
• Focus on making integration of 3D content and creation of 3D user interfaces easier than ever
• Qt Canvas3D 1.0• WebGL based 3D canvas for Qt Quick• 3D content creation with JavaScript• Integration to popular three.js 3D library
• Qt 3D 2.0 Technology Preview• High level 3D API built on top Qt & OpenGL• C++ and Qt Quick APIs• Use any rendering technique available
Canvas3DNew Qt Quick Element in Qt 5.5 for 3D Drawing
© 201518
About WebGL – What Is It?
WebGLA low level, state based, 3D vector graphics rendering API for HTML JavaScriptOften described as “OpenGL ES 2 for the web”
Khronos Group• Non-profit technology consortium that manages WebGL API (plus OpenGL/ES etc.)• WebGL 1.0 is based on OpenGL ES 2.0• Initial WebGL Standard Release in 2011• Stable Release 1.0.2 in 2013• https://www.khronos.org/registry/webgl/specs/latest/1.0/
About WebGL – Browser Support
DesktopGoogle Chrome 9Mozilla Firefox 4.0Safari 6.0Opera 11Internet Explorer 11
MobileSafari from iOS 8 Android BrowserGoogle Chrome 25Internet Explorer on WP 8.1Firefox for MobileFirefox OSTizenUbuntu Touch…
© 201519
Widely supported in modern web browsers:
Motivation for Qt Canvas 3D
WebGL is a nice and productive environment for implementing 3D contentQtQuick is a nice and productive environment for doing 2/2.5D UI→ Combining these two makes for a very productive environment
There is a lot of innovation around WebGL, allowing porting of that content to QtQuick on top of Canvas3D makes a lot of sense.
Qt Canvas3D allows developers to make 3D QtQuick applications that re-use existing 3D assets.
There are a lot of resources on the web to get started with WebGL, these resources translate to Qt Canvas3D as well.
All developers who know WebGL can easily use Qt Canvas3D efficiently
© 201520
21
• Qt Canvas 3D• WebGL based 3D canvas for Qt Quick• 3D content creation with JavaScript• Requires OpenGL (ES) 2.0 or higher
• Integration to three.js 3D library• Increases the abstraction level• Utilizes the innovation around WebGL
• Qt 5.6 new features• Qt Quick items as textures• Rendering directly to Qt Quick scene background or foreground
Qt Canvas
What is Canvas3D?
• Qt 5.5 module• 3D canvas component• Used in a QtQuick scene. • Requires OpenGL or OpenGL ES.
Canvas3D provides• WebGL like (non-conformant)
context API• All the functions from WebGL 1.0.3
© 201522
© 201523
About the Implementation
Implementation is based on analysis of the behavior of typical WebGL apps. There are two stages to all the WebGL apps:
1. Initialization (initializeGL):• Start loading resources (textures, models etc.)• Note: Loading is asynchronous, so rendering WILL start before all resources are loaded.
Must be handled gracefully in the rendering code.2. Drawing (paintGL):
• Handle inputs, state changes, transitions and animations.• Render each frame of 3D content animation.
You get two signals to QtQuick that you handle in your JavaScript code initializeGL and paintGL
Incidentally, this is a familiar pattern from QOpenGLWidget :initializeGL(), paintGL(), resizeGL()
© 201524
About the Implementation
In HTML WebGL canvas element has separated “logical layout size” and “physical pixel size”.This is also reflected in the design and behavior of the Canvas3D.
onResizeGL signal notifies whenever the canvas width, height or devicePixelRatio changes, this maps to onWindowResize handlers in most HTML/WebGL content.
WebGL/HTML Canvas3D/QtQuick
Layout Size canvas.style.widthcanvas.style.height
canvas.widthcanvas.height
Drawing Size canvas.widthcanvas.height
canvas.pixelSize.widthcanvas.pixelSize.height
three.js
© 201526
Motivation
Writing low level OpenGL code is not very productive.
A 3D scene graph library helps with productivity by offering higher level abstraction:• Load a 3D model from a file and place it in the scene.• Place the lights in the scene.• Place the camera in the scene.• DONE! Start rendering the content..
What is three.js?
three.js Upstream:• Three.js is one of the most active open source WebGL
based scene graph libraries.• Implemented 100% with JavaScript on top of WebGL• A lot of WebGL content on the web uses three.js.• See: http://threejs.org
three.js Canvas3D Port:• Enables easy porting of three.js content
from the Web to QtQuick• Port 3D content, Implement 2D UI with QtQuick,
Enhance with QtQuick transitions and animations.
© 201527
© 201528
What’s Next?
Canvas3D 1.0 in Qt 5.5• Uses the new TypedArrays from QtQuick JavaScript Engine (V4VM)
Will compile and run only on Qt 5.5Uses less memory and performs even better than the technology preview versions
• Porting some of the Khronos WebGL conformance tests and fixing bugs found in those tests.• Implementing three.js examples for release with Qt 5.5.• Bugfixing…
Qt 5.6• Aiming to bundle three.js as part of Qt as atested release.• Looking at 3D tooling to make the developer experience smoother• Prototyping move of the rendering to the QtQuick scene graph rendering thread, this would allow us to
integrate Qt Multimedia and rendering of QtQuick elements as textures. It is still early days, but looking quite promising..
Qt 5.7 and beyond• We are working with the Automotive industry—and all of our customers—to create a great roadmap.
Qt 3D 2.0Technology Preview in Qt 5.5
© 201530
Motivation for Qt 3D
• Modern GPU’s are very powerful and offer fantastic capabilities • Beyond what could be done with Qt3D 1.x.
• Modern CPU’s on all platforms are now multicore designs• Qt3D 1.x engine could not leverage this.
• Leverage modern rendering algorithms like deferred rendering• Not supported in Qt3D 1.x.
Qt3D 2.0 is a completely re-engineered and re-written Qt3D module. The goal is to make Qt 3D more relevant in the modern world.
© 201531
Qt 3D Key Features
• Extensible and flexible architecture• Fully configurable renderer that allows to quickly implement any
rendering pipeline that you need• 3D content adaptation can be done during runtime or based on
environment• Scalability from OpenGL 2.0 to OpenGL 4.3• C++ and QML language API’s• Rendering tasks can be distributed across several GPU cores by a
scheduler for improved performance
© 201532
Qt 3D 2.0 Building Blocks
• Based on a framework that maintains the object graph using queue of jobs that can be spread to multiple execution threads.
• The object graph can be parsed to build the rendering graph (for visual elements) or e.g. to build a audio graph (for audible elements).
• This flexible framework allows plugging in new “aspects” to the engine like audio or physics.
http://www.kdab.com/overview-qt3d-2-0-part-1
© 201533
Qt 3D 2.0 Building Blocks
• Qt 3D 2.0 is fully data driven• You can define how the rendering of the scene is done using render graphs.
• Any new 3D rendering algorithm that appears in the latest scientific papers can be implemented with Qt 3D 2.0.
• Qt 3D enables easy use of advanced OpenGL programming • Instancing.
What’s Next?
• Qt 3D 2.0 Technology Preview has been released• API can change based on user feedback• Current platform support is limited (mainly tested on desktop only)• Parts of documentation are still missing• Second technology preview available with Qt 5.6
• Target for Qt 3D 2.0 final is Qt 5.7• Qt 5.6 will already bring many improvements. • API will be finalized• Input methods (touch, mouse and keyboard) improved• Integration with Qt Quick 2 content improved• Picking—select specific 3D objects from a scene• Documentation and examples improved
Summary
36
The near future will bring big changes to the industries• How we interact with technology changes• User experiences are becoming more immersive• 3D content creation and consumption will change radically
Qt is a technology that actively takes part in the development and enable our users to create the technology of tomorrow
Summary
© 201537
Summary
• Many leading 3D design tools and 3D solutions have been made with Qt
• Qt has the building blocks that make integrating 3D / OpenGL content easy
• Qt 5.5 introduced 2 new 3D technologies.• Qt Canvas3D & Qt 3D
• High level API for defining e.g. 3D objects, lights and cameras• Extensible architecture• Wide support for 3D formats (through assimp and three.js libraries)• Integration of 3D content and creation of 3D user interfaces is super
© 201538
Comparison between Canvas3D and Qt3D
• Canvas3D• QML/JavaScript, low level API• Provides an easy and widely used solution when combined with three.js 3D
library• Possibility to utilize the application code also in Hybrid and Web applications
• Qt3D• Both QML and C++ APIs• Extensible architecture multithreaded architecture• Qt3D 2.0 technology preview is available in 5.5 and 5.6.
