cascades opengl webcast

Building Compelling 3D Apps using Cascades and OpenGL

Roberto Speranza

rsperanza@rim.com | @RSSessantotto

Ramprasad Madhavan

rmadhavan@rim.com | @rmadhavan7

July 31, 2012

Outline

Why use Cascades and OpenGL?

Cascades + OpenGL App Structure

OpenGL Integration Best Practices

Cascades UI Best Practices

Sample App #1: GoodCitizenCascades

Sample App #2: OpenGLDragNDrop

Q & A

2

Why use Cascades and OpenGL?

3

Compelling Content

Compelling Content

Visually Engaging Overlays

Personalized Avatars

Dynamic, Multi-Dimensional Graphs and Charts

Custom 3D Controls

Animated Menus ,Panels and Spinners

Progress Bars and Sand clocks

Trackballs and Knobs

Cool shader effects

Point lights and Spot lights effects

Objects morphing

Falling Pictures 5

Full-Featured UI / Interactivity

Cascades UI is a high-level API built on OpenGL ES

Wide selection of UI controls to choose from

Support for complex custom controls such as WebView, ContactsPicker and ForeignWindow

Built-in support for multiple panels and animations

Touch / event handing

Includes classes for processing sensor input

6

Cascades + OpenGL ES App Structure

7

OpenGLThread

OpenGLView n

Cascades + OpenGL ES App Structure

App Initialization

OpenGL Hooks

Cascades Event Handling

QML / UI

ForeignWindow

Actions / Event Handling

OpenGL Thread / Views

OpenGL Thread

OpenGL Views

OpenGL Event Handling

8

QML / UI

Initialization

OpenGLView 2

OpenGLView 1

App Initialization OpenGL Hooks

Initialize Cascades UI / load main QML file

QmlDocument *qml = QmlDocument::create("main.qml");

if (!qml->hasErrors()) {

…

}

9

App Initialization OpenGL Hooks

Traverse the control hierarchy and obtain ForeignWindow control

// The application NavigationPane is created from QML.

NavigationPane *navPane = qml->createRootNode<NavigationPane>();

if (navPane) {

Page* page = (Page *)(navPane->top());

Container* container = (Container *)(page->content());

m_pForeignWindow = (ForeignWindow *)(container->at(0));

}

Tip: Define all ForeignWindows in main.qml for easy access

10

App Initialization OpenGL ES Hooks

Connect ForeignWindow instances to OpenGL Objects // Create an instance of the GoodCitizen OpenGL ES Object

m_pGoodCitizen = new GoodCitizen(

m_pForeignWindow->mainWindowGroupId(),

m_pForeignWindow->mainWindowGroupId(),

m_pForeignWindow->preferredWidth(),

m_pForeignWindow->preferredHeight());

Register OpenGL Objects with the global context for access from within QML

qml->setContextProperty("_goodCitizen", m_pGoodCitizen);

11

App Initialization Cascades Event Handling

Connect ForeignWindow signals to OpenGL Object slots to process user input

// Connect ForeignWindow signals to slots

QObject::connect(m_pForeignWindow,

SIGNAL(touch(bb::cascades::TouchEvent *)),

m_pGoodCitizen,

SLOT(onTouch(bb::cascades::TouchEvent *)));

• Connect Navigator signals to OpenGL Object event handlers to handle App Exit and Minimized

12

QML / UI Foreign Window

Example ForeignWindow declaration in QML

ForeignWindow {

id: viewWindow

preferredWidth: 768

preferredHeight: 1140

visible: attached

layoutProperties: DockLayoutProperties {

horizontalAlignment: HorizontalAlignment.Center

verticalAlignment: VerticalAlignment.Top

}

}

13

QML / UI Actions / Event Handling

Follow QT_OBJECT design pattern for OpenGL Object’s Event Handling

Action or event handlers can now easily call slots defined in the OpenGL Objects!

onValueChanging: {

…

_goodCitizen.setObjectColor( newObjectColor);

}

14

QML / UI Actions / Event Handling

Actions, event handlers or QML functions can easily call property methods defined in the same OpenGL ES objects to query data needed for the UI

onCreationCompleted : {

var loadColor = _goodCitizen.objectColor;

…

}

15

OpenGL Thread

16

Initialize

Handle Events

Update

Render

// Initialize screen, windows, EGL etc.

initialize();

while(animating) {

// Handle event touch, keyboard etc.

handleEvents();

// Update state of component to show

// interaction.

update();

// Draw content to the screen.

render();

}

OpenGL Object Interface

Abstracts platform related implementation

Blackberry Platform Service

Screen

Window

EGL

Implementation focus on OpenGL component logic

Facilitates multiple 3D component support

Allows channeling of input events to appropriate 3D components

17

OpenGL Integration Best Practices

18

Thread Efficiency

Minimize the number of Threads

Run all OpenGL ES Components in one Run-Loop

Minimize EGL Context switches

Try to use one OpenGL Context for all the components / OpenGL ES objects

Sleep the OpenGL Thread whenever possible

Throttle Run-Loop at a maximum of 30 FPS

Avoid animating obscure components

19

Overlays vs Underlays

Overlays – Window lies above Cascades

Suitable for Transparent 3D controls and content

Note: Transparency add Blending cost!

Input events directly tapped from OpenGL Thread for in OpenGL Object interaction

Underlays - Window lies beneath Cascades

Suitable for opaque 3D controls and content

Input events from cascades can be tapped for OpenGL Object interaction

20

Cascades UI Best Practices

21

Cascades UI Best Practices

Properties, Signals and Slots

Foreign Window Alignment

UI Structure

UI Navigation, Tabs and Actions

Fixed vs. Sliding Panels

Custom Controls leveraging OpenGL

Recommended Configurations

22

Properties , Signals and Slots

Ensure OpenGL Object is a Q_OBJECT

Define properties for manipulating OpenGL state and data

Pass Objects and Arrays using Javascript in the QML

Properties can receive data using QVariantList datatype

Use qmlRegisterType() to register a custom C++ class as a QML type for use in QML

23

Foreign Window Alignment

ForeignWindow punches a hole through Cascades UI.

ForeignWindow and OpenGL

ForeignWindow can be connected to the OpenGL Window but you must explicitly attach the two

If you chose to not attach the two, always track ForeignWindow position and align OpenGL Window to avoid artifacts.

Two approaches:

Poll ForeignWindow dimensions in update() method.

Connect to a signal reflecting ForeignWindow property change.

24

UI Structure Navigation / Tabs and Actions

Use tabs for apps with multiple views or tasks

For apps with a single view, use a Navigation pane with an embedded Page or just a Page if you plan on using Sheets and / or custom panels for all other panels

Pages should have one or more actions to act on the 3D content

25

UI Structure Fixed vs. Sliding Panels

Fixed panels

Beneficial if the user needs to access certain controls to change content quickly

Ideal for Landscape layouts but works for select portrait layouts too!

Usually implemented as container of cascades and/or custom 3D controls.

Tip: for both fixed and sliding panels, location of the controls near the bottom of the screen (above the action bar) facilitates one-handed operation

26

UI Structure Fixed vs. Sliding Panels

Sliding panels

Suitable for 3D controls that are not on screen at all times

Implemented as:

Pages via NavigationPane

Sheets

Animated Custom pop-up panels

Tip: Embeded another ForeignWindow in a Page or Sheet to let the content show through

27

Custom OpenGL ES Controls Recommended Configurations

Transient / Floating Objects

Implement as overlays – no ForeignWindow required!

Objects are triggered upon events such as Timers and user input events

Intercept OpenGL Thread events directly for input events such as Touch, Move and Gestures

Object is disabled upon Touch up, Time outs and task completion.

Object event processing / rendering is paused when obscured

28

Custom OpenGL ES Controls Recommended Configurations

Custom Interactive controls

Controls with a refined 3D / realistic appearance such as real control panel, instruments and dash boards

Implement as underlays – a ForeignWindow is required

Object is triggered upon loading

Intercept Cascades input events such as Touch, Move and Gestures

Object terminates on component exit

Object is paused when obscured

29

Sample App: Good Citizen Cascades

30

GoodCitizenCascades

Cascades version of GoodCitizen native sample app

Demonstrate best practices for combining Cascades and OpenGL

App Structure

UI Elements / QML

Classes

31

App Structure UI Elements / QML

Main Window

main.qml

Navigation Pane with embedded Page with ForeignWindow and several actions

32

App Structure UI Elements / QML

Main Window Actions

Translate, rotation and scale on the action bar

Action overflow menu contains action bar actions plus color, objects and reset (these actions are here because they should be accessed less frequently)

33

App Structure UI Elements / QML

Color panel

Color.qml

Page with Sliders, Labels and a ForeignWindow to allow user to change object colour

34

App Structure UI Elements / QML

ToolAxis panel

ToolAxis.qml – custom pop-up panel with ImageToggleButtons used by the user to select current axis for touch control for the current tool associated with the panel

35

App Structure UI Elements / QML

Objects panel

Objects.qml

Custom pop-up panel with ImageToggleButtons used by the user to select current model that can be manipulated onscreen

36

App Structure Classes

App

Cascades App class where all of the initialization code resides

GoodCitizen

OpenGL ES object contains code derived from the GoodCitizen NDK example app

Q_OBJECT containing several properties and slots for interaction with the app’s UI QML layer

Contains event handlers for processing input events

37

Sample App: OpenGLDragNDrop

38

OpenGLDragNDrop

Based on GoodCitizenCascades

Demonstrates how to implement OpenGL drag and drop

Example of how to implement multiple views with the framework discussed today

Work-in-Progress

39

App Structure Concepts

Drag and Drop to change the model

The Object panel will be re-implemented as a custom pop-up panel that appears above the action bar (for easy reach)

When the user clicks on the model, a new OpenGLView is created of the new model that the user can drag over to the main view

Upon release, the new view disappears and the main view will change models to match

40

App Structure New Classes

ObjectDrag

Contains code derived from the GoodCitizen class except that the view is smaller, the background is transparent and the z index is specified as above the Cascades window

Contains event handlers for processing input events from the OpenGL event loop to implement the dragging operation

41

Questions and Answers

42

For More Information…

Download our sample apps, for review and inspiration, at:

https://github.com/blackberry/Cascades-Community-Samples

43

THANK YOU

Roberto Speranza

rsperanza@rim.com | @RSSessantotto

Ramprasad Madhavan

rmadhavan@rim.com | @rmadhavan7

July 31, 2012