Understanding Hardware Acceleration on Mobile

Browsers

Ariya Hidayat

Magical AdviceUse translate3d for hardware

acceleration

Challenges

Game vs Web

Game

Animation

Physics

Textured objects

Text

Transformation

Large but Still Bounded

Web Page

Images

Text

Challenges

Predictable contents (assets) ✔

Mostly text ✔

Mostly images ✔

immediateInitial response

Browser Rendering

From Wave to Pixels

Layout engine

Network stack

GraphicsJavaScript engine

User interface

WebKit Components

Render Engine

WebCore

JavaScript Engine (JSC/V8)Client Interface

HTML

DOMCSS

SVG

Canvas

Platform Abstraction

Network Unicode Clipboard

Graphics Theme Events

Thread Geolocation Timer

Graphics Abstraction

iOS Android

CoreGraphicsSkia

Graphics stack

GraphicsContext

SoC = System-on-a-Chip

CPU GPU

GPU Workflow

Vertices

Matrix

Rendered Textured

Optimized for Games

Transformation

Textured triangles

“Fixed” geometry

Parallelism

Powerful Capabilities

http://epicgames.com/technology/epic-citadel http://www.rage.com/

http://www.trollquotes.org/619-super-spider-bat-troll-quote/

Fundamental Physical Limitations

Memory- Can’t store too much stuff

Speed- Quad-core CPU can do certain operations better

Bandwidth- Bottleneck can be in the data transfer

Traffic Congestion

Primitive Drawing

Path is Everything

Path

Triangle Rectangle

PolygonEllipse

Stroke = Outline

Solid Dashed Dotted Textured

Brush = Fill

SolidNone Gradient Textured

Gradient = Non-uniform Pixel Values

Drawing Time: Solid vs Gradient

Only border

Solid color fill

Linear gradient fill

Radial gradient fill

14x slower(depending on the dimension)

Path Approximation

Curves

Polygon

Smooth via Antialiasing

Multiple levels of transparency

Perfect for parallelism

Shadow

Involved pixel “blending” Often obstructed

Drawing Time: Solid vs Blur Shadow

Solid shadow

Blur shadow

20x slower(depending on the blur radius)

CSS = Box Model

http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of Stacking Contexts

Shadow Abuse

12,000 pixels

Blur shadow

Transformation

Scaling

RotationPerspective

Text Rasterization

Font Atlas

Bye

Buffer

From Simple to Complex

Hello!

Simple shapeSolid color

CurvesGradient brushTextured stroke

Blur shadowSerif textRotated

Make it as an image

Library Instrumentation

platformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff fffillRect 0,0 800x556 color ff ff ff ff

Painting Performance

16:24:04.070 D/webcoreglue(  273): drawContent 11 ms16:24:04.110 D/webcoreglue(  273): drawContent 13 ms16:24:04.150 D/webcoreglue(  273): drawContent 13 ms16:24:04.190 D/webcoreglue(  273): drawContent 10 ms16:24:04.240 D/webcoreglue(  273): drawContent 10 ms16:24:04.280 D/webcoreglue(  273): drawContent 13 ms16:24:04.320 D/webcoreglue(  273): drawContent 13 ms16:24:04.360 D/webcoreglue(  273): drawContent 13 ms16:24:06.080 D/webcoreglue(  273): drawContent 12 ms16:24:06.140 D/webcoreglue(  273): drawContent 10 ms16:24:06.180 D/webcoreglue(  273): drawContent 13 ms16:24:06.230 D/webcoreglue(  273): drawContent 14 ms16:24:06.600 D/webcoreglue(  273): drawContent 26 ms16:24:06.640 D/webcoreglue(  273): drawContent 13 ms16:24:06.860 D/webcoreglue(  273): drawContent 33 ms16:24:06.890 D/webcoreglue(  273): drawContent 12 ms16:24:06.930 D/webcoreglue(  273): drawContent 13 ms16:24:06.960 D/webcoreglue(  273): drawContent 13 ms16:24:07.000 D/webcoreglue(  273): drawContent 13 ms

Backing Store

Maps

Tiles

Responsive Interface

Rendering

User interactionProcessor

Pinch to Zoom

when you pinch...

Rendering vs Interaction

Web Page

Rendering

User interaction

Checkerboard Pattern

Poor man’s indicator of performance

Progressive Rendering

Crisp but slow

Fast but blurry

Perceived Responsiveness

User pinches

Smooth scaled

Blocky (but fast!)

Tiling System

CPU GPU

........

Streamed

What happens if the page background changes color?

Tile Transformation

Panning = Translation Zooming = Scaling

Y U NOposition:fixed

Layer & Compositing

Typical Animation

opacity, movement, scaling, rotation, ...

Principles of Fluid Animation

1 At the beginning, push as many resources as possible to the GPU

2 During the animation, minimize CPU-GPU interaction

Immediate vs Retained

draw the shape at (x, y)x = x + 10

blit the buffer at (x, y)x = x + 10

For every animation tick...

At the beginning...

draw the shape onto a buffer

Off main thread

Mechanics of Animation

Initialization

Animation step

“Hey, this is good stuff. Cache it as texture #42.”

“Apply [operation] to texture #42.”

Elements = Layer

Logical 3-D

Compositing By Force

-webkit-transform: translateZ(0)

Not needed for CSS animation!

.someid { -webkit-animation-name: somekeyframeanimation; -webkit-animation-duration: 7s; -webkit-transform: translateZ(0);}

Don’t do that

Debugging in Safari

defaults write com.apple.Safari IncludeInternalDebugMenu 1

Compositing Indicators

Composited layer

Container layer

No associated texture

Overflow

Texture (number = upload)

Avoid Texture Reupload

No reupload Upload

OpacityPosition

SizeAnimation

Everything else!

“Hardware accelerated CSS”

Examples

Color Transition@-webkit-keyframes box { 0% { -webkit-transform: background-color: blue; } 100% { -webkit-transform: background-color: green; }}

Need a new texture uploaded to the GPU for every in-between color

Color Transition: The Trick

Blended with

Prepare and Reuse

Viewport

Hide the layer offscreen

Timer Latency

Depending on user reaction

•Animation end triggers the JavaScript callback•JavaScript code kicks the next animation

Prepare both animation and hide the “wrong” one

Avoid Overcapacity

........

Texture upload

Think of the GPU memory like a cache.

Wrap-up

Hardware Acceleration

Drawing primitives Backing stores Layer & compositing

Traditional graphics programming has been

always full of tricks.

It will always be.

There is no such

Thank You

ariya.hidayat@gmail.com

@AriyaHidayat

ariya.ofilabs.com

Wed 1:45 PM

Office Hour