minko stage3d workshop_20130525
TRANSCRIPT
![Page 1: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/1.jpg)
Jean-Marc Le Roux @promethe42 http://blogs.aerys.in/jeanmarc-leroux
![Page 2: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/2.jpg)
Resources
Sources https://github.com/aerys/minko
Forum http://forum.minko.io
Samples
http://github.com/aerys/minko-examples
Documentation http://doc.minko.io
![Page 3: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/3.jpg)
@MINKO3D Follow us on Twitter!
![Page 4: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/4.jpg)
http://soccerpun.ch
Full 3D game built with Minko in just 48h! http://www.youtube.com/watch?v=Hv0cp4NwSoY
Your smartphone is the gamepad: up to 8 players!
All assets hanlded through the editor Imported from 3DS Max Editor available for free on http://minko.io Physics rigging in a few clicks
Extensive use of particles
Designed with the new particles editor Running with a new C++ particles engine
![Page 6: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/6.jpg)
Last time…
![Page 7: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/7.jpg)
… and now!
What are shaders?
Why is AGAL hard to work with and what solutions does Minko provide?
A few simple shader examples
Hardware accelerated particles with shaders
Particles engine and editor!
![Page 8: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/8.jpg)
HDR Rendering Dynamic lights
Static lights Dynamic shadows
Static shadows Diffuse texture
Noise
Diffuse texture
![Page 9: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/9.jpg)
Hardware specific shader bytecode
Program3D
AGAL bytecode
Vertex Shader Fragment Shader
GLSL/HLSL
Pro
gram3
D
PixelBender3D AGAL assembly
Act
ion
Scri
pt
3
Flas
h
Har
dw
are
![Page 10: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/10.jpg)
![Page 11: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/11.jpg)
![Page 12: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/12.jpg)
Program3D – Vertex Shader
va1.x va1.y 0 0 …
x y z w …
v0 …
Varying Registers
IndexBuffer3D
-5 -5 0 0 1 0 5 0 0 0.5 5 -5 0 1 1 …
x y z u v x y z u v x y z u v x
VertexBuffer3D
0 1 2
0 2 1 …
Inp
ut
Pro
gram
O
utp
ut
0 (-5, -5, 0)
1 (0, 5, 0)
2 (5, -5, 0)
![Page 13: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/13.jpg)
Inp
ut
Pro
gram
O
utp
ut
Program3D – Fragment Shader
va1.x va1.y 0 0 …
x y z w …
v0 …
Varying Registers
Texture3D
0 (0, 1) 2 (1, 1)
1 (0.5, 0)
![Page 14: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/14.jpg)
Kids, this is the story of how I met your shader…
![Page 15: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/15.jpg)
1992 2001 2004 2011
Mortal Kombat GTA III Doom 3 Crysis 2
1996
Evolution of shader languages
![Page 16: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/16.jpg)
What about AGAL?
1992 2001 2004 2011
Mortal Kombat GTA III Doom 3 Crysis 2
1996
Language Features
![Page 17: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/17.jpg)
In 1992, Flash was called FutureSplash Animator and written by a single man: Jonathan Gay
![Page 18: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/18.jpg)
Simple scene use case
How many shaders?
Shadows
Lights
Animations
![Page 19: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/19.jpg)
How many « possible shaders » ?
Handle 0 to 8 joints
With normals
With tangents
Handle 0 to 5 lights
With bump-mapping or not
Handle shadows
(9 x 3 x 6 x 2 x 2)²
= 419904 different shaders!
• To write • To test • To debug • To maintain • To distribute • To adapt • …
![Page 20: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/20.jpg)
Shaders are « static »
They handle one « scenario »
A lot of shaders to
write…
![Page 21: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/21.jpg)
learning AGAL != learning GPU programming
AGAL is awesome for 3D engines developers
Low-level binary assembly code
Cross-platform
AGAL is a nightmare for 3D applications developers
![Page 22: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/22.jpg)
With Minko you can program the GPU using ActionScript 3.0 !
![Page 23: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/23.jpg)
![Page 24: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/24.jpg)
Minko embedded JIT Compiler
Shader ByteCode ActionScript Shader
Code
public class MyShader { public function getVertexPosition() : SFloat { return localToScreen(vertexXYZ); } public function getPixelColor() : SFloat { return sampleTexture( meshBindings.getTextureParameter(‘texture’), interpolate(vertexUV) ); } }
m44 vt0, va0, vc0 m44 vt0, vt0, vc5 mov oc, vt0 mov v0, va1 tex ft0, v0 <linear,2d> mov oc, ft0
at runtime compilation
![Page 25: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/25.jpg)
ActionScript Shader
•Use all ActionScript 3.0 features
•getOutputPosition => Vertex Shader
•getOutputColor => Fragment Shader
•OOP style
•CPU/GPU balancing
ASG
•Abstract Shader Graph
•Optimizations
•Constants and temporary registers allocation
•Operations re-ordering for faster execution and optimal use of temporary registers
AGAL
•Direct bytecode generation at runtime
•Custom compiler
•Optional debug data
•AGAL assembly output
•Shader graph output
•Memory allocation map
Exec
uti
on
Full AS3 workflow Conditionnals and loops Classes and methods Dynamic OOP coding style
Exhaustive AGAL implementation
Extra high-level operations set Re-usable « shader parts »
Shaders compiled at-runtime
Just like any other AS3 code
Dynamic according to The scene properties The mesh properties/material Any constant, variable, etc…
Redistributable as SWF/SWC files
![Page 26: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/26.jpg)
public class RedShader extends Shader
{
override protected function getVertexPosition() : Sfloat
{
return worldToScreen(localToWorld(vertexXYZ));
}
override protected function getPixelColor() : SFloat
{
return float4(1, 0, 0, 1);
}
}
RedShader – Step 1: AS3
![Page 27: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/27.jpg)
RedShader – Step 2 : ASG
![Page 28: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/28.jpg)
RedShader – Step 3 : AGAL
- vertex shader
m44 vt0.xyzw, va0.xyzw, vc0.xyzw
m44 vt0.xyzw, vt0.xyzw, vc4.xyzw
mov op.xyzw, vt0.xyzw
- fragment shader
mov oc.xyzw, fc0.xyzw
![Page 29: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/29.jpg)
AS Shaders Workflow
Yes
shaderSignature(scene, mesh)
Does the signature already exists ?
No
Set streams, textures and constants: • Vertex streams • Vertex constants • Fragment constants • Textures
vsGraph = shader.getVertexPosition(); fsGraph = shader.getPixelColor();
Optimize ASG
Memory allocation • Vertex attributes • Vertex constants • Fragment constants • Resolving varying
Draw triangles
Shader Compilation
save(signature, compiledBytecode)
![Page 30: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/30.jpg)
AC TIONSCRIPT IS NOW
THE MOST POWERFUL S H A D E R L A N G U A G E
![Page 31: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/31.jpg)
AC TIONSCRIPT IS NOW
THE MOST POWERFUL S H A D E R L A N G U A G E
(as a language, but not GPU feature wise because of Stage3D limitations )
![Page 32: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/32.jpg)
More about JIT shaders compilation…
JIT shaders for better performance article on my blog
![Page 33: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/33.jpg)
![Page 34: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/34.jpg)
EXAMPLES
![Page 35: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/35.jpg)
Solid Red
override protected function getPixelColor() : SFloat
{
return float4(1, 0, 0, 1);
}
![Page 36: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/36.jpg)
Solid Parametrized Color
override protected function getPixelColor() : SFloat
{
return meshBindings.getParameter(‘diffuseColor’, 4);
}
mesh.material.diffuseColor = Vector4(1., 0., 0., 1.);
![Page 37: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/37.jpg)
Texture Mapping
override protected function getPixelColor() : SFloat
{
var texture : Sfloat = meshBindings.getTextureParameter(
‘diffuseMap’
);
return sampleTexture(texture, interpolate(vertexUV));
}
mesh.material.diffuseMap = TextureLoader.load(
new URLRequest(‘texture.jpg’)
);
![Page 38: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/38.jpg)
interpolate()
Vertex Shader Value
Fragment Shader Value
interpolate()
![Page 39: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/39.jpg)
interpolate()
Fragment Shader
Varying Registers
Vertex Shader
vertex0
x y r g b
Vertex Shader Output
Varying Registers
Vertex Shader
vertex1
x y r g b
Vertex Shader Output
Varying Registers
Vertex Shader
vertex2
x y r g b
Vertex Shader Output
Pixel Shader Output
Linear Interpolation
color
x y z w
Linear Interpolation
vertex0
vertex1 vertex2
![Page 40: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/40.jpg)
Particles
1. Fill vertex buffer with batched « quads »
2. (optional) Edit quads position on the CPU State-related animations
3. Render all the quads in a single draw call Stateless position/size/rotation animation in the
vertex shader
Stateless color animation in the fragment shader
![Page 41: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/41.jpg)
Particles – Quads?
quadVertex = { particleId, x, y, z, offsetX, offsetY, offsetZ , rotation }
0 1
2 3
(x, y, z)
(offsetX, offsetY, offsetZ)
rotation
quad = { 0, 1, 2, 3}
particleId ϵ * 0 … numParticles ]
![Page 42: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/42.jpg)
Particles – Quads?
quadVertex = { particleId, x, y, z, offsetX, offsetY, offsetZ , rotation }
0 1
2 3
(x, y, z)
(offsetX, offsetY, offsetZ)
rotation
Fixed for the 4 vertex of a particule quad
![Page 43: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/43.jpg)
Particles – Quads?
0
1
2
3
0 = { 0, 1., 4., 0., -0.5, 0.5, 0., 1.4 } 1 = { 0, 1., 4., 0., 0.5, 0.5, 0., 1.4 }
2 = { 0, 1., 4., 0., 0.5, -0.5, 0., 1.4 } 3 = { 0, 1., 4., 0., -0.5, -0.5, 0., 1.4 }
y
x
(1., 4., 0.)
![Page 44: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/44.jpg)
Vertex shader logic
1. Transform quad center to view space
2. Add offset (~= corner position) to the view space quad center
3. Apply rotation
4. Project on the screen
![Page 45: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/45.jpg)
Particule Properties
Each particle can have many properties! Start/end color
Start/end velocity
Start/end force
…
Particles design can be done with code…
… but it is much more efficient with an editor!
![Page 46: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/46.jpg)
Particle Editor
Full WYSIWYG particles editor! http://www.youtube.com/watch?v=tyW2RUm2naI
Stay tuned for the release (June)
Follow @Minko3D
Read http://minko.io
![Page 48: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/48.jpg)
GLOBAL ILLUMINATION! Next time…
![Page 49: Minko stage3d workshop_20130525](https://reader033.vdocuments.us/reader033/viewer/2022060119/558e34931a28abe13f8b463a/html5/thumbnails/49.jpg)
MERCI !