ios opengl

116
Elastic Image Software LLC iOS OpenGL Douglass Turner Elastic Image Software email: [email protected] mobile: 781 775 3708

Upload: douglass-turner

Post on 11-May-2015

14.116 views

Category:

Technology


6 download

DESCRIPTION

Original Presentation: http://bit.ly/Lm8DyH Tweets: @dugla Email here: [email protected]

TRANSCRIPT

Page 1: iOS OpenGL

Elastic Image Software LLC

iOS OpenGL

Douglass TurnerElastic Image Software

email: [email protected]: 781 775 3708

Page 2: iOS OpenGL

Elastic Image Software LLC

2

A Talk in Three Acts

Page 3: iOS OpenGL

Elastic Image Software LLC

• Introduction to Computer Graphics• Computer Graphics in OpenGL• OpenGL and CocoaTouch Playing Nice Together

3

Page 4: iOS OpenGL

Elastic Image Software LLC

4

Introduction to Computer Graphics

A Trip Down the Graphics Pipeline

Page 5: iOS OpenGL

Elastic Image Software LLC

Copyright Steve Jubinville5

Build and Tesselate Model into Facets

Page 6: iOS OpenGL

Elastic Image Software LLC

Copyright Steve Jubinville6

Pose Model

Page 7: iOS OpenGL

Elastic Image Software LLC

Aim Camera

7

Page 8: iOS OpenGL

Elastic Image Software LLC

8

Design the Look

Page 9: iOS OpenGL

Elastic Image Software LLC

Design Appearance. Light Scene.

Copyright © 2006 Holger Schömann9

Page 10: iOS OpenGL

Elastic Image Software LLC

Copyright © 2006 Florian Wild

Design Appearance. Light Scene.

10

Page 11: iOS OpenGL

Elastic Image Software LLC

Copyright © 2006 Jack Qiao

Design Appearance. Light Scene.

11

Page 12: iOS OpenGL

Elastic Image Software LLC

12

Render Scene

Page 13: iOS OpenGL

Elastic Image Software LLC

13

Clip scene to the bounds of the viewing frustrum.

Page 14: iOS OpenGL

Elastic Image Software LLC

Rasterize Scene

14

Page 15: iOS OpenGL

Elastic Image Software LLC

15

for each object in the scene

! determine its corresponding pixels

! for each corresponding pixel

! ! calculate depth of object at pixel

! ! if depth is closer than any object yet seen

! ! ! put this object in the pixel

! ! endif

! end

end

The GPU implements a z-buffer algorithm in hardware

Page 16: iOS OpenGL

Elastic Image Software LLC

In OpenGL, the Look is evaluated at the same time that depth comparison is performed.

Together these are the core activities of computer graphics. They are the raison d'être of the GPU. Compute intensive, Highly optimized and parallelized.

16

Page 17: iOS OpenGL

Elastic Image Software LLC

17

Update Model State. This could entail geometric transformation or perhaps taking a time step in the physic engine.

Page 18: iOS OpenGL

Elastic Image Software LLC

18

Rinse, repeat.

Page 19: iOS OpenGL

Elastic Image Software LLC

19

Example.

Beautiful Panoramas iPad app. In the App Store: http://bit.ly/QVIDgg

Page 20: iOS OpenGL

Elastic Image Software LLC

20

Computer Graphics in OpenGL

Page 21: iOS OpenGL

Elastic Image Software LLC

OpenGL Cocoa

21

Page 22: iOS OpenGL

Elastic Image Software LLC

OpenGL is Old School

22

Page 23: iOS OpenGL

Elastic Image Software LLC

You are flipping levers on a large state machine.

23

Page 24: iOS OpenGL

Elastic Image Software LLC

OpenGL is brutally spare. Just the essentials.

float oldSchool[]

glPushMatrix()

glPopMatrix()!

24

Page 25: iOS OpenGL

Elastic Image Software LLCKeep in mind OpenGL is a dragster, not a Prius. So, just buckle up ...

25

Page 26: iOS OpenGL

Elastic Image Software LLC

... and enjoy the ride, baby!

26

Page 27: iOS OpenGL

Elastic Image Software LLC

27

Posing models. Positioning light sources. Aiming the camera. All these task require and understanding of transformation matrices.

Page 28: iOS OpenGL

Elastic Image Software LLC

28

Transformation matrix. Coordinate frameWorld space, Screen space, etc.

All are different ways of thinking about the same underlying concept.

Page 29: iOS OpenGL

Elastic Image Software LLC

29

When we move an object we apply a transformation via a transformation matrix. This transformation is defined in terms of a coordinate frame.

A specific location on the transformed object can be interpreted in terms of any coordinate frame we choose.

Page 30: iOS OpenGL

Elastic Image Software LLC

30

The motivation for a coordinate frame transformation is not only motivated by the need to pose a model or aim a camera as in the physical world.

In computer graphics land it often has to do with the convenience of performing a particular calculation in one space rather then another.

Page 31: iOS OpenGL

Elastic Image Software LLC

31

We refer to the transformation pipeline: the successive application of a series of transformations as a 3D model wends its way from application land to GPU land.

Page 32: iOS OpenGL

Elastic Image Software LLC

Hierarchical modeling is based on the hierarchical organization of transformations matrices. Many objects - trees, robots, cars - form natural hierarchies.

32

Page 33: iOS OpenGL

Elastic Image Software LLC

33

The UIView class supports all the key concepts of transformations and hierarchy. It is a nice, friendly way to gain intuition about facility with this powerful concept.

Page 34: iOS OpenGL

Elastic Image Software LLC

34

Hierarchical modeling in Cocoa Touch

Page 35: iOS OpenGL

Elastic Image Software LLC

35

struct CGAffineTransform { CGFloat a, b, c, d; CGFloat tx, ty;};

struct CATransform3D { CGFloat m11, m12, m13, m14; CGFloat m21, m22, m23, m24; CGFloat m31, m32, m33, m34; CGFloat m41, m42, m43, m44;};

Transforms

Page 36: iOS OpenGL

Elastic Image Software LLC

36

UIView properties

// Coordinate frame in parent space@property(nonatomic) CGRect frame;

// Dimensions of view in object space@property(nonatomic) CGRect bounds;

// View center in parent space @property(nonatomic) CGPoint center;

// View transformation matrix@property(nonatomic) CGAffineTransform transform;

// Parent view@property(nonatomic,readonly) UIView *superview;

// Child views@property(nonatomic,readonly,copy) NSArray *subviews;

Page 37: iOS OpenGL

Elastic Image Software LLC

37

UIView methods

// Transform 2D point from/to another view’s coordinate frame- (CGPoint)convertPoint:(CGPoint)point toView:(UIView *)view;- (CGPoint)convertPoint:(CGPoint)point fromView:(UIView*)view;

// Transform rectangle from/to another view’s coordinate frame- (CGRect)convertRect:(CGRect)rect toView:(UIView *)view;- (CGRect)convertRect:(CGRect)rect fromView:(UIView*)view;

// Manipulate view hierarchy- (void)removeFromSuperview;- (void)insertSubview:(UIView *)view atIndex:(NSInteger)index;- (void)addSubview:(UIView *)view;- (void)insertSubview:(UIView *)view belowSubview:(UIView *)siblingSubview;- (void)insertSubview:(UIView *)view aboveSubview:(UIView *)siblingSubview;

Page 38: iOS OpenGL

Elastic Image Software LLC

38

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { [window addSubview:viewController.view]; [window makeKeyAndVisible]; return YES;}

View hierarchy in iOS

Page 39: iOS OpenGL

Elastic Image Software LLC

39

Transformation matrices in iOS

- (void)setTransform:(CGAffineTransform)newValue {

// Scale along the y-axis only CGAffineTransform constrainedTransform = CGAffineTransformScale(CGAffineTransformIdentity, 1.0, newValue.a); newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d;

[super setTransform:newValue];}

Page 40: iOS OpenGL

Elastic Image Software LLC

HelloTeapot Demo and Code Walkthrough

http://github.com/turner/HelloTeapot

40

Page 41: iOS OpenGL

Elastic Image Software LLC

In what follows, we will be referring to the ES1 “fixed-function” transformation pipeline provided by the GPU.

In ES2 (GLSL) the transformation pipeline must be handled entirely in application space.

41

Page 42: iOS OpenGL

Elastic Image Software LLC

42

Page 43: iOS OpenGL

Elastic Image Software LLC

43

A camera with a spotlight attached to it orbits around a stationary teapot model lit by three stationary colored point light sources.

Page 44: iOS OpenGL

Elastic Image Software LLC

In iOS we never explicitly say “render scene”. Rather we inform the system that now would be a good time to redisplay the scene:

[myView setNeedsDisplay] and [myView drawRect]

44

Page 45: iOS OpenGL

Elastic Image Software LLC

- (void)startAnimation {

animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES];}

45

In OpenGL we manage rendering ourselves

Page 46: iOS OpenGL

Elastic Image Software LLC

46

In OpenGL we manage rendering ourselves

- (void)startAnimation {! if (!self.isAnimating) {! ! self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)];

[self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];

! ! ! self.animating = YES;! ! } // if (!self.isAnimating)!}

Page 47: iOS OpenGL

Elastic Image Software LLC

! Position the camera

! glMatrixMode(GL_MODELVIEW);!! glLoadIdentity(); ! glLoadMatrixf(_openGLCameraInverseTransform);

47

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Page 48: iOS OpenGL

Elastic Image Software LLC

Position spotlight at camera location glPushMatrix();! glMultMatrixf(_cameraTransform);!!! // A white spotlight for a camera headlight! GLfloat spotLightColor[] = { 1.0, 1.0, 1.0, 1.0 }; GLfloat spotLightPosition[] = { 0.0, 0.0, 2.0, 1.0 }; GLfloat spotLightDirection[] = { 0.0, 0.0, -1.0, 1.0 }; GLfloat spotCutOff!! ! = 3.5;!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLightColor);!! glLightfv(GL_LIGHT3, GL_POSITION, spotLightPosition);! glLightfv(GL_LIGHT3, GL_SPOT_DIRECTION, spotLightDirection);! glLightf(GL_LIGHT3, GL_SPOT_CUTOFF, spotCutOff);!! glPopMatrix();!

48

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Page 49: iOS OpenGL

Elastic Image Software LLC

49

glPushMatrix & glPopMatrix

These functions push/pop a transformation matrix on/off the transformation stack.

This stack is the key data structure for presenting a hierarchical model to the GPU in fixed-function OpenGL.

Page 50: iOS OpenGL

Elastic Image Software LLC

50

glPushMatrix - Create a new matrix and push it on the stack. Subsequent matrix concatenation is done with stack.top.

stack.top is the current coordinate frame.

glPopMatrix - Pop the stack. The pre-existing matrix is now stack.top.

We revert to the coordinate frame the existed prior to the push.

! glPushMatrix();! glMultMatrixf(_cameraTransform);!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);! glPopMatrix();!

Page 51: iOS OpenGL

Elastic Image Software LLC

51

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position the red stationary light source

! const GLfloat redLightColor[]! ! = { 1.0f, 0.0f, 0.0f, 1.0f };! const GLfloat redLightPosition[]! = { 10.0f, 0.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, redLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, redLightPosition);

Page 52: iOS OpenGL

Elastic Image Software LLC

52

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position the green stationary light source

! const GLfloat greenLightColor[]! = { 0.0f, 1.0f, 0.0f, 1.0f };! const GLfloat greenLightPosition[]!= { 0.0f, 10.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, greenLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, greenLightPosition);

Page 53: iOS OpenGL

Elastic Image Software LLC

53

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position the blue stationary light source

! const GLfloat blueLightColor[]!! = { 0.0f, 0.0f, 1.0f, 1.0f };! const GLfloat blueLightPosition[]! = { 0.0f, 0.0f, 10.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, blueLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, blueLightPosition);

Page 54: iOS OpenGL

Elastic Image Software LLC

54

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position and draw the teapot

! glPushMatrix();

! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);

! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {

! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );

! } // for (num_teapot_indices)

! glPopMatrix();

Page 55: iOS OpenGL

Elastic Image Software LLC

55

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position and draw the teapot

! glPushMatrix();

! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);

! glMultMatrixf(concatenation);

! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {

! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );

! } // for (num_teapot_indices)

! glPopMatrix();

Page 56: iOS OpenGL

Elastic Image Software LLC

56

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Position and draw the teapot

! glPushMatrix();

! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);

! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {

! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );

! } // for (num_teapot_indices)

! glPopMatrix();

Page 57: iOS OpenGL

Elastic Image Software LLC

57

Notice the absence of GL_CAMERA?

You may be surprised to learn there is no concept of camera in OpenGL. Camera posing is equivalent to inverting the camera transform and applying it to the scene observed by the camera.

Page 58: iOS OpenGL

Elastic Image Software LLC

camera teapot

world

wTtwTc

58

Page 59: iOS OpenGL

Elastic Image Software LLC

camera teapot

glMatrixMode(GL_MODELVIEW);glLoadMatrixf(_openGLCameraInverseTransform);

?

cTt

59

Page 60: iOS OpenGL

Elastic Image Software LLC

camera teapot

world

wTt(wTc)

cTt

cTt wTtcTw *=

-1

60

Page 61: iOS OpenGL

Elastic Image Software LLC

cTt wTtcTw *=

61

Page 62: iOS OpenGL

Elastic Image Software LLC

- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up;

62

http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Page 63: iOS OpenGL

Elastic Image Software LLC

The vertices comprising the 3D geometry of the teapot

float teapot_vertices [] = { 0.0663, 0.1178, 0.0, 0.0672, 0.1152, 0.0, 0.0639, 0.1178, 0.0178043,

... };

The surface normals associated with each vertex

// The list of surface normals corresponding to the verticesfloat teapot_normals[] = { -0.987635, -0.156768, 0, -0.902861, -0.429933, 0, -0.953562, -0.156989, -0.257047,

... };

63

Page 64: iOS OpenGL

Elastic Image Software LLC

The teapot model is defined as a collection of triangle stripes that combine to form the teapot surface.

short indicies[] = {

// how many vertices in vertex strip26,

// vertex strip indices1122, 1243, 1272, 1242, ... ,1283, 1199, ... };

64

Page 65: iOS OpenGL

Elastic Image Software LLC

glEnableClientState(GL_VERTEX_ARRAY);glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices);

glEnableClientState(GL_NORMAL_ARRAY);glNormalPointer(GL_FLOAT, 0, teapot_normals);glEnable(GL_NORMALIZE);!

65

! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {

! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );

! }

Page 66: iOS OpenGL

Elastic Image Software LLC

Design the Look

66

Page 67: iOS OpenGL

Elastic Image Software LLC

67

We now begin leaving behind the fixed-function pipeline of ES1 and focus on ES2, GLSL, and visual coolness in general.

Page 68: iOS OpenGL

Elastic Image Software LLC

The Look: the lighting, illumination model, and surface shaders that in combination achieve the desired look of a model in a scene.

68

Page 69: iOS OpenGL

Elastic Image Software LLC

Lights

69

Page 70: iOS OpenGL

Elastic Image Software LLC

70

Point Light Directional Light

Spot Light Area Light

Page 71: iOS OpenGL

Elastic Image Software LLC

71

Page 72: iOS OpenGL

Elastic Image Software LLC

Illumination Model

72

Page 73: iOS OpenGL

Elastic Image Software LLC

Ambient Diffuse

SpecularDiffuse

+Specular

73

Page 74: iOS OpenGL

Elastic Image Software LLC

Diffuse(NormalVector, LightVector)Ambient(ConstantReflectance)

Specular(LightReflectionVector, EyeVector) Shiny(Specular, Diffuse)74

Page 75: iOS OpenGL

Elastic Image Software LLC

Surface Shaders

75

Page 76: iOS OpenGL

Elastic Image Software LLC

Surface shaders were introduced to the field of computer graphics by Pixar with its rendering API and shading language called RenderMan.

A shader is a small functions evaluated at every location on a being rendered.

GLSL borrows heavily from the RenderMan model.

76

Page 77: iOS OpenGL

Elastic Image Software LLC

Copyright © Douglass Turner77

I created this gallery of shaders using a rendering/shading system I wrote that was inspired by RenderMan.

Page 78: iOS OpenGL

Elastic Image Software LLC

78

Copyright © Douglass Turner

Page 79: iOS OpenGL

Elastic Image Software LLC ZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf

79

Page 80: iOS OpenGL

Elastic Image Software LLCCopyright: Stephen Molyneaux

80

Page 81: iOS OpenGL

Elastic Image Software LLC

Hello Shader Demo and Code Walkthrough

https://github.com/turner/HelloShader

81

Page 82: iOS OpenGL

Elastic Image Software LLC

82

Before we dive into shaders lets get a lay of the land. There are two flavors of shaders in GLSL.

A vertex shader is evaluated at each vertex.

A fragment shader is evaluated at each screen space pixel corresponding to a sampled on the facet being rasterized.

Page 83: iOS OpenGL

Elastic Image Software LLC

83

Vertex and fragment shaders work together in a pipeline fashion. Vertex attributes - color, surface normal, texture coordinate - are evaluated in the vertex shader then passed on to the fragment shader where those values are interpolated across the surface.

Page 84: iOS OpenGL

Elastic Image Software LLC

84

ShowST. A shader that visualizes the texture coordinate parameterization of a surface.

Page 85: iOS OpenGL

Elastic Image Software LLC

85

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh

// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;

// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;

varying vec2 v_st;void main() {

gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;

}

ShowST - vertex shader

Page 86: iOS OpenGL

Elastic Image Software LLC

86

attribute vec2 myVertexST - Vertex attribute

varying vec2 v_st - Varies across the surface

uniform sampler2D myTexture_0 - Invariant throughout rendering cycle

Shader variables come in different flavors

Page 87: iOS OpenGL

Elastic Image Software LLC

87

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh

Vertex attribute myVertexST is a texture coordinate passed to the fragment shader via varying variable v_st.

// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;

// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;

varying vec2 v_st;void main() {

gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;

}

Page 88: iOS OpenGL

Elastic Image Software LLC

88

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh

// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0;}

ShowST - fragment shader

Page 89: iOS OpenGL

Elastic Image Software LLC

89

http://github.com/turner/HelloiPadGLSL/blob/master/Shaders/ShowST.fsh

// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0;}

The texture coordinate values v_st.s and v_st.t are for the red and green channels of the shader color.

Page 90: iOS OpenGL

Elastic Image Software LLC

90

TexturePairShader. A shader that mixes the colors of a pair of textures using the tweaked alpha channel of one of the pair.

Fail Whale over Mandril

Page 91: iOS OpenGL

Elastic Image Software LLC

91

TexturePairShader - vertex shader

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh

// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;

// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;

varying mediump vec2 v_st;

void main() {

gl_Position = myProjectionViewModelMatrix * myVertexXYZ;

v_st = myVertexST;}

Page 92: iOS OpenGL

Elastic Image Software LLC

92

TexturePairShader - vertex shader

// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;

// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;

varying mediump vec2 v_st;

void main() {

gl_Position = myProjectionViewModelMatrix * myVertexXYZ;

v_st = myVertexST;}

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh

Page 93: iOS OpenGL

Elastic Image Software LLC

93

TexturePairShader - fragment shader

// TEITexturePairShader.fshvarying mediump vec2 v_st;uniform sampler2D myTexture_0;uniform sampler2D myTexture_1;void main() {

vec4 rgba_0 = texture2D(myTexture_0, v_st); vec4 rgba_1 = texture2D(myTexture_1, v_st);

// Interpolate between the two textures using the alpha// channel of texture 0 as the interpolant.

float interpolate = rgba_0.a; gl_FragColor.r = rgba_0.r + (1.0 - interpolate) * rgba_1.r; gl_FragColor.g = rgba_0.g + (1.0 - interpolate) * rgba_1.g; gl_FragColor.b = rgba_0.b + (1.0 - interpolate) * rgba_1.b; gl_FragColor.a = rgba_0.a + (1.0 - interpolate) * rgba_1.a;}

https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh

Page 94: iOS OpenGL

Elastic Image Software LLC

94

attribute vec2 myVertexST - Vertex attribute

varying vec2 v_st - Varies across the surface

uniform sampler2D myTexture_0 - Invariant throughout rendering cycle

Shader variables come in different flavors

Page 95: iOS OpenGL

Elastic Image Software LLC

95

An application communicates with its shaders in the follow ways:

• attributes - geometry, color, normal, texture coordinate• uniforms - texture sampler, matrix, eye vector, light vector• textures - texture channels - r, g, b, a - are not just for color

attribute vec2 myVertexST - Vertex attribute

varying vec2 v_st - Varies across the surface

uniform sampler2D myTexture_0 - Invariant throughout rendering cycle

Page 96: iOS OpenGL

Elastic Image Software LLC

96

Lets look at how we wire our iOS app together with our shaders. We will use a texture shading example.

https://github.com/turner/HelloShader/blob/master/HelloShader/Classes/Renderer/GLRenderer.m

Page 97: iOS OpenGL

Elastic Image Software LLC

97

First make a texture object ...

TEITexture *t = [[ [TEITexture alloc] initWithImageFile:@"twitter_fail_whale_red_channnel_knockout" extension:@"png" mipmap:YES ] autorelease];[glView.renderer.rendererHelper.renderables setObject:t forKey:@"texture_0"];

Page 98: iOS OpenGL

Elastic Image Software LLC

98

Load shaders, Bind shader attribute to application handle.

// Vertex shader NSString *vertShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"vsh"];

[self compileShader:&vertShader type:GL_VERTEX_SHADER file:vertShaderPathname];!

// Fragment shader NSString *fragShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"fsh"];

[self compileShader:&fragShader type:GL_FRAGMENT_SHADER file:fragShaderPathname];!

// Bind attribute glBindAttribLocation(m_program, VertexSTAttributeHandle,! "myVertexST");

Page 99: iOS OpenGL

Elastic Image Software LLC

99

Associate shader uniform with application object instance ...

TEITexture *t = (TEITexture *)[self.rendererHelper.renderables objectForKey:@"texture_0"];

t.location = glGetUniformLocation(m_program, "myTexture_0");

glActiveTexture( GL_TEXTURE0 );

glBindTexture(GL_TEXTURE_2D, t.name);

glUniform1i(t.location, 0);

Activate a texture unit, bind it to a texture object, and assign a number to the corresponding texture sampler used in the fragment shader

Page 100: iOS OpenGL

Elastic Image Software LLC

100

We now have a linkage between our application texture object and a texture sampler in our fragment shader. Rock.

Page 101: iOS OpenGL

Elastic Image Software LLC

101

Page 102: iOS OpenGL

Elastic Image Software LLC

102

Page 103: iOS OpenGL

Elastic Image Software LLC

103

Lets Talk Textures

Page 104: iOS OpenGL

Elastic Image Software LLC

104

Texture mapping is the fundamental tool for creating visual complexity

Page 105: iOS OpenGL

Elastic Image Software LLC

105

By establishing a mapping between a surface and a texture we can “attach” the texture to the surface.

The interpretation of the a can go far beyond that of a decal to be applied to a surface. Bumps, opacity, displacement, and much more can be designed with a texture.

Page 106: iOS OpenGL

Elastic Image Software LLC

106

Page 107: iOS OpenGL

Elastic Image Software LLC

107

Shaders can have it both ways: Compute surface color (or surface roughness, or opacity) algorithmically or look it up in a texture. Or a bit of both.

Page 108: iOS OpenGL

Elastic Image Software LLC

OpenGL and iOS

Playing Nice Together

108

Page 109: iOS OpenGL

Elastic Image Software LLC

• OpenGL is mostly M• Keep V dumb

• Lots of chatter between M and C

MVC

109

Page 110: iOS OpenGL

Elastic Image Software LLC

110

Lets take a look at how iOS and OpenGL interact. We will us Hello iPad OpenGL as our guide.

https://github.com/turner/HelloShader

Page 111: iOS OpenGL

Elastic Image Software LLC

111

-(id)initializeEAGL {!! CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer;!! eaglLayer.opaque = TRUE;! eaglLayer.drawableProperties = ! [NSDictionary dictionaryWithObjectsAndKeys:! [NSNumber numberWithBool:FALSE], kEAGLDrawablePropertyRetainedBacking, ! kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat, ! nil];!! self.renderer = [[[ES2Renderer alloc] init] autorelease];!! m_animating = FALSE;! animationFrameInterval = 1;! m_displayLink = nil;!! return self;!}

The view’s layer is used as the rendering surface

Page 112: iOS OpenGL

Elastic Image Software LLC

112

Rendering frame rate is sync-ed to the screen refresh rate. The actual rendering is handled via a selector.

- (void)startAnimation { if (!self.isAnimating) { self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)]; [self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; self.animating = YES; } // if (!self.isAnimating)}

Page 113: iOS OpenGL

Elastic Image Software LLC

113

// framebuffer glGenFramebuffers(1, &m_framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer); // rgb buffer glGenRenderbuffers(1, &m_colorbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, m_colorbuffer); [m_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:layer]; glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &m_backingWidth); glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &m_backingHeight); // z-buffer glGenRenderbuffers(1, &m_depthbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_depthbuffer); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, m_backingWidth, m_backingHeight); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, m_depthbuffer);

A framebuffer object is created. A color and depth buffer are attached.

Page 114: iOS OpenGL

Elastic Image Software LLC

114

- (void) render {! ! [EAGLContext setCurrentContext:m_context]; glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer);

// transform, light, shade, etc.

glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); [m_context presentRenderbuffer:GL_RENDERBUFFER];}

The render loop. Draw to the colorbuffer then present to the display. This is the classic “ping pong” between back buffer and front buffer.

Page 115: iOS OpenGL

Elastic Image Software LLC

Thank You

Douglass TurnerElastic Image Software

email: [email protected]: 781 775 3708

115

Page 116: iOS OpenGL

Elastic Image Software LLC

116

Copyright © Douglass Turner

Elastic Image Software LLC