Shading Languages

Yung-feng Chiu

2

Agenda

• Introduction

• Pixar’s RenderMan, MS’s HLSL, NV’s CgFX, ATI’s RenderMonkey

• Demos

• .fx file

• Comparsion

3

Key Idea of a Shading Language

• Image synthesis can be divided into two basic concerns– Shape: Geometric Objects, Coordinates,

Transformations, Hidden-Surface Methods…– Shading: Light, Surface, Material, Texture, …

• Control shading not only by adjusting parameters and options, but by telling the shader what you want it to do directly in the form of a procedure

4

Pixar’s RenderMan

• Separation of Modeling and Rendering– RenderMan serves as the interface.

• Scene = Shape + Shading

• The power of RenderMan is in the shading part.

5

Pixar’s RenderMan -2

RmanGeomCode

cc RmanProgram

RmanShader

.sl

Shader(slc)

Byte-codeShader

.slc

RIBFile.rib

renderProgram(rendrib)

TIFFimage

Rmantexture

ImageFile

txmake

6

RenderMan’s RIB File Structure

Options global to the entire animation

Frame Block

Image options

Camera options

World Block

Attributes, lights, primitives

Changed Options

Another world block

Next frame block

7

RenderMan’s Shader

• Phong shader

surface phong( float Ka = 1, Kd =1, Ks = 0.5;float roughness = 0.1;

color specularcolor = 1; ){ normal Nf = faceforward( normalize(N), I ); vector V = -normalize(I); color C = 0; illuminance( P ) { vector R = 2*normalize(N)* (normalize(N) . normalize( L )) - normalize( L ); C += Ka*Cs + Kd*Cs*( normalize(N) . normalize(L) ) + Ks*specularcolor* pow(( R . V ), 10); } Ci = C*Cs;}

8

RenderMan’s Shader -2

• Attaching to the RIB file***********AttributeBeginTranslate 0 -1.5 0Rotate 20 0 0 1Color [ 0.8 0.0 0.0 ]Surface "phong" "Ka" [.1] "Kd" [.8] "Ks" [1] "roughness" [0.1] "specularcolor" [1 1 1] Basis "bezier" 3 "bezier" 3PatchMesh "bicubic" 13 "nonperiodic" 10 "nonperiodic" "P" [1.5 0 0 1.5 0.828427 0 0.828427 1.5 0 0 1.5 0 -0.828427 1.5 0 -1.5 0.828427 0 -1.5 0 0 -1.5 -0.828427 0 -0.828427 -1.5 0 0 -1.5 0 0.828427 -1.5 0 1.5 -0.828427 0 1.5 0 0 1.5 0 0.075 1.5 0.828427 0.075 0.828427 1.5 0.075 0 1.5 0.075 -0.828427 1.5 0.075 -1.5 0.828427 0.075 -1.5 0 0.075 -1.5 -0.828427 0.075 -0.828427 -1.5 0.075 0 -1.5 0.075 0.828427 -1.5 0.075 1.5 -0.828427 0.075 1.5 0 0.075 2 0 0.3 2 1.10457 0.3 1.10457 2 0.3 0 2 0.3 -1.10457**************

9

Misrosoft’s HLSL (.fx)

Assembly …

dp3 r0, r0, r1

max r1.x, c5.x, r0.x

pow r0.x, r1.x, c4.x

mul r0, c3.x, r0.x

mov r1, c2

add r1, c1, r1

mad r0, c0.x, r1, r0

...

HLSL…float4 cSpec = pow(max(0, dot(Nf, H)),

phongExp).xxx;float4 cPlastic = Cd * (cAmbi + cDiff) + Cs *

cSpec;…

Simple Blinn-Phong shader expressed in both assembly and HLSL

10

nVidia’s CgFX Overview

11

nVidia’s CgFX Overview -2

• Supports Microsoft .fx files

• Cg plus:– Multi-pass– Hardware fallbacks (techniques)– Complete Hardware states– Tweakables

• MS .fx plus:– DirectX8 and OpenGL

12

nVidia’s CgFX Overview -3

• CgFX (.fx)– Manages whole rendering process– Handles render states – cross API support– Convenient exposure of tweakables & artist controls

• Cg Shaders (.cg)– semantics directives to match your C++ and other custom

hardware shaders– Bind textures/parameters to specific HW registers

• Cg Runtime– Thin API to compile on demand at runtime– Optimizes & manages .Cg for range of target HW

13

nVidia’s CgFX Viewer

14

nVidia’s CgFX Viewer -2

• Scene graph GUI• .fx parameters edition• Error reporting for easy .fx file problem

identification• Runs OpenGL, DirectX8, DirectX9

– Switch between devices at any point

15

Production Pipeline with CgFX

16

ATI’s RenderMonkey

17

ATI’s RenderMonkey -2

• Try to solve some of the problems developers face when designing software on emerging hardware

• Create a flexible, extensible shader development environment that allows easy incorporation of existing APIs – Support for low level DirectX8/9 in the current version

– Extensible framework to support emerging HLSL standards• DirectX9 HLSL • OpenGL 2.0 Shading Language• RenderMan • Maya Shade Trees • ……

18

ATI’s RenderMonkey -3

• Encapsulate all effect data in a single XML text file

• Each Effect Workspace consists of– Effect Group(s)

• Effect(s)– Pass(es)

• Render state• Pixel Shader• Vertex Shader• Geometry• Textures

– Variables and stream mapping nodes

19

Demos !

20

Bump Map Example

• Bump mapping simulates detail with a surface normal that varies across a surface

21

RenderMan Example

displacement

lumpy ( float Km = 1, frequency = 1, maxoctaves = 6;

string shadingspace = "shader";

float truedisp = 1;)

{

point Pshad = transform (shadingspace, frequency*P);

float dPshad = filterwidthp(Pshad);

float magnitude = fBm (Pshad, dPshad, maxoctaves, 2, 0.5);

N = Displace (normalize(N), shadingspace, Km*magnitude, truedisp);

}

22

23

Pixel Shader Example

ps.1.1

tex t0 // base map

tex t1 ; // bump map

tex t2 ; // light vector from normalizer cube map

dp3_sat r1, t1_bx2, t2_bx2 ; // N.L

mul r1, r1, c0 ; // N.L * diffuse_light_color

mul_sat r0, t0, r1; // (N.L * diffuse_light_color) * base

24

Cg Examplef2fb DiffuseBumpPS(v2f IN, uniform sampler2D DiffuseMap,

uniform sampler2D NormalMap, uniform float4 bumpHeight)

{

f2fb OUT;

float4 color = tex2D(DiffuseMap); //fetch base color

//fetch bump normal

float4 bumpNormal = expand(tex2D(NormalMap)) * bumpHeight;

//expand iterated light vector to [-1,1]

float4 lightVector = expand(passthrough(IN.LightVector));

//compute final color (diffuse + ambient)

float4 bump = uclamp(dot3_rgba(bumpNormal.xyz, lightVector.xyz));

OUT.col = color * bump;

return OUT;

}

25

26

Fixed Function Example

Texture[0] = <normalMap>;

Target[0] = Texture2D;

MinFilter[0] = Linear;

MagFilter[0] = Linear;

MipFilter[0] = Linear;

Texture[1] = <diffuseTexture>;

Target[1] = Texture2D;

MinFilter[1] = Linear;

MagFilter[1] = Linear;

MipFilter[1] = Linear;

ColorOp[0] = DotProduct3;

AlphaOp[0] = SelectArg1;

ColorArg1[0] = Texture;

ColorArg2[0] = Diffuse;

AlphaArg1[0] = Texture;

AlphaArg2[0] = Diffuse;

ColorOp[1] = Modulate;

ColorArg1[1] = Current;

ColorArg2[1] = Texture;

AlphaOp[1] = SelectArg1;

27

28

GL2 Logical Diagram

29

OpenGLFixed Function Vertex

TransformTransform[MVP],[MV],[MV][MVP],[MV],[MV]-T-T

LightinLightingg

[0,1][0,1]

[0,1][0,1]

TexgenTexgen TextureTextureMatrixMatrixnn

ColorColorSecondaryColSecondaryColoror

TexCoordTexCoordnn

EdgeFlagEdgeFlag

NormalNormal

Vertex Vertex (object)(object)

TexCoordTexCoordnn

EdgeFlagEdgeFlag

Vertex (eye)Vertex (eye)

Vertex (clip)Vertex (clip)

Front&BackFront&BackColorColor

Front&BackFront&BackSecondaryColSecondaryColoror

30

GL2 Vertex Processor

TemporariesTemporaries

VertexVertexShaderShader

UniformUniform

ColorColorSecondaryColSecondaryColoror

TexCoordTexCoordnn

EdgeFlagEdgeFlag

NormalNormal

Vertex Vertex (object)(object)

TexCoordTexCoordnn

EdgeFlagEdgeFlag

Vertex (eye)Vertex (eye)

Vertex (clip)Vertex (clip)

Front&BackFront&BackColorColor

Front&BackFront&BackSecondaryColSecondaryColoror

31

FrontFacing

ColorColor

Coord

Depth

OpenGLFixed Function Fragment

TexTexnn

TETEnn

SuSumm

FogFog[0,1][0,1]

Coord

FrontFacing

ColorColor

SecondaryColSecondaryColoror

TexCoord[TexCoord[nn]]

zz (|zzee|,f )

Depth

32

GL2 Fragment Processor

TexCoord[TexCoord[nn]]

FrontFacing

zz (|zzee|,f )

Coord

FrontFacing

ColorColor

Coord

ColorColor

SecondaryColSecondaryColoror

Depth Depth

TemporariesTemporaries

FragmenFragmentt

ShaderShader

UniformUniform TextureTexture

33

Using FX in Your Application• Load effect• Validate technique for hardware• Detect parameters for technique• Render Loop (for each object in scene):

– Set technique for the object– Set parameter values for technique– For each pass in technique

• Set state for pass

• Draw object

34

Using FX – The FX APILPD3DXBUFFER pError = NULL;D3DXCreateEffectFromFile(m_pd3dDevice, _T("simple.fx"),

NULL, NULL, 0, NULL, &m_pEffect, &pError);SAFE_RELEASE(pError);

. . .UINT iPass, cPasses;m_pEffect->SetTechnique(“Simple");

m_pEffect->SetVector(“var1", v);

m_pEffect->Begin(&cPasses, 0);for (iPass = 0; iPass < cPasses; iPass++){

m_pEffect->Pass(iPass);m_pMesh->Draw();

}m_pEffect->End();

35

Effect File Structure• An effect is made up of multiple rendering algorithms

(techniques) each made up of one or more passes• Effect File Structure:

Variable declarationsTechnique 1

• Pass 1• …• Pass n

…Technique n

• Pass 1• …• Pass n

36

.fx filefloat4x4 worldMatrix : World; // World or model matrixfloat4x4 mvpMatrix : WorldViewProjection; // Model * View * Projectionfloat4x4 worldViewMatrix : WorldView; // World * View

texture diffuseTexture : DiffuseMap // Diffuse Map semantic<

string File = "default_color.dds"; // Default texture file annotation>;

texture normalMap : NormalMap // Normal Map semantic<

string File = "default_bump_normal.dds";// Default texture file annotation>;

float4 ambientColor : Ambient < > = {0.1, 0.1, 0.1, 1.0};

float bumpHeight < // GUI annotations

string gui = "slider"; float min = 0; float max = 1; float step = 0.1;

> = 0.5;

37

.fx file

void DiffuseBumpVS(float4 Position : POSITION, // Position in object spacefloat2 TexCoord : TEXCOORD0,// Texture coordinates

…out float4 TexCoord0 : TEXCOORD0, // Texture coordinatesout float4 Position : POSITION)// Position in projection space

{TexCoord0.xy = IN.TexCoord.xy; // Pass texture coordinates for the diffuse map…Position = mul(WorldViewProj, IN.Position);// Compute position in projection space

}

void DiffuseBumpPS(float4 Position : POSITION, // Position in projection space…out float4 col : COLOR)

{float4 color = tex2D(DiffuseMap, UV); // Look up the diffuse map…col = color * light; // Modulate the diffuse color by the light

}

38

.fx filesampler2D diffuseSampler = sampler_state {

Texture = <diffuseTexture>;MinFilter = Linear;MagFilter = Linear;MipFilter = Linear;

};sampler2D normalSampler = sampler_state {

Texture = <normalMap>;MinFilter = Linear;MagFilter = Linear;MipFilter = Linear;

};technique CgTechnique // Both the vertex and the fragment shaders are in Cg{

pass p0{ ZEnable = true; ZWriteEnable = true; CullMode = None;

VertexShader = compile vs_1_1 DiffuseBumpVS(mvpMatrix, WorldIMatrix, lightPos); PixelShader = compile ps_1_1 DiffuseBumpPS(diffuseSampler, normalSampler, bumpHeight);}pass p1{...}

}

39

Cg Pipeline

Graphics programs are written in Cg ...

... and compiled to ...

... low-levelassembly code ...

Cg Runtime API

... that runs on any GPU compatible with DirectX or OpenGL

40

Using the Cg Compiler

//// Diffuse lighting//float d = dot(normalize(frag.N), normalize(frag.L));if (d < 0) d = 0;c = d*f4tex2D(t, frag.uv)*diffuse;…

…DP3 r0.x, f[TEX0], f[TEX0];RSQ r0.x, r0.x;MUL r0, r0.x, f[TEX0];DP3 r1.x, f[TEX1], f[TEX1];RSQ r1.x, r1.x;MUL r1, r1.x, f[TEX1];DP3 r0, r0, r1;MAX r0.x, r0.x, 1.0;MUL r0, r0.x, DIFFUSE;TEX r1, f[TEX1], 0, 2D;MUL r0, r0, r1;…

Cg programsource code

Shader programassembly code

Application Development Your Application

1) Load/bind program2) Specify program parameters3) Specify vertex inputs4) Render

Cg Compiler

Shader Compiler(nvasm.exe, psa.exe)

Shader Binary

41

Game Image

Application (game, renderer, …)

DCC Image

Typical Production Pipeline

DCC tool (Maya, Max, SoftImage, …)

Scene exporter plug-in

Scene managerApp Scene Manager

hard-coded to chooseat run-time

the appropriate ASM shaders + statefor the hardware

Artists create models, textures, maps, … in DCC tool of choice

ASM Shaders (HW1)

Programmerswrite

assembly for different hardware

ASM Shaders (HW2)

Not the same!Models, Textures, Maps, …

42

Game Image

Application (game, renderer, …)

DCC Image

FX-Enabled Production Pipeline

DCC tool (Maya, Max, SoftImage, …)

Scene exporter plug-in

For any FX, App Scene Manager chooses

at run-timethe appropriate

techniquefor the hardware

Artists assignFX files

to scene objectsand tweak parameters

for each objectin real-time

Same Image!

FX material plug-in

FX

files

Programmers and/or artists

write FX effects

FX runtime

Scene manager

Models, Textures, Maps, FX effects + parameters

43

Comparison

RenderMan OpenGL 2.0 D3D VS (2.0/3.0) D3D PS (2.0/3.0)

Program Size No limit No limit 256/256 256/256

No limit >=16 16/16

No limit >= 40 interpolators 10/10 10/10

No limit No limit 16/16

No limit No limit 12/16 12/16

Yes Yes Yes/Yes No/Yes

Yes Yes No/No No/No

Vertex Attributes

No limitVS:>=512 floatsPS: >= 64 floats

128/256 32/128Constants

Varying Parameters

Texture Samplers

Temp RegistersConstant-Based Flow Control

Variable-Based Flow Control