wilf lalonde ©2012 comp 4501 95.4501 rendering architectures

Wilf LaLonde ©2012Comp 4501

95.450195.4501

Rendering

Architectures


• There are 2 aspects to lighting.

• Light coloring; e.g., a red light colors nearby walls red.

• Light shadowing; e.g., walls not visible to a light are not colored...

PreamblePreamble

Lighting in this section deals exclusively with LIGHT COLORING. Occlusion knowledge is completely ignored...


• A forward renderer is one in which shader computations include all lighting aspects in the same shader (otherwise, it’s not a forward renderer).

• A light pass renderer is one which iterates over all the lights to process them for some reason.

• A light indexed renderer is one which maintains information about lights via indices rather than their properties...

• A deferred renderer is one that uses information previously stored in specific passes for use in a later pass...

TerminologyTerminology


Game Engine VariationsGame Engine Variations

Light Pass Deferred Renderers

Light-Prepass Deferred Renderer

Light-Indexed Light Pass Forward Renderers

Designing a Renderer for Multiple Lights: The Light Pre-Pass Renderer, Engel (Rockstar; GTA), pp 655-666, ShaderX7, 2009.

Designing a Renderer for Multiple Lights: The Light Pre-Pass Renderer, Engel (Rockstar; GTA), pp 655-666, ShaderX7, 2009.

Of interest for historical reasons

Forward Renderers

Light Pass Renderers

Light Indexed Renderers

Deferred Renderers

Light Pass Forward Renderers

Light-Indexed Light Pass Renderers


95.450195.4501

Renderers


• Overdraw is the notion that portions of the screen are drawn multiple times each with their own completely different shaders.

OverdrawOverdraw

camera

Top view

Much wasted GPU power goes in drawing the red walls when all

that is visible when done is the blue wall.


• Renderers deal with overdraw either by ignoring it (assuming a powerful GPU) or by invoking a special pass called the depth only pass (invented by Carmack in Doom3).

• The depth only pass draws all geometry with

color writing offdepth writing on, depth testing on

and subsequent passes are drawn with

color writing ondepth writing off, depth testing on

RenderersRenderers


• If we draw a building model, many walls will draw (most overdrawn by a closer wall)...

• With color writing off and depth writing on, only the vertex shader runs computing clip coordinate x/y/z values (in particular, the z-value establishes depth).

Depth Only Pass Only Uses Cheap Vertex ShaderDepth Only Pass Only Uses Cheap Vertex Shader

camera

Top view

Only this wall ends up WITH DEPTH INFO IN THE DEPTH BUFFER

AND NO PIXEL SHADERS RUN

2 to 8 times faster if color writing is off (Engel, Rockstar Games)


• With color writing on and depth testing on (but not depth writing), the complex shader runs only IF the depth test is passed.

The 2nd Standard Draw PassThe 2nd Standard Draw Pass

camera

Top view

Only this wall ends up being visible (Vertex shader AND COMPLEX PIXEL SHADER RUNS) because it’s visible...

These ALL FAIL THE DEPTH TEST (Vertex shader runs BUT NOT PIXEL SHADER)


• The simplest approach used by older engines is to have each pixel shader loop over the lights (if you have many objects, it’s equivalent to)

Dealing with Several LightsDealing with Several Lights

for each object dofor each light do

lighting += computeLighting (object, light);

Feasible for a handful of lights BUT NOT FOR 200 LIGHTS MOST OF WHICH ARE TOO FAR AWAY FROM THE PIXEL

what each shader does.


• A more efficient approach breaks objects into smaller objects and determines which lights affect which objects and draws one at a time.

Dealing with Several LightsDealing with Several Lights

for each objectfor each light that can affect this object do


1. Instead of 1 draw call, you have many draw calls (substantially slower).

2. The engine has to do a lot of work breaking up the large draw call and determining which lights affect which objects (ALL DONE ON CPU)

for each light dofor each object affected by light do


OR


• Instead of an O(n2) loop of form

OR

Taking the Lighting Computation Completely Out Of The Standard Renderer

Taking the Lighting Computation Completely Out Of The Standard Renderer

for each object dofor each light than can affect this object do


for each light dofor each object affected by light do


• We would like two O(n) loops...


More specifically, we would LikeMore specifically, we would Like

for each object (via large triangle buffers) do g-buffer = non-light specific lighting properties

of visible object;

for each light dolighting += light_model (g-buffer, light);

A g-buffer is a buffer of geometric attributes (in this case, lighting attributes); implemented as a collection of textures each able to hold 4 float attributes...

Light Pass Renderers

Deferred Renderers


• Forward renderer: conventional style that processes lighting in every single shader.

• Deferred renderer: non-conventional style that tries to deal with lighting in a separate lighting pass by grabbing and deferring the use of some information.

Problems with TransparencyProblems with Transparency

Both have problems dealing with transparent objects since these need to be (1) captured, (2)

sorted by depth, and (3) drawn last…

Both have problems dealing with transparent objects since these need to be (1) captured, (2)

sorted by depth, and (3) drawn last…


• Game engine needs to break world geometry into groups affected by sets of lights (say a dozen). Leads to many draw calls.

• Each group needs to be drawn separately. Not too hard for static lights... much harder for dynamiclights...

• Groups affected by more than the light limit need to be drawn again BLENDING the new lighting with the old (multitexturing)

Lighting with Forward Renderers: SummaryLighting with Forward Renderers: Summary

You can’t have every shader loop over 100 lights…You can’t have every shader loop over 100 lights…

Many draw calls

Many state changes


• Scales poorly with complex lighting or large numbers of lights.

• Increasingly important as the game quality climbs up.

What’s Wrong With Forward RenderersWhat’s Wrong With Forward Renderers

Particularly with hundreds of lights…Particularly with hundreds of lights…


• Shaders are rewritten to produce deferred information but don’t themselves use this deferred information. Drastically cuts down on the number of shaders; e.g., a relief mapper, non-relief mapper, skinning shader.

• On PC at least, such renderers use MRT (multiple render targets) to support g-buffers with many attributes; e.g. 24 floats.

Lighting with Deferred RenderersLighting with Deferred Renderers

Not clear what kinds of special effects are no longer supported.

Not clear what kinds of special effects are no longer supported.


• One pass deals with non-lighting.• One pass deals with lighting (they are light

pass renderers).

• By not mixing them, the goal is to simplify the game engine... and in the process, be able to support a much larger set of lights.

Deferred Rendering is a multiphase techniqueDeferred Rendering is a multiphase technique

There are 2 different ways of tracking the lighting information: either the lights

themselves or the lights’ lighting materials

There are 2 different ways of tracking the lighting information: either the lights

themselves or the lights’ lighting materials


95.450195.4501

Variations of the

Basic Deferred

Renderer


• Standard deferred renderer use 4 texture g-buffer.

• Light prepass deferred renderer uses 2 texture g-buffer; recovers missing information by rendering geometry twice...

Variations Due To G-Buffer StingynessVariations Due To G-Buffer Stingyness

Normal.x Normal.y Normal.z Unused

Diffuse Diffuse Diffusematerial.x material.y material.z Unused

Specular Specular Specular Specular material.x material.y material.z power

Position.x Position.y Position.z Unused

R G B A




• Standard deferred renderer

• Light prepass deferred renderer (poor name)

Another ViewpointAnother Viewpoint

REAL draw

geometry

SIMPLE draw

geometry

draw lights

large g-buffer

small g-buffer

draw lights

light map

light map

REAL draw

geometry

simple DRAW QUAD combine

back buffer

back buffer

geometry

geometry

LIGHTS AFTER MATERIALS

LIGHTS BEFORE MATERIALS

Uses geometry materials to

finish lighting

“real draw after lights” renderer


Hieroglyph3 FPS Comparison: 33 = 27 lights Hieroglyph3 FPS Comparison: 33 = 27 lights

206 85

• Standard renderer more than twice as fast

standarddeferred renderer

light prepass deferred renderer

Deferred Rendering, Practical Rendering and Computation with Direct3D11 pp 491-541, CRC Press, 2011.



Hieroglyph3 FPS Comparison: 73 = 343 lights Hieroglyph3 FPS Comparison: 73 = 343 lights

39 27

• Neither is a winner (BOTH too slow)...

Varied 26-37Varied 31-41

standarddeferred renderer

light prepass deferred renderer


• To run the hieroglyph demo, startup Visual Studio 2010.

• Select the “Deferred Rending” OR the “LightPrepass”project.

• Select “Project” and click on “Set as startup project”.

• Then RUN.

Note to WilfNote to Wilf


• We will only investigate standard deferred renderers.

• No clear advantage to light prepass deferred renderer.

• We will also investigage light-Indexed renderers (which like the above are light pass renderers) but pick up which light instead of which light information (each light has its own lighting information).

ConclusionsConclusions


95.450195.4501

Standard Deferred

Renderers


Hieroglyph3 Demo (343 Lights)Hieroglyph3 Demo (343 Lights)




• Pass 1 renders a g-buffer (which is where the relief map shader would run; the demo only support normal mapping; can’t do self occlusion since lights not yet available).

• Pass 2 renders the lights to create a light map; each light adds to it…

• Pass 3 draws one fullscreen quad to blend materials in the g-buffer with the light map (output to the backbuffer).

Investigating Hieroglyph3: Traditional Deferred RenderingInvestigating Hieroglyph3: Traditional Deferred Rendering

so we won’t show shader code for this.

It is advantageous to have a z-buffer prepass 0 so that pass 1 does not run multiple overdrawn relief map shaders…

Warning: Converts everything to world rather than camera space.


95.450195.4501

Pass 1 of Standard

Deferred Renderers:

Rendering G-Buffer


Hieroglyph3: The G-Buffer Rendering ShaderHieroglyph3: The G-Buffer Rendering Shader

• The g-buffer layout… NOTHING LIGHT SPECIFIC.

R G B A

Render target 0:

Render target 1:

Render target 2:

Render target 3:


Diffuse Diffuse Diffusealbedo.x albedo.y albedo.z Unused

Specular Specular Specular Specular albedo.x albedo.y albedo.z power


Albedo MEANS material…


• The demo provides additional passes to perform MSAA (multi-sample anti-aliasing) if you enable it in the user interface (too complicated to consider).

Hieroglyph3Hieroglyph3

Warning: The shader code presented below has been slightly simplified for presentation purposes.

• The “.cpp” code is

for all objects in world dodraw via g-buffer rendering

shader Gbuffer.hlsl


//Vertex shader and pixel shader for filling the G-Buffer of STANDARD//deferred renderer (SLIGHTLY MODIFIED BY WILF)//Note: The word albedo MEANS material…

cbuffer Transforms {matrix WorldMatrix;matrix WorldViewMatrix;matrix WorldViewProjMatrix;

};

cbuffer MatProperties {float3 SpecularAlbedo;float SpecularPower;

};

Texture2D DiffuseMap : register (t0);Texture2D NormalMap : register (t1);SamplerState AnisoSampler : register (s0);


Albedo MEANS material…

Diffuse map is the texture map...

Should discard register keywords


struct VSInput {float4 Position : POSITION;float2 TexCoord : TEXCOORDS0;float3 Normal: NORMAL; float4 Tangent: TANGENT; float4 Bitangent: BITANGENT;

};

struct VSOutput {float4 PositionCS: SV_Position;float2 TexCoord: TEXCOORD; float3 NormalWS: NORMALWS; float4 PositionWS: POSITIONWS;float3 TangentWS: TANGENTWS;float3 BitangentWS: BITANGENTWS;

};

struct PSInput {float4 PositionSS : SV_Position;float2 TexCoord: TEXCOORD;float3 NormalWS: NORMALWS; float4 PositionWS: POSITIONWS;float3 TangentWS: TANGENTWS;float3 BitangentWS: BITANGENTWS;

};

struct PSOutput {float3 NormalWS: SV_TARGET0; float4 DiffuseAlbedo: SV_TARGET1;float3 SpecularAlbedo: SV_TARGET2;float3 PositionWS: SV_TARGET3;

};


CS MEANS clip space (not camera space)…

WS MEANS world space…

SS MEANS screen space (camera space)…

SV_Position means x/y are texture coordinates rather than deuce box -1/+1

coordinates (even though the vertex shader computes it as clip coordinates)

4-way MRT (multiple render targets)

Added by WILF

Added WS by WILF


VSOutput VSMain (in VSInput input) {VSOutput output; //All except PositionCS is converted to world space.//Convert position and normal to world space.Output.PositionWS = mul (input.Position, WorldMatrix).xyz;output.normalWS = normalize (mul (input.Normal, (float3x3)

WorldMatrix));

//Build the tangent frame vectors. output.TangentWS = (normalize (mult (input.Tangent.xyz), (float3x3)

WorldMatrix); output.BitangentWS = (normalize (mult (input.Bitangent.xyz), (float3x3)

WorldMatrix); //Calculate the clip space position. output.PositionCS = mul (input.Position, WorldViewProjMatrix);

//Pass along the texture coordinate.Output.TexCoord = input.TexCoord;return output;

}


Code and comments slightly modified by Wilf

Note that it’s just as easyto convert everything into

CAMERA space to match what we did when computing ambient

occlusion


PSOutput PSMain (in PSInput input) {VSOutput output;//Sample the diffuse map.float3 diffuseAlbedo = DiffuseMap.Sample (AnisoSampler,

input.TexCoord).rgb;

//Build the tangent frame. float3x3 tangentFrameWS = float3x3 (normalize (input.TangentWS),

normalize (input.BitangentWS), normalize (input.NormalWS));//Sample the tangent space normal map and convert to vector.float3 normalTS = NormalMap.Sample (AnisoSampler,

input.TexCoord).rgb;normalTS = normalize (normalTS * 2 – 1); //Convert from 0/1 to -1/+1

coords.

//Convert from tangent space to world space.Float3 normalWS= mul (normalTS, tangentFrameWS);

//Output the g-buffer. output.NormalWS = float4 (normalWS, 1.0);output.DiffuseAlbedo = float4 (diffuseAlbedo, 1.0);output.SpecularAlbedo = float4 (specularAlbedo, SpecularPower);output.PositionWS = float4 (input.PositionWS, 1.0); return output;

}


WS MEANS world space…

Shader variables

TBNWS matrix

support normal mapping


95.450195.4501

Pass 2 of Standard

Deferred Renderers:

The Light Pass


• The frame buffer is set up to draw into a lightmap texture initially all black using an additive blend (1 * source + 1 * destination)…

• The engine draws the light shapes one by one where a point light is a sphere, a spot light is a cone, and a directional light is full screen quad (since you can’t place it anywhere)…

Hieroglyph3: The Light Rendering ShaderHieroglyph3: The Light Rendering Shader

Warning: There is a problem drawing light shapes. We’ll address it later…

• The “.cpp” code is for all lights in world do

draw light shape via light rendering shader


• The demo provides one shader with macros that are externally defined.

• By compiling the shader ONCE for each light type, it gets the effect of having multiple shaders.

• We’ll add an extra lightType variable to the shader so we can use one shader...

Other ChangesOther Changes

Lights.hlsl


//Vertex shader and pixel shader for drawing lights in a STANDARD//deferred renderer (SLIGHTLY MODIFIED BY WILF)//The following 4 textures make up the g-buffer...Texture2D NormalWSTexture : register (t0);Texture2D DiffuseAlbedoTexture : register (t1);Texture2D SpecularAlbedoTexture : register (t2);Texture2D PositionWSTexture : register (t3);

cbuffer LightParameters {float3 LightPositionWS;float3 LightColor; float3 LightDirectionWS; float2 SpotlightAngles; float LightRadius; int LightType;

};

cbuffer CameraParameters {float3 CameraPositionWS;

};


In world space…

Added by WILF

Changed by WILF

Added WS by WILF

Added WS by WILF

Added WS by WILF


struct VSInput {float4 Position : POSITION;

};

struct VSOutput {float4 PositionCS: SV_Position;

};

VSOutput VSMain (in VSInput input) {//Calculate the clip space position. output.PositionCS = mul (input.Position, WorldViewProjMatrix);

return output;}


Doesn’t really do anything (other than converting to clip space)


#define clamp01 saturate

#define innerAngleAsCosine SpotlightAngles.x /* larger angle */#define outerAngleAsCosine SpotlightAngles.y /* smaller angle */

float spotLightAttenuation (float3 toLight) {//Spotlight specific attenuation for inside/outside spot light zone.float pixelAngleAsCosine = dot (-toLight, LightDirectionWS);//Before clamping, >1 if > inner; < 0 if < outer; 0 to 1 otherwise.return clamp01 (

(pixelAngleAsCosine – outerAngleAsCosine) / (innerAngleAsCosine – outerAngleAsCosine ));

}

Hieroglyph3: A Soft SpotlightHieroglyph3: A Soft Spotlight

WILF: Tricky to Decipher intensity 0 outside outer zone,

0 to 1 between outer and inner zone (fuzzy area), 1 in inner zone (bright area)

inner

outer1

2

bright area

fuzzy area


#define POINT_LIGHT 0#define DIRECTIONAL_LIGHT 1#define SPOT_LIGHT 2

void readGBufferAttributes (float2 screenPosition, out float3 pixelPositionWS, out float3 normalWS, out float3

diffuse, out float3 specular, out float specularPower) {int3 coordinate = int3 (screenPosition.xy, 0); //So we use LOD 0...pixelPositionWS = PositionWSTexture.Load (coordinate).xyz;

normalWS = NormalWSTexture.Load (coordinate).xyz; diffuse = DiffuseAlbedolTexture.Load (coordinate).xyz;

float4 fullSpecular= SpecularAlbedoTexture.Load (coordinate);specular = fullSpecular.xyz; specularPower = fullSpecular.w;

}


Pick up all g-buffer attributes via 2D screenPosition

WILF: Enum instead of bool


float4 lightingFor (int lightType,in float3 pixelPositionWS, in float3 normalWS, in float3 diffuse, in float3 specular, in float specularPower) {float3 toLight; float attenuation; //To be computed 3 ways...switch (lightType) {

case POINT_LIGHT:toLight = LightPositionWS – pixelPositionWS;float3 distance = length (toLight); toLight /=

distance; //normalizeattenuation = max (0.0, 1.0 – distance/lightRadius);break;

case DIRECTIONAL_LIGHT:toLight = -LightDirectionWS; attenuation = 1.0;break;

case SPOT_LIGHT:toLight = LightPositionWS – pixelPositionWS;float3 distance = length (toLight); toLight /=

distance; //normalizeattenuation = max (0.0, 1.0 – distance/lightRadius);attenuation *= spotLightAttenuation (toLight); break;

}


continued

same


//Calculate light visibility.float visibility = clamp01 (dot (normalWS, toLight));//Calculate the specular term.float3 toCamera = CameraPositionWS – pixelPositionWS;float3 halfwayVector = normalize (toLight + toCamera);float3 specularity =

pow (clamp01 (dot (normalWS, halfwayVector), specularPower) *

specular;return //Convert float3 to opaque float4 result...

float4 (visibility * LightColor * (diffuse + specularity) * attenuation, 1.0);}


Not much different than our lighting routines but has 3 types.


struct PSInput {float4 screenPosition : SV_Position;

};

float4 PSMain (PSInput input) : SV_Target0 {float3 pixelPositionWS; float3 normalWS; float3 diffuse, float3 specular; float specularPower;readGBufferAttributes (input.screenPosition.xy,

pixelPositionWS, normalWS, diffuse, specular, specularPower);

return lightingFor (lightType,pixelPositionWS, normalWS, diffuse, specular,

specularPower) ; }


SV_Position means x/y are texture coordinates rather than deuce box -1/+1

coordinates (even though the vertex shader computes it as clip coordinates)

For a red light, this would draw 1 red pixel. When the next light draws, it might draw a green pixel at the same place which is

blended onto the lightmap texture (to give yellow = red + green)


• All triangles are outward facing

An Example Light ShapeAn Example Light Shape

• To draw front faces, enable back face culling.

• To draw back faces, enable front face culling.

camera

camera


• What’s so hard about drawing a light shape? A wall needs to go red if it’s IN a red light...

Drawing a Light ShapeDrawing a Light Shape

camera camera

Should notdraw on walls

(not touching it)Both Should draw on walls

(red part)

problem

camera

Do you draw front faces or back faces?

tough casetough case


• Original technique given in terms of complex stencil buffer rules.

• We provide those rules but will skip them so we can present a simpler approach that doesn’t need the stencil buffer...

Presentation DeviationPresentation Deviation

But, like for the ambient occlusion demo, we need to be using camera space for everything instead of world space.

We’ll assume shaders can be rewritten to use camera space instead...


• Need a shape with outward-facing faces. Initially, mask for each pixel cleared to 0. Draw with z-buffer testing and writing disabled.

• 1st stencil draw: Draw only front faces on mask with rule: “If pixel behind z-buffer, set mask 1”.

• 2nd color draw: Draw only back faces on non-mask with rule: “If mask 0 and pixel behind z-buffer, draw”. In any case, reset mask to 0.

Drawing a Light Shape: One Way Via Stencil Buffer (Conceptually, a Mask)Drawing a Light Shape: One Way Via Stencil Buffer (Conceptually, a Mask)

This is SIMILAR but NOT IDENTICAL to what all published deferred renderers seem to do including this one.

Wilf SKIP

THIS



camera camera


(not touching it)Both Should draw on walls

(red part)

camera

1st stencil draw: Draw only front faces on mask with rule: If pixel behind z-buffer, set mask 1. RULE OUT WHERE FRONT FACES ARE BEHIND Z-BUFFER.

2nd color draw: Draw only back faces on non-mask with rule: If mask 0 and pixel behind z-buffer, draw. In any case, reset mask to 0. RULE IN BACK FACES BEHIND Z-BUFFER.

RULE OUTRULE IN

Wilf SKIP

THIS


• Requires that we work in camera space...When drawing light shape, we have positionLIGHT

From uv = positionLIGHT .xy, we can get positionMRT .

• Does NOT require 2 passes...

• Can all be done by the shader...

• UNTESTED...

Coloring by Drawing Light Shapes Without Stencil BuffersColoring by Drawing Light Shapes Without Stencil Buffers


Accessing MRT at light pixel uv coordinateAccessing MRT at light pixel uv coordinate

camera

wallWall pixel

in CAMERA space

Drawing world stores into MRT texturespositionMRT, normalMRT, colorMRT, etc. at particular UV screen coordinate

camera

wall

Light pixel positionLIGHT has UV screen coordinate used to access wall pixel information in MRT

Light pixelin CAMERA space

MRT Drawing

Light DrawingWAS CALLED pixelPositionWS

UV

UVOne of many

triangles in the light shape

WAS type SV_Position

MRT IS ENCODING OF

WORLD

DRAWING INTO

LIGHTMAP

positionLIGHT

positionMRT


z = -100 + discard the pixel (not draw) if

positionLIGHT.z > positionMRT.z

positionMRT distance to light center > light radius

(normalMRT dot (toLightMRT)) < 0

Light Draws if it TOUCHES an object; e.g., a WallLight Draws if it TOUCHES an object; e.g., a Wall

“outside the light”

“can’t see the light”

“in front”

toLightMRT = normalize (light center - positionMRT ) CameraLOOKING UP

light can’t reach wall

wall can’t reach light

wall lit

wall normal aligned against tolight

• To draw light shape “if it lights wall”, draw ONLY back faces + disable depth testing/writing

positionMRT positionLIGHT

wall

z = -10



camera

camera


(not touching it)

Both Should draw on walls(red part)

camera

If “in front” || “outside the light” || “can’t see the light” {discard;}

camera

lit lit


95.450195.4501

Review: Standard

Deferred Rendering


• Geometry phase: Process the scene with z-buffered drawing attributes to an intermediate buffer (G-buffer; typically several textures) for use later, usually via MRT.

• Composition phase: Create a light map from the G-buffer portion that survives depth buffering. Blend it on top to colorize result.

Standard Deferred RenderersStandard Deferred Renderers

Would be simpler if we could output into a buffer consisting of an arbitrary struct (instead of multiple RGBA textures)

Would be simpler if we could output into a buffer consisting of an arbitrary struct (instead of multiple RGBA textures)

avoids standard shaders looping

over lightsMay also want a

depth only prepass


• Material shaders do not perform lighting.• New light types or models can be added

without changing material shaders.• Lighting costs independent of scene

complexity.• Lighting costs scales linearly with the

number of lights.• No overhead to determine which lights

affect which materials.

Advantages of Standard Deferred RenderersAdvantages of Standard Deferred Renderers


• Creating a lightmap using additive blending will ultimately give a washed out look...

(0.4, 0.4, 0.4)

+ (0.4, 0.4, 0.4)

+ (0.4, 0.4, 0.4)

= (1.0, 1.0, 1.0)

Disadvantages of Light Map ApproachDisadvantages of Light Map Approach

since it gets clamped to 1

If you have too many lights on the same pixel...


• Position, Normal.• Material parameters (diffuse, emissive,

specular components, specular power, etc).• Typical layout 128 bits per pixel = 12 meg @

1024x768 (4 MRT + depth):• Depth R32F

• Position A32B32G32R32F

• Normal + scattering A2R10G10B10

• Diffuse color + emissive A8R8G8B8

• Specular intensity + power, occlusion + shadow factor A8R8G8B8

Typical PropertiesTypical Properties

Deferred Shading, Hargreaves, Game Developers Conference, 2004.

Deferred Shading, Hargreaves, Game Developers Conference, 2004.


95.450195.4501

Light Indexed Deferred

Renderers


Humus Deferred Rendering Demo (Emil Persson)Humus Deferred Rendering Demo (Emil Persson)


Trebilco Modified Humus Demo (256 lights)Trebilco Modified Humus Demo (256 lights)

Light-Indexed Deferred Rendering, Trebilco, pp 243-256, ShaderX7, 2009.



• There exists a single array of lights (whatever is needed to describe the light); either as shader variables accessible to the lighting shader (easiest is to use a NAMED constant buffer which is a system wide global for all shaders) or as a texture with the data encoded within it...

• The trick is to build a full screen texture in which each pixel contains the indices of 4 lights (this is easy to build with shader bit manipulation but tricky to understand).

Light-Indexed Deferred RenderersLight-Indexed Deferred Renderers

If the light shapes were handled as models, we could use instancing to draw the lights in ONE DRAW instead of one by one…


• Standard z-Buffer prepass deferred renderer

• Light-indexed renderer

Comparing With Z-Buffer Prepass Deferred RendererComparing With Z-Buffer Prepass Deferred Renderer

REAL draw geometry

draw lights

large g-buffer

light map

simple DRAW QUAD combine

back buffer

Z-Buffer prepass draw

geometry

geometry

SIMPLE flat triangle draw geometry with

z-buffer setup

draw lightindicies

small g-buffer

light index map

back buffer

REAL draw geometry

with 4 lightsgeometry

To draw light indices, need position + normal + z-buffer (real geometry can compute its own)

or create light map like above


• When you draw light 27, have the shader that draws the light shape return an encoded version of 27 as a color...

• Later when you want light information like the color, fetch 27 from the light index map by decoding it and get the color information from “Light [27]”.

So What’s Different?So What’s Different?


• Let n be the number of bits for the light index; e.g., n = 16 for RGBA16 textures (for 64K lights)...

• Divide the number into 4 parts, each 4 bits for n = 16:

and encode as an RGBA color:

How Do You Encode 27 into a Color...How Do You Encode 27 into a Color...

0 0 0 0 0 0 0 0 0 0 0 0

a b c d

a b c d

red green blue alpha


• Ultimately, a color in the color index map can encode 4 light indices.

How Do You Encode 27 into a Color...How Do You Encode 27 into a Color...

Light index 0: all 1st parts

Light index 1: all 2nd parts

Light index 2: all 3rd parts

Light index 3: all 4th parts

a1 b1 c1 d1a2 b2 c2 d2a3 b3 c3 d3a4 b4 c4 d4

For RGBA16 texture, each quarter is 4 bits



• Set the light index texture that you want to draw into with a special blend mode...

Destination: Constant color c wherec = [0.0625, 0.0625, 0.0625,

0.0625]Source: 1

• The way blending works, whatever you draw, the card blends as follows.

What’s already there * constant color

+

New color * 1

How to Setup For Drawing How to Setup For Drawing

What’s so special about 0.0625?


number >> 1 means divide by 2 or * 0.5number >> 2 means divide by 4 or * 0.25number >> 3 means divide by 8 or * 0.125number >> 4 means divide by 16 or * 0.0625

How Do We Shift Right 4?How Do We Shift Right 4?

1 / 16 = 0.0625


• Summary of blendingNew color {a,b,c,d} + (What’s already there >> 4)

So it Performs?So it Performs?

If you draw 6 lights on the same pixel, you do lose 2 of them…

a1 b1 c1 d1a2 b2 c2 d2a3 b3 c3 d3a4 b4 c4 d4


0 0 0 0a1 b1 c1 d1a2 b2 c2 d2a3 b3 c4 d3after >> 4

0 0 0 0 0 0 0 0 0 0 0 0a b c d

already there

+ new color

a b c da1 b1 c1 d1a2 b2 c2 d2a3 b3 c4 d3answer

• Example


• Download a constant buffer of light information (can change every tick; don’t if it doesn’t).

• Pass 1: Render scene into g-buffer with simpler shader to set up position, normal and depth buffer only. After disable depth writes.

• Pass 2: Render light volumes into a light index texture (initially RGBA = 0) with magic blend. Outputs last 4 light indices that can affect this pixel (index 0 means no light)..

• Pass 3: Render scene as forward rendering using 4 lights provided in light index texture (the only complex shader pass)...

Light-indexed Rendering: SummaryLight-indexed Rendering: Summary



Transparent objects can be done via forward rendering pass 4 like all other engines


• Since it has light indices, can use many types of light models and blend the results using arbitrary shader code... (new possibilities; e.g., a light cancelling light or saturation prevention function; e.g., instead of adding, use s-style function from 0 to 1)

Blending color with other techniques can only use rigid hardware blend rules.

• Note that light positions are available after light draw pass. So you can do “lit fog” in transparency phase...

Advantages of Light-Indexed RenderingAdvantages of Light-Indexed Rendering

0

1

0

1or


• Relief map self shadowing is easy to do since you have AT MOST 4 coloring lights to query.

What do you do in deferred renderer approach (you have only blended color; NO POSITION list)?

• Pipeline is simple since g-buffer is smaller...

• MSAA as a result is easier to implement.

• Not all cards have MRT support; e.g., mobile game. Simulating small g-buffer is possible.

Advantages of Light-Indexed RenderingAdvantages of Light-Indexed Rendering


• It could be argued that 4 lights per pixel is not enough? As a player, could you tell that a light or two is missing when their colors interact to begin with.

LimitationsLimitations

LIMIT OF 8 CAN BE DONE WITH rgba32 TEXTURE.


95.450195.4501

Special Topic: Treating

Transparent Draws AS IF

THEY WERE OPAQUE


• In the geometry phase, interlace opaque and transparent objects; e.g., odd lines are opaque and even lines are transparent.

• In the composition phase, de-interlace the images and blend the transparent part on top of the opaque parts; via a pixel shader that samples 2 pixels…

Deferred Rendering of TransparencyDeferred Rendering of Transparency

Downside: Only 1 transparency layer…; NO MSAA.

Deferred Rendering Transparency, Pangerl, pp 217-225, ShaderX7, 2009.

Deferred Rendering Transparency, Pangerl, pp 217-225, ShaderX7, 2009.


95.450195.4501

Conclusions


• Light indexed renderers seems to have more advantages than traditional deferred renderers.

• Some notion of deferred properties are still needed though it’s difficult to argue that this means it’s a deferred renderer (does supporting ambient occlusion mean you are using a deferred renderer).

• Since it can be implemented primarily as a forward renderer, it also handles transparency “natively”.

ConclusionsConclusions


• Deferred Shading, Hargreaves, Game Developers Conference, 2004.

• Overcoming Deferred Shading Drawbacks, Placeres, pp 115-130, ShaderX5, 2007.

• Deferred Shading in Tabula Rasa, Koonce, pp 429-457, GPU Gems 3, 2008.

• Deferred Rendering Transparency, Pangerl, pp 217-225, ShaderX7, 2009.

• Light-Indexed Deferred Rendering, Trebilco, pp 243-256, ShaderX7, 2009.

• Designing a Renderer for Multiple Lights: The Light Pre-pass Renderer, Engel (Rockstar Games), pp 655-666, ShaderX7, 2009.

• Light Pre-Pass Renderer: Using the CIE Luv Color Space, Engel (Rockstar Games), pp 667-677, ShaderX7, 2009.

ReferencesReferences


• Global Lights are things like sunlight, fog, and directional lights that affect the entire scene.

• Draw them as a fullscreen quad where position, normal, color, and material settings are read from texture inputs.

• Lighting calculation is evaluated in the pixel shader.

• Output goes to an intermediate lighting buffer OR direct to the backbuffer.

Aside: Drawing Global LightsAside: Drawing Global Lights

Often handled specially anyway to deal with shadows…

wilf lalonde ©2012 comp 4501 95.4501 rendering architectures

Documents

light prepass renderer

light indexed renderer

depth bufferand

depth info

depth test vertex shader

light coloring

later pass

special pass