unreal summit 2016 seoul lighting the planetary world of project a1
TRANSCRIPT
UNREAL SUMMIT 2016
Lighting the Planetary World of Project A1
Hyunwoo KiLead Graphics Programmer
UNREAL SUMMIT 2016
A1
• New IP of Nexon
– High-End PC / AAA-quality visuals
– MOBA / Space Opera
– UE4 + @@@
– In Development
• Announced our development last month
UNREAL SUMMIT 2016
A1
• Talk about character rendering at last NDC 2016 talk
• This talk presents techniques for lighting the world of A1
– Used a test scene
– Not the game world
UNREAL SUMMIT 2016
World of A1
• Spherical planet
• Real-time day and night cycle
• Partially environment destruction
– Trees, buildings, etc.
• Partially terrain modification
– Craters, explosion, etc.
UNREAL SUMMIT 2016
Challenges
• Spherical coordinates
• Longitudinal time variation
• Time of day lighting changes
• Dynamic
– Moving Sun
– Destruction
– Modification
UNREAL SUMMIT 2016
Our Approach
• “Fully Dynamic” if possible
– Shadow maps, SSAO, SSR, SSIS Screen Space Inner Shadows, etc.
• Partially Precomputation and Relighting
– Global illumination, sky lighting and reflection environment
UNREAL SUMMIT 2016
Changes for Planet: 1
• Vector towards the sky
– Vary according to latitude and longitude
– Standard world: just (0, 0, 1)
– Planetary world: normalize(WorldPosition)
• Assuming (0, 0, 0) is the center of the world
• We call this ‘Planet Normal’
UNREAL SUMMIT 2016
Changes for Planet: 2
• Longitudinal time variation
– Like GMT
float ComputeGMTFromWorldPosition(float3 WorldPosition){
float Longitude = atan2(WorldPosition.y, WorldPosition.x) / PI * 0.5f + 0.5f; // [0, 1]float NormalizedGMT = frac(Frame.NormalizedDayTime – Longitude); // East to Westreturn abs(NormalizedGMT);
}
* Note: Frame.NormalizedDayTime = GMT+0 = [0. 1)
UNREAL SUMMIT 2016
Agenda
• Directional Light
• Global Illumination
• Sky Light
• Reflection Environment
UNREAL SUMMIT 2016
Directional Light
UNREAL SUMMIT 2016
Directional Light
• Symmetry of two directional lights
– Sun and Moon
• Movable mobility
– For dynamic scenes
– Real-time lighting
• Deferred rendering: opaque materials
• Forward+ rendering: transparent materials
UNREAL SUMMIT 2016
Two of Directional Lights
• Sun:
– Dominant light
• Moon:
– Night area
– Adding direct specular
UNREAL SUMMIT 2016
Two of Directional Lights
• Problem:
– Directional lights commonly affect all surfaces on the world
– Incorrect results
• Ex) Moonlight leaks in the daytime
• Slow (2X)
• Solution:
– Cull backside of the planet from the light
– Smoothly attenuate radiance at boundaries
UNREAL SUMMIT 2016
Time of Day Lighting
• Different time for each pixel
• Overriding light color by using a hand-painted texture in the shader
• Different methods for GI, sky light and reflection environment
– See further slides
UNREAL SUMMIT 2016
Sun Shadows
• Shadows have an important role to recognize time during game play
• Presented at my NDC 2016 talk
– Use UE4 implementation
• CSM + PCF
– Add improved PCSS
• To control shadow softness by time: only for 0 and 1 cascade splits
• Shadow normal offset: to remove Peter Panning
• Temporal reprojection: to reduce flickering due to slow moving
UNREAL SUMMIT 2016
Tighter Shadow Bounds
• For both quality and speed
• Setting tighter bounds
– Assuming very far objects on the view do not cast shadows
– Backside of the planet culling + planetary view frustum culling
– More than 1.5X faster rendering
UNREAL SUMMIT 2016
Planetary View Frustum Culling
• Limit the far plane as distance between the camera and the center of the planet
– Assume that we can’t see backside of the planet
• Closer the camera, shorter the far plane
– A proportional expression betweenthe center of the planet and view frustum planes
UNREAL SUMMIT 2016
Before
Backside culling
Backside culling+ View frustum culling
UNREAL SUMMIT 2016
Global Illumination
UNREAL SUMMIT 2016
Existing Solutions in UE4
• Lightmaps (X)
– Static, and high memory consumption
• LPV (X)
– Slow, and low quality
• DFGI (X)
– Slow, and not supporting skeletal meshes
• Indirect lighting cache
– Be possible!
UNREAL SUMMIT 2016
UE4 Indirect Lighting Cache
• SH irradiance volume
• Per-primitive caching
– 5x5 volume -> upload to the global volume texture atlas
– For movable components or preview / Update when the component is moved
• “Try to use volume ILC for all types of components in the scene”
– Including static components and terrain
UNREAL SUMMIT 2016
Lacks of Volume ILC
• Lighting discontinuity (a.k.a. seams)
• Low density at a large geometry
– Need size-dependent cache distribution
• High memory consumption and slow cache update
• High CPU costs
– Per-primitive computation on the render thread
– Need update cache if a primitive is moved or lighting is changed
• Unsuitable for our game
UNREAL SUMMIT 2016
Need of a New Method
• Keep using SH irradiance volume
• More efficient data structure
• Seamless
• Faster update (or no cache update)
• Time of day lighting changes
UNREAL SUMMIT 2016
Related Work
• Far Cry series
• Assassin Creed series
• Quantum Break
• TC: The Division
• …
UNREAL SUMMIT 2016
Deferred Cubic Irradiance Caching
• ‘Deferred’:
– As post processing: avoiding overdraw
– Faster development iteration: quick recompile shaders
– But use forward rendering for transparency
• ‘Cubic’:
– Exploiting cubemaps: fit to GPUs
– Cache placement on the world: seamless
– Faster addressing: using planet normal = normalize(WorldPosition)
UNREAL SUMMIT 2016
UNREAL SUMMIT 2016
UNREAL SUMMIT 2016
Time of Day Lighting
• All day light
– Affect all day
• Time of day light
– Limited by time span
– 12 time spans: 0, 2, 4, …, 20, and 22 hour
• Twelve SH irradiance volumes
– Interpolate lighting from nearest 2 volumes
– No cache update
UNREAL SUMMIT 2016
Overview
• Offline
– Cache placement
– Photon emission
– Irradiance estimation
• Run-time
– Cubemap caching
– SH lighting
UNREAL SUMMIT 2016
Offline: Cache Placement
• Based on texels of the cubemap
– N x N x 6
– 50 cm space
– Finding Z by using ray casting
UNREAL SUMMIT 2016
Offline: Cache Placement
• 2.5D cache placement
– Above the surfaces
– No multi-layers or indoor in our game
– Incorrect results for flying characters
• Multiple layered cubemaps?
UNREAL SUMMIT 2016
Offline: Photon Emission
• UE4 Lightmass: a photon mapping based light builder
• Store ‘time of day light index’ for deposited photons
• No changes for remainders
class FIrradiancePhotonData{
FVector4 PositionAndDirectContribution;FVector4 SurfaceNormalAndIrradiance;
int32 TimeOfDayLightIndex;};
UNREAL SUMMIT 2016
Offline: Irradiance Estimation
• Twelve SH irradiance volumes
– Sharing world position, bent normal and sky occlusion
– Difference irradiance by time spans
• Per-time span irradiance
– Indirect photon final gathering
• All day lights: always
• Time of day lights: filtering by its index
• Global sky occlusion and bent normal
UNREAL SUMMIT 2016
UNREAL SUMMIT 2016
Run-time: Cubemap Caching
• Once at loading time
• CPU irradiance cache -> GPU cubemaps
– half4 texture cube array
• 2nd Order SH: encoded on 3 textures (RGB)
• 12 time spans * 3 = 36 elements
• Misc.:
– Bent normal and sky occlusion: 1
– Average color and directional shadowing: 12 – for reflection environment relighting
• Average color is computed by integrating incident radiance from all directions
UNREAL SUMMIT 2016
Run-time: SH Lighting
• Every frame
• Texture addressing
– Coordinates: planet normal
– Array index: time of day light index + 12 * {0|1|2}
• SH2 diffuse lighting
– Interpolate radiance from nearest 2 time spans
• Total 6 times fetches of a texture cube array
UNREAL SUMMIT 2016
Run-time: SH Lighting
UNREAL SUMMIT 2016
Performance
• Light building
– Scene-dependent
– Costly 12 times final gathering but faster than lightmaps
• Rendering
– Scene-independent
• approximately 0.4 ms: GTX970 1080p / ignoring transparency
• Stable visuals
– No seams or flickering
– Consistent looks for characters and environment
UNREAL SUMMIT 2016
Sky Light
UNREAL SUMMIT 2016
Transform to Spherical World
• Sky vector
– Not (0, 0, 1)
– Planet normal = normalize(WorldPosition)
• Rotate planet normal to (0, 0, 1) basis
– Heavy ALUfloat3 TransformVectorToLandsphereSpace(float3 InVector, float3 WorldPosition){
float3 PlanetNormal = normalize(WorldPosition);float3 SkyUp = float3(0, 0, 1);float3 RotationAxis = normalize(cross(PlanetNormal, SkyUp));float RotationAngle = acos(dot(SkyUp, PlanetNormal));float3x3 Rotator = RotationMatrixAxisAngle(RotationAxis, RotationAngle);return mul(Rotator, InVector);
}
UNREAL SUMMIT 2016
Time of Day Lighting
• Multiple sky cubemaps generated by artists
• Interpolated lighting like GI
UNREAL SUMMIT 2016
Bent Normal and Sky Occlusion
• As large scale ambient occlusion
– RGB: bent normal
– A: sky occlusion
– One R8G8B8A8 cubemap (no time variation)
• Diffuse lighting
– Widen and soft: sqrt
• Specular lighting
– Narrow and sharp: Square
Sky Occlusion + Bent Normal
UNREAL SUMMIT 2016
Screen Space Inner Shadows
• For GI and sky lighting
• Better looks when a character is on shadowed surfaces
• SSR styled ray marching
– Tracing on scene depth
– Directionality rather than SSAO
– No precomputation or asset building
– See my NDC 2016 presentation
UNREAL SUMMIT 2016
Reflection Environment
UNREAL SUMMIT 2016
Capture Probe Placement
• Uniform distribution on the sphere as the base
– Using golden spiral
• Additional placement by artists
UNREAL SUMMIT 2016
Time of Day Lighting
• Use relighting instead of pre-capturing all day
– Capture the scene without lighting
– Relight probes with tweak
• Relighting
– Geometric properties
• Relighting position = world position + (reflection vector * capture radius * specular occlusion)
• Relighting normal = normalize(relighting position)
– Direct lighting: Sun and SH2 diffuse sky lighting
– Indirect lighting: deferred cubic irradiance caching
• RGB: irradiance = average color of GI
• A: directional shadowing = occlusion of light sources
UNREAL SUMMIT 2016
UNREAL SUMMIT 2016
Volumetric Lighting
• Average color of GI (irradiance) is
also used for volumetric lighting
• For each ray marching step (for high spec.)
or at the surface (for low spec.)
UNREAL SUMMIT 2016
Summary
• Spherical world of A1
– Partially dynamic
– Time of day lighting changes
• Different approaches
– Deferred cubic irradiance caching
– Twelve time spans
– Relighting
UNREAL SUMMIT 2016
Future Work
• Improved GI
– 2nd Order SH -> 3th Order SH
– Layered cubemaps
– Volumetric fog
UNREAL SUMMIT 2016
Future Work
• Environment destruction and modification
– Multiple versions of irradiance volumes
• Pre-build for destroyed scenes
• Run-time update of cubemaps
• Like Quantum Break did
– Real-time GI
• SSGI?
– Recapture reflection environment
UNREAL SUMMIT 2016
Thank you
WE ARE HIRING!