Visual Effect Graph

Julien FRYER @JulienFryer Graphics Engineer

Thomas ICHÉ @peeweekVFX Visual Effect Artist

Unite Seoul 2019

Visual Effect Graph

3

— Concepts and Philosophy

— Technical Overview

— Spaceship Demo : Pin Screen VFX Breakdown

— Roadmap

4

Video

Replace with image

Visual Effect Graph

5

Next Generation Visual Effects tool for Unity

Tailored for Next-Generation platforms (GPU/Compute)

Self-contained asset templates with Event and Parameter interface

Programmable, hybrid, Stack/Node-Based Effect behaviors

Replace with image

Goals and Philosophy

6

Industry trend : Programmable effect behaviors

Scalability of FX: from small to very large, from simple to complex

Artist friendly / Data Driven

Scalability of features: compiled vs composed

7

CPU GPU

PARTICLE COUNT Thousands Millions

SIMULATION Simple Complex

PHYSICS Underlying Physics System• Primitives• Scene Representation (SDF…)• Depth Buffer

GAMEPLAY READBACK Yes No** Potentially small data with latency

OTHER Can read Frame Buffers-

CPU vs GPU Particles

Concept and Philosophy

9

0

Asset Instance Configuration● Use Template in Scenes● Edit Parameters● Call Events● C# Scripting

Levels of Editing

10

Graph Asset Authoring● Author Effect Behaviors● Define Events● Setup Parameter Interface

1

Levels of Editing

11

2

Feature Creation/Editing● Author New Features● Improve Library

Levels of Editing

Logic Flow● Chained States (Contexts)● Contexts made of Blocks● Blocks can be parametrized

Graph Flow

Logic Flow● Chained States (Contexts)● Contexts made of Blocks● Blocks can be parametrized

SpawnHow many particles?

InitializeWhat are their initial values?

UpdateHow do they behave over time?

OutputHow are they rendered?

Node Blocks● Reorderable Stack● Define Context Behaviour● Alter Simulation Attributes● Expose Slots that can be fed by

operators

Blocks

Logic Flow● Chained States (Contexts)● Contexts made of Blocks● Blocks can be parametrized

Operator Flow● Chained Operators● Access to Simulation Attributes● Can Scale CPU/GPU (per-element)

Attributes

16

— Define Particle State

— Stored only if needed by processing

— Standard Attributes (position, color, …)

— Custom Named Attributes

Generic Read/Write System from various Modes (Constant/Random/Curve/Texture)

Parameter Interface● Defined in Blackboard● Exposed at Component Level● Access to Parameters in Graph● Edit Gizmos in Scene

Parameter Interface● Defined in Blackboard● Exposed at Component Level● Access to Parameters in Graph● Edit Gizmos in Scene

Parameter Interface● Defined in Blackboard● Exposed at Component Level● Access to Parameters in Graph● Edit Gizmos in Scene

Parameter Interface● Defined in Blackboard● Exposed at Component Level● Access to Parameters in Graph● Edit Gizmos in Scene

Events● Default OnPlay / OnStop● Custom Named Events● Event Attribute Payload● C# API● Timeline Integration

Events

22

Technical Overview

VFX Compilation

23

High level graph model

< Expression Graph

— Data layout (buffer description)— Shaders (Compute for simulation, VS/PS for rendering)— Bytecode for our CPU interpreter— PropertySheet (inputs and exposed parameters)— VFXSystems description

Target < 500 ms for compilation

Expression Graph

24

Graph from the pink Unity logo sample

Expression Graph

25

particle attributes reads

Expression Graph

26

Evaluated on GPU: Generates shader code

Expression Graph

27

Evaluated on CPU: Generates interpreter bytecode

Expression Graph

28

FX time is not known at compile time

Expression Graph

29

Exposed parameter can be overriden per instance or via script

Expression Graph

30

Unknown at compile time: Evaluated at runtime

Statically known at compile time: Constant folded

Expression Graph - Low level

31

— Intermediate language

— Abstract representation allows several backends (CPU particles but not yet…)

Expression Graph - Low level

32

— 2 versions of runtime data

– Edition - All inputs are accessible

– Runtime - Constant folded graph and optimized effect

Rendering

33

— Built-in primitives

– Points, lines, quads, octagon, hexahedron, meshes...

— Built-in shaders

– HDRP Lit, Unlit, Decals (Hexahedron), ray-traced spheres (Quads)...

– Features: soft particles, flipbook blending…

— We need shader graph integration with VFX graph (2019.3)

34

Point Line Quad Cube Mesh

Decal Sphere Lit Quad Lit Cube Lit Mesh

Rendering - output types

Sorting

35

— Rendering order hell with transparent objects

— We didn’t sort that out… (no OIT or global per particle sorting)

— …instead we gave controls at different granularity (per component / per system / per particle)

— GPU Sorting for per particle sorting

Input Data

36

— Textures and buffers on GPU

— Generated constant buffers for expressions values

— Curve and gradient baked in a generated texture

— Point caches: attributes packed in textures

— Volumetric data as texture3D + oriented box– Vector field: forces

– SDF: collision and conformance

SDF - Conformance

37

SDF - Collision

38

VFX Toolbox

39

— A suite of tools to generate data– Houdini exporters for SDF, Vector Fields, point cache– Point cache baker from mesh– Image sequencer for flipbooks

40

Breakdown

Looking into Spaceship Demo

41

Spaceship Demo

42

Spaceship Demo

43

Let’s Break it Down( Hey ! We even made a public sample for this ! )

Control Room : Scene Setup

44

— Scene Setup

– Holographic Table (2 Lights + 1 VFX)

– Simple Animation

– Pin Screen Effect : 1 Visual Effect

— PinScreen VFX

– Only 1 System

– Multiple Outputs : Pins, Caps and Rays

– Custom Update

45

Visual Effect Overview

PinScreen VFX : In a nutshell (1/2)

46

— Spawn : Simple Burst : count = 65536

— Initialize:

– position From pCache Attribute Map + backup in BasePosition

– Everlasting particles (no lifetime Attribute)

— Update:

– Simulate Heightfield animation in position Attribute

– Sum of position offsets : HeightField, Wave, Glitch

– Blend position : smooth animation

– No Aging / Reaping / Velocity Integration

PinScreen VFX : In a nutshell (2/2)

47

— For each Pin : Multiple Outputs

– Cap : Lit quad / At position / Facing World Up

– Pin : Lit quad / stretched between position and BasePosition

— Light Rays :

– Start at position / Stretches towards upper source

– Scale based on pin height or cull if pin is too small

48

Breaking Down MechanicsOne feature at a time

Spawn On Point Cache

49

— pCache File : 65536 items

– Baked in Houdini

– Relax Iterations

– Custom Sorting

— Initialize:

– Read Position in Cache based on particleID

– Back-up initial position in BasePosition (Custom Attribute)

Heightfield Push

50

— Compute 2D Sample Position

– Based on BasePosition.xy (World .zx)

– Apply Scale + Bias based on World-Space AABoxControl parameter

— Sample HeightField:

– Use Texture Sample Operator (SampleLOD)

— Scale Pin Heights:

– Use Reference Height to scale input (read from Texture) value.

– Scale Height based on Distance to Table Center : Curve Mapped

Scan Wave

51

— Inner > Outer Animation

– 1D Coordinate : Distance to center + Time Cycle

– 1D Coordinate Samples Curve to get Height

— Adding Noise:

– Modify 1D Coordinate by adding noise offset.

– 3D Noise :

– XZ : 2D Coordinates

– Y : Time

Glitches

52

— 3D Offset on Pins

– 2D Polar Coordinate

– Samples 2D Texture : use XYZ value as offset for the pin tip.

– Unsigned normalized (centered on 0.5)

— Coordinate Computation

– Add 2D Noise to XZ Coordinate (Offset)

– 2D Rectangular to Radial conversion

53

Scene Control / Sequencing

PinScreen VFX : Scene Control

54

— Exposed Parameters :

– AABoxControl : Axis Aligned Box

– _center : Bound by rig GameObject using ParamterBinder

– _size : Used By Animation

— Lighting :

– Fixed : One Top Spotlight + One Cyan PointLight

– Animated Pointlight (Red, Toggled in Animation)

— Animation/Sequencing :

– Simple Animation : Controls 1 Vector Parameter + 1 GameObject (Parameter Binder)

55

Rendering Pin Screen

PinScreen VFX : Rendering Pins

56

— All Opaque :

– 2 DrawCalls : 1 Quad (Tip) + 1 Quad (Body) : 8vx/pin

– Normals : Fake volume / Cast Shadows + Contact Shadows

– Transmission + Fake Occlusion gradient in BC

— Light Rays :

– Potential Frame rate Killer : Additive Blending

– Only one ray out of 15 pins

– Camera Fade for near particles (Fade + Cull)

57

Roadmap

Roadmap

58

— 2019.2

– Improved HDRP integration

– Subgraphs

– Static mesh sampling

– Motion vectors

– LWRP lit support (compute only)

– Ribbon/trails

Roadmap

59

— 2019.3 - Target Out of Preview

– Shader graph integration

– Skinned mesh sampling

– Profiling/debugging tools

We need your feedback!

60

— Join the discussions on the forums

– https://forum.unity.com/threads/feedback-wanted-visual-effect-graph.572110/

— Follow the devs on Twitter:

– @JulienFryer

– @PaulDemeulenaer

– @peeweekVFX

– @iTris666

– @VladimirNeykov

— #vfx channel on Unity Community Discord

Thank You!Time for some Q&A?

Download this sample and 9 others for 2019.1 at :https://github.com/Unity-Technologies/VisualEffectGraph-Samples