a dynamic noise primitive for coherent stylization, egsr 2010

A DYNAMIC NOISE PRIMITIVE

FOR COHERENT STYLIZATION

Pierre BénardJoëlle ThollotGrenoble Universities / INRIA Rhône-Alpes

Ares LagaeKatholieke Universiteit LeuvenREVES - INRIA Sophia-Antipolis

Peter VangorpGeorge DrettakisREVES - INRIA Sophia-Antipolis

Sylvain LefebvreALICE - INRIA Nancy / Loria

Stylization of 3D Animations• 3D scene 2D appearance

2

Stylization of 3D Animations• 3D scene 2D appearance

• Stylized color regions– 2D medium: a pattern– Temporal coherence

3

Paper

Pencil strokes

Watercolor pigments

Paint strokes

Hand –made animation« Il pleut bergère », Jérém

y Depuydt (2005)

4

Popping Temporal continuity

Naïve CG solutions

5

Shower-door effect Coherent Motion

Traditional mapping Flatness

Temporal Coherence Problem• Extreme cases Requirements

6

Coherent motion Temporal continuity

Flatness

Traditional mapping

Shower-doorPopping

Contradictory requirements:solution find a compromise

3 goals to ensure at best

• Additional challenges– Flexibility variety of styles– Interactivity artistic control– Evaluation quality of the trade-off

Coherent motion Temporal continuity

Flatness

7

PREVIOUS WORK

Texture -Based methods• Object-space Blending artifacts Perspective distortion

Coherent motion

Temporal continuity

Flatness

Traditional mapping

Shower-doorPopping

Object-space texture mapping[KLK*00,PHWF01,FMS01,BBT09]

[BBT09]

9

Texture -Based methods• Object-space• Screen-space Sliding

Coherent motion

Temporal continuity

Flatness

Popping

[CTP*03]

Screen-space texture mapping[CTP*03,CDH06,BSM*07,BNTS07]

Object-space texture mapping[KLK*00,PHWF01,FMS01,BBT09] 10

Shower-door

Few -Primitive methods Clutter / holes Popping

11

Coherent motion

Temporal continuity

Flatness

Popping

[Mei96]

Few-primitive methods [Mei96,Dan99,HE04,VBTS07]

Object-space texture mapping[KLK*00,PHWF01,FMS01,BBT09]

Screen-space texture mapping[CTP*03,CDH06,BSM*07,BNTS07]

or

Few -Primitive methods

12

Vanderhaeghe et al. EGSR 2007

Key Insight• Blending a large number of primitives

– Reduce popping artifacts– Individual primitives merge texture

13

Many-Primitive methods

Coherent motion

Temporal continuity

Flatness

[KC05]

Many-primitive methods[KC05,BKTS06]

14

Object-space texture mapping[KLK*00,PHWF01,FMS01,BBT09]

Few-primitive methods [Mei96,Dan99,HE04,VBTS07]

Screen-space texture mapping[CTP*03,CDH06,BSM*07,BNTS07]

NPR GABOR NOISE

Procedural noises• Sparse convolution [Lewis 84,89]• Spot Noise [van Wijk 91]• Gabor Noise [LLDD09]

Our trade-off: NPR Gabor Noise

16

Gabor Noise [LLDD09]• Offers significant spectral control• Support anisotropy• Is fast to evaluate

17

See “State of the Art in Procedural Noise Functions”, EG 2010 for comparisons with previous work

Gabor Noise [LLDD09]• Definition

Sum of randomly positioned and weighted kernels

18

Gabor kernel noiserandom positionsand weights

NPR Gabor Noise• Basic principles follow from the goals

– Flatness Noise parameters in 2D screen space Evaluation in 2D screen space

19

2D Gabor noise [LLDD09]

NPR Gabor Noise• Basic principles follow from the goals

– Flatness– Coherent motion

Point distribution on the surface of the 3D model

20

Surface Gabor noise [LLDD09]

2D Gabor noise [LLDD09]

NPR Gabor Noise• Basic principles follow from the goals

– Flatness– Coherent motion

21

Surface Gabor noise [LLDD09]

NPR Gabor noise2D Gabor noise [LLDD09]

NPR Gabor Noise• Basic principles follow from the goals

– Flatness– Coherent motion– Continuity

Smooth LOD mechanism

22

Surface Gabor noise [LLDD09]

NPR Gabor noise2D Gabor noise [LLDD09]

GPU Splatting Algorithm• Sample 3D triangles

– 2D Poisson distribution with constant screen space density

– PRNG: seed = triangle ID

23

Farsmall screen

arealess points

Closelarge screen

areamore points

GPU Splatting Algorithm• Generate 2D point sprites

24

Point distribution Texture sprites

LOD Mechanism• Blending scheme using statistical properties

– Reduce popping– Preserve noise appearance

25

STYLES

Style Design• Standard techniques from procedural texturing

and modeling [EMPPW02]– Threshold

• Smooth step function • X-toon textures [BTM06]

– Compositing (alpha-blending, overlay)– Local control

• Curvature noise orientation• Shading noise frequency

• Interactive feedback

Threshold texture

27

Style Design

28

Results :

29

isotropic as well asanisotropic patterns

30

Results :local variationaccording to shading

Results :

31

local orientation guided by surface curvature

USER STUDY

User Study: Motivation• Evaluate success of various solutions

according to

• Relative importance of these criteria

33

Coherent motion Temporal continuity

Flatness

User Study: Setup• Methodology

– 15 naïve subjects, ~ 20-30 minutes• Ranking tasks

“Rank the images/videos according to … ”

34

User Study: Compared methods

35

Adv D2D DST

ours SD TM

Extreme cases

Local screen-spaceGlobal screen-space Object-space

User Study: Flatness• Simple stimuli

Adv D2D DST

ours SD TM

36

Object-space

User Study: Flatness• Complex stimuli

Adv D2D DST

ours SD TM

37

User Study: Flatness“Rank the images according to

how flat they appear.”• Simple stimuli

– Image-space methods more flat– Object-space methods less flat

• Complex stimuli– High variance confusing question– Many 3D cues flatness not perceived

38

• Simple stimuli

User Study: Dynamic stimuli

39

• Complex stimuli

User Study: Dynamic stimuli

40

User Study: Coherent motion“Rank the videos according to how coherently

the pattern moves with the object.”• Simple stimuli

– Object-space methods more coherent– Shower-door least coherent– Image-space methods provide a tradeoff

• Complex stimuli– Same conclusions– Our approach slightly better than other

image-space methods41

“Rank the videos according to how much the pattern changes over time.”

• Simple stimuli– High variance– Advection and ours produce more changes

“swimming” artifacts

• Complex stimuli– Shower door and D2D least changes– Others perceived equally

User Study: Temporal continuity

42

User Study: Pleasantness“Rank the videos according to how

pleasant you find them in the context of cartoon animation.”

• Complex stimuli– Object-space approaches more pleasant– Our methods first image-space approaches

43

User Study: Pleasantness• Strong correlation with “motion coherence”

most important criteria to preserve

• NPR Gabor noise performs well

44

CONCLUSIONSAND FUTURE WORK

45

User Study• First step toward formal evaluation

– Flatness hard to see in complex scenes– Motion coherence predominant criteria

• Intrinsic limitations– Hatching other styles– Naïve users professional artists

• Objective metric– Statistical texture measures [BTS09]– Optical flow analysis

46

NPR Gabor Noise

• New primitive for coherent stylization

• Interactive scheme: remaining popping “Procedural” evaluation [LLDD09]

– Slower, but should avoid popping– Useful for high quality offline rendering

• Temporally coherent spot noise Additional patterns

47

Thanks !

Styles cookbook and experiment stimuli:

http://artis.inrialpes.fr/Publications/2010/BLVLDT10

Acknowledgments• Laurence Boissieux, Kartic Subr, Adrien Bousseau and Marcio Cabral• The participants of the study• Research Foundation-Flanders (FWO), CREA (K.U.Leuven)

User Study: Flatness• “Rank the images according to how flat

they appear.”– Simple stimuli

– Complex stimuli

50

less flat more flat

DST TM Adv ours SD D2D

less flat more flat

TM SD DST ours D2D Adv

50

User Study: Coherent motion• “Rank the videos according to how coherently

the pattern moves with the object.”– Simple stimuli

– Complex stimuliless coherent more coherent

SD D2D ours Adv DST TM

SD D2D Adv ours TM DST

SD Adv D2D ours TM DST

SD Adv ours D2D TM DST

translate:

rotate:

zoom:

less coherent more coherent

51

Object-space

• “Rank the videos according to how much the pattern changes over time.”– Simple stimuli

– Complex stimuli

User Study: Temporal continuity

52

Adv ours D2D DST TM SD

TM DST ours Adv D2D SD

Adv SD TM DST ours D2D

Adv ours D2D DST SD TM

more change less change

52

translate:

rotate:

zoom:

more change less change

User Study: Pleasantness• “Rank the videos according to how

pleasant you find them in the context of cartoon animation.”– Complex stimuli

80

SD D2D Adv ours DST TM

less pleasant more pleasant

53

Object-space