naïve compositing

1

2

naïve compositing

3

Poisson compositing

4

our result

5

Objectives

• Robust to inaccurate selection

• Output Quality

- Limit color bleeding

• Time-Performance

- Efficient method

Seamless compositing

6

Poisson Compositing Result

Pros:• Blends seamlessly• Linear computationCons:• Bleeding visible• Foreground to background bleeding

Related Work: Poisson CompositingPasting gradients instead of pixelsPérez 03, Georgiev 06, …

7

Pros:• Reduced bleedingCons:• Nonlinear

- Computationally intensive

L1 Norm Result

Related Work: L1 NormIntroduced in shape from shadingReddy 09

8

Move the boundaries to avoid bleeding

Related Work: Moving Boundaries

Pros:• Avoids bleedingCons:• We don’t want boundaries to change• Changed composition

Changing boundaries

Jia 06

9

• Conceal bleeding in textured areas

• Better gradient field at boundary

• Efficient linear scheme

Contributions

10

• Problem Description

• Hiding Residuals with Visual Masking

• Generating a low-curl boundary

• Results and Comparisons

Overview

Algorithm Overview

Selection

Background

Foreground

Algorithm Overview

Selection

Background

Foreground

grad

grad

Algorithm Overview

Selection

Background

Foreground

Targetgradient

fieldpastegrad

grad

Algorithm Overview

Selection

Background

Foreground

Result 14

Targetgradient

fieldpaste

integrate

grad

grad

15

We seek image I with gradients close to target v.

target gradient field

=

Standard Approach: Poisson compositing

16

We seek image I with gradients close to target v.

target gradient field

output gradient field

=

Least-squares:

Standard Approach: Poisson compositing

17

target gradient field

=

Our Approach

value that minimizes curl

output gradient field

Weighted least-square:

We seek image I with gradients close to target v.

weight

18

• Problem Statement

• Hiding Residuals with Visual Masking

• Generating a low-curl boundary

• Results and Comparisons

Overview

19

Our Strategy

• Use the weights to locate integration residuals where they are less visible

• Exploit perceptual effect: visual masking– human perception affected by texture.

20

Visual Masking Demo

Can you spot all the dots?

21

Visual Masking Demo

Texture hides low-frequency content

22

Visual Masking Demo

Can you spot all the dots?

23

Design of the Weights

smooth region:needs high weightto prevent bleeding

textured regions:low weight is okbecause bleedingless visible

weights (white is high)

24

Estimating the Amount of Texture

RGB gradient field

25

Estimating the Amount of Texture

RGB gradient field Noise controlling function

Small Gaussian convolution

Large Gaussian convolution

26

Estimating the Amount of Texture

small Gaussian large Gaussian

gradient field g

27

Estimating the Amount of Texture

Texture Map Output (white indicates more texture)

28

• v depends only on foreground and background– does not depend on the unknown I– weights are constant in the optimization

• our energy is a classical least-squares optimization

Weight Formula

= 1 –

Weight output

29

Hiding Residuals with Visual Masking

Poisson compositing With texture weights

Bleeding only in textured areas

30

Hiding Residuals with Visual Masking

Poisson compositing With texture weights

Residuals

31

Hiding Residuals with Visual Masking

Poisson compositing With texture weights

Reduced bleeding but not fully

32

• Problem Statement

• Hiding Residuals with Visual Masking

• Generating a low-curl boundary

• Results and Comparisons

Overview

33

• Target field v integrable iff curl(v)=0• Only boundary has non-zero curl– Consequence of

target field construction(see paper)

• Our strategy: reducing the curl

curl usingstandard strategy

Boundary Problems

34

• Minimize

• Unfortunately, the result is not seamless:

output of naïve approach close-up

Naïve Approach

35

• Unknowns: target field v along boundary• Weights depend on texture (detail in paper)

zero curl seamless compositingweight

Our Approach: Least Squares Trade-off

36

• Problem Statement

• Hiding Residuals with Visual Masking

• Generating a low-curl boundary

• Results and Comparisons

Overview

37

Results and Comparisons

Copy-and-paste

38

Results and Comparisons

Poisson Reconstruction : Pérez 03, Georgiev 06, …

39

Results and Comparisons

Maximum Gradient : Pérez 03

40

Results and Comparisons

Diffusion : Agrawal 06

41

Results and Comparisons

Lalonde 07

42

Results and Comparisons

L1 Norm : Reddy 09

43

Results and Comparisons

Our Result

44

Results and Comparisons

Poisson

Max

Diffusion

Lalonde

L_1 Norm

Our Result

0 2 4 6 8 10 12 14 16 18 20

Running Time (Seconds)

45

Results and Comparisons Poisson

46

Results and Comparisons Our Result

47

Results and Comparisons Poisson

48

Results and Comparisons Our Result

49

• Several parameters

- all examples use the same parameters

• Discoloration may happen- happens in all gradient based operators

• Our model of visual masking is simple- good for performance

- more complex model could be used

Discussion

50

• Robust compositing using visual masking

• Better gradient field at boundary

• Efficient linear scheme

Conclusion

Poisson Reconstruction Our Result

51

Special Thanks

Todor GeorgievKavita BalaGeorge DrettakisAseem AgarwalaBill FreemanTim ChoBiliana KanevaMedhat H. IbrahimRavi Ramamoorthi

This material is based upon work supported by the National Science Foundation under Grant No. 0739255 and No. 0924968.

52

• Robust compositing using visual masking

• Better gradient field at boundary

• Efficient linear scheme

Conclusion

Poisson Reconstruction Our Result