ICT Junior Camp 2007 @ Mahidol University

1

University

Image Matting and Composition

Dr. Rawesak Tanawongsuwan

ccrtw@mahidol.ac.th

Outline

• Image Composition

• Matting Process

• Blue Screen Matting

• Environment/Shadow Matting

• Recent Matting Techniques

2

• Recent Matting Techniques

(Knockout/Bayesian/Grabcut/Poisson)

• Interactive Matting Techniques (Robust

matting/Easy matting/Closed-form matting)

• Flash Matting

• Video Matting (Video cutout/Video

defocus/Camera Array matting)

Digital Image Composition

3

Digital Image Composition

4

Matte

How does Superman fly?

5

Pulling a matte

Problem Definition:

• The separation of an image into• A foreground object image

• A background image

6

• An alpha matte

Hard problem

Matte creation

• From digitized images

– Blue-screen matting

– Video or chroma-keying

– Image processing

7

– Image processing

Blue Screen

8

Nothing special about blue

9

Digital Matting Equation

• For each pixel

C = αF + (1- α)B

10

• C: Rendered Color

• F: Foreground color

• B: Background color

� α: Amount of light passing through the

foreground – called the Alpha-Channel

Challenges to Matting &

Compositing

• Extracting the Matte (alpha

• channel) from natural scenes

• Handling the occluding contour?

11

• Handling the occluding contour?

• Foreground has color of background

• Refracted and reflected light

• …

What about semi-transparent

mattes?

12

We want a semi-transparent

matte

13

Solution…

14

More example…

15

Problems with Matting

• Images do not look realistic

• Lack of Refracted Light

• Lack of Reflected Light

16

Environment Matting and

Composition

17

Process of environment

matting

• Learn how light is reflected and refracted from

a backdrop and from various sidedrops

18

Process of environment

matting

19

Results of environment

matting

20

Results of environment

matting

21

Movies of environment

matting

22

Shadow matting and

compositing

23

Shadow matting and compositing

24

Shadow matting

• Show clip ShadowMattingDivXS.avi

25

Natural image matting

• Beyasian approach

26

Natural image matting

27

Level of user interaction

28

Trimap

GrabCut

Robust matting tool

• Demo

29

Interactive matting

• Easy matting

30

Interactive matting

• Closed form matting

31

Interactive matting

• Soft Scissors

32

Flash matting

• Extract mattes using a pair of flash/no-flash images

33

Image Matting is Difficult

34

Image Matting is Easy with

FLASH

35

Flash Matting

36

Flash matting

• Show clip FlashMatting.wmv

37

Video

• Let’s understand the nature of video clip

• A sequence of image frames

38

• Show videoCubism.avi

Video matting

• Easier or more difficult?

• Issues…

39

• Issues…

• Show VideoCutout.avi

Special hardware/cameras

• Need more than a single camera

• Defocus video matting

40

• Defocus video matting

• Multiple cameras matting

Defocus video : Capture Hardware

41

Capture Hardware

42

Preview: Input

43P

I

Everything in Focus

FI

Actor in Focus

BI

Background in Focus

Defocus video

• Show clip DefocusMatting.avi

44

Multi-cameras system

45

Using a Camera Array

46

500

2500

4500

47

48

Multi-camera system

• Automatic matte extraction

• Toward real-time performance

• Show clip CameraArrayMatting.mp4

49

• Show clip CameraArrayMatting.mp4