iccv 2003 colour workshop 1 recovery of chromaticity image free from shadows via illumination...

ICCV 2003Colour Workshop

1

Recovery of Chromaticity Image Free from Shadows via

Illumination Invariance

Mark S. Drew1, Graham D. Finlayson2,

& Steven D. Hordley2

2School of Information Systems, University of East Anglia, UK

1School of Computing Science, Simon Fraser University, Canada


2

Overview

Introduction

Shadow Free Greyscale images

- Illuminant Invariance at a pixel -- 1D image

Shadow Free Chromaticity Images

- Better-behaved 2D-colour image invariant to lighting

Application- For shadow-edge-map aimed at re-integrating to obtain full colour, shadow-free image


3

The Aim: Shadow Removal

We would like to go from a colour image with shadows to the same colour image, but without the shadows.


4

Why Shadow Removal?For Computer Vision, Image Enhancement, Scene Re-lighting, etc.

- e.g., improved object tracking, segmentation etc.

Two successive video frames

Motion map, original colour space

Motion map, invariant colour space

snake


5

What is a shadow?

Region Lit by Sunlight and

Sky-light

Region Lit by Sky-light only

A shadow is a local change in illumination intensity and (often) illumination colour.


6

Removing Shadows

So, if we can factor out the illumination locally (at a pixel) it should follow that we remove the shadows.

Can we factor out illumination locally? That is, can we derive an illumination-invariant colour representation at a

single image pixel?

Yes, provided that our camera and illumination satisfy certain restrictions ….


7

Conditions for Illumination InvarianceAssumptions (but works anyway…!):

(1) If sensors can be represented as delta functions (they respond only at a single wavelength)

(2) and illumination is restricted to the Planckian locus

(3) then we can find a 1D coordinate, a function of image chromaticities, which is invariant to

illuminant colour and intensity

(4) this gives us a greyscale representation of our original image, but without the shadows

(so takes us a third of the way to the goal of this talk!)

(5) But the greyscale value in fact lives in a 2D log- chromaticity colour space, (so takes us a 2/3 of the way) [and exponentiating goes back to a

rank-3 colour].


8

Chromaticity: )/(},,{ BGRBGR

2D chromaticity is much more information than 1D greyscale:

Can we obtain a shadowless chromaticity image?

greycolour chromaticity


9

dQSEB

dQSEG

dQSER

)()()(

)()()(

)()()(

3

2

1

)(S

)(E )()( SE

Image Formation

Camera responses depend on 3 factors: light (E), surface (S),

and sensor (Q) is Lambertian shading


10

Q2()

Sen s

i tiv

i ty

Q1() Q3()

=

Delta functions “select” single wavelengths:

R R1 qQ

Using Delta-Function Sensitivities

RRRRR SEqdESq

GGqQ 2

BBqQ 3

RRR SEqR

GGG SEqG

BBB SEqB


11

Characterizing Typical Illuminants

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

r/(r+g+b)

g/(

r+g

+b

)

Illuminant Chromaticities Most typical illuminants lie on, or close to, the Planckian locus (the red line in the figure)

So, let’s represent illuminants by their

equivalent Planckian black-body

illuminants ...


12

1

51 1)(

2

T

c

ecIE Here I controls the overall intensity of light, T is the

temperature, and c1, c2 are constants

Planckian Black-body Radiators

For typical illuminants, c2>>T.

So, Wien’sapproximation:

T

c

ecIE 2

51)(


13

How good is this approximation?

2500 Kelvin

10000 Kelvin

5500 Kelvin


14

For delta-function sensors and Planckian illumination we have:

Back to the image formation equation

T

c

kkkkkecIqSR 2

51)(

Surface Light


15

Band-ratio chromaticity

G

R

B

Plane G=1

Perspective projection onto G=1

,2..1,/ kRR pkk

Let us define a set of 2D band-ratio chromaticities:

p is one of the channels, (Green, say)


16

Let’s take log’s:

Band-ratios remove shading and intensity

Teess pkpkkk /)()/log()log('

with ,)(51 kkkk qScs kk ce /2

Gives a straight line:

)(

)())/log(()/log(

1

21

'12

'2

p

ppp ee

eessss

Shading and intensity are gone.


17

Calibration: find invariant direction

Log-ratio chromaticities for 6 surfaces under 14 different Planckian

illuminants, HP912 camera

Macbeth ColorChecker:

24 patches


18

Deriving the Illuminant Invariant

This axis is invariant to shading + illuminant

intensity/colour


19

Algorithm:

,2..1,/ kRR pkk

)log('kk

Plot, and subtract mean for each colour patch:

SVD (2nd eigenvector) gives invariant direction.


20

Algorithm, cont’d:

eI k''

Form greyscale I’ in log-space:

)'exp(II exponentiate:


21

Obtaining invariant Chromaticity image (1):

We observe: line in 2D chromaticity space is still 2D, if we use projector,

rather than rotation:

,||||

)(

e

eeP

T

e

''~kek P 2-vector

eI '~'


22


However, we have removed all lighting! put back offset in e-direction equal

to regression on top 1% brightness pixels:


23


offset in e-direction:

)'~exp(~,'~'~'~ extralight

We are most familiar with L1-chromaticity)(/},,{ BGRBGR

)1(/},1,{ 2121


24


In terms of L1-chromaticity:

orig. recovered


25


Projection line becomes a rank ~3

curve in L1 chromaticity space


26


We can do better on fitting recovered chromaticity to original — regress on

brightest quartile:


27

Improves chromaticity:

orig. recovered

regressed


28

Some Examplescolour chromaticity recovered


29

Main Advantage: chromaticity invariant (in [0,1]) is better-behaved than greyscale invariant –– betterfor shadow-free re-integration (ECCV02)


30

Acknowledgements

The authors would like to thank the Natural Sciences and Engineering Research Council of Canada, and Hewlett-Packard Incorporated for

their support of this work.

iccv 2003 colour workshop 1 recovery of chromaticity image free from shadows via illumination...

Documents

colour workshop

illumination colour

illuminant colour

dcolour image invariant

grey colour chromaticity

shadowfree image slide

original image

shadowless chromaticity