Shades of Gray and Colour Constancy

IS&T/SID Twelfth Color Imaging Conference pp. 37-41, 2004

Graham D. Finlayson and Elisabetta Trezzi

Presented by Jung-Min Sung

School of Electrical Engineering and Computer Science Kyungpook National Univ.

Abstract Proposed method

– Max-RGB & Gray-World • Instantiations of Minkowski norm

– Optimal illuminant estimate • 𝐿6 norm: Working best overall

2/19

Introduction Categories of color constancy

– Representing an image by illuminant invariant descriptors – Color constancy methods

• Physical-based algorithm • Statistic-based algorithm

− Max-RGB, Gray-World, Gray-Edge • Gamut constrained algorithm • Probability-based algorithm

− Markov Random Field, Conditional Random Field • Learning-based algorithm

3/19

Problem of Max-RGB & Gray-World – Two extremes in the Minkoswki family norm

• Mean(𝐿1) and Maximum(𝐿∞) – Assuming the optimal illuminant estimate is between 𝐿1 and 𝐿∞

4/19

Modeling a color signal – Assuming illuminance 𝐸 𝜆 is uniform over a scene – A Lambertian surface illuminated by a spectral distribution

𝐶 𝜆 = 𝐸 𝜆 𝑆 𝜆

where 𝐸 𝜆 : Spectral distribution 𝑆 𝜆 : Lambertian surface 𝐶 𝜆 : Color signal

Background

(1)

5/19

– Intensity on three sensors (𝑅,𝐺,𝐵)

𝑅 = � 𝐸 𝜆 𝑆 𝜆 𝑅 𝜆 𝑑𝜆𝜔

𝐺 = � 𝐸 𝜆 𝑆 𝜆 𝐺 𝜆 𝑑𝜆𝜔

𝐵 = � 𝐸 𝜆 𝑆 𝜆 𝐵 𝜆 𝑑𝜆𝜔

– An image represented by three N-dimensional vector

• Given image 𝐼

𝑅 = [𝑅1,𝑅2,⋯ ,𝑅𝑁]𝑇 𝐺 = [𝐺1,𝐺2,⋯ ,𝐺𝑁]𝑇 𝐵 = [𝐵1,𝐵2,⋯ ,𝐵𝑁]𝑇

Sensor response curve Or

Sensitivity function

𝐼:image

𝑆 𝜆

𝐸 𝜆 𝑅 𝜆 𝐺 𝜆 𝐵 𝜆

6/19

– One pixel intensity over the image

𝑅𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝑅 𝜆 𝑑𝜆𝜔

𝐺𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝐺 𝜆 𝑑𝜆𝜔

𝐵𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝐵 𝜆 𝑑𝜆𝜔

𝐼:image

Position: 𝑖

(3)

7/19

Conventional algorithms – Max-RGB

• Assuming that at least a white patch exist in an image

max𝑖∈ 1,2,⋯,𝑁

𝑅𝑖 = � 𝐸 𝜆 𝑅 𝜆 𝑑𝜆𝜔

= 𝑅𝑒

max𝑖∈ 1,2,⋯,𝑁

𝐺𝑖 = � 𝐸 𝜆 𝐺 𝜆 𝑑𝜆𝜔

= 𝐺𝑒

max𝑖∈ 1,2,⋯,𝑁

𝐵𝑖 = � 𝐸 𝜆 𝐵 𝜆 𝑑𝜆𝜔

= 𝐵𝑒

𝐼:image

1

(7)

8/19

– Gray-world • Assuming that a scene average is grey

𝜇 𝑆 𝜆 = �𝑆𝑖 𝜆𝑁

𝑁

𝑖=1

= 𝑘

𝜇 𝑅 = � 𝐸 𝜆 �𝑆𝑖 𝜆𝑁

𝑁

𝑖=1

𝑅 𝜆 𝑑𝜆𝜔

= 𝑘𝑅𝑒

𝜇 𝐺 = � 𝐸 𝜆 �𝑆𝑖 𝜆𝑁

𝑁

𝑖=1

𝐺 𝜆 𝑑𝜆𝜔

= 𝑘𝐺𝑒

𝜇 𝐵 = � 𝐸 𝜆 �𝑆𝑖 𝜆𝑁

𝑁

𝑖=1

𝐵 𝜆 𝑑𝜆𝜔

= 𝑘𝐵𝑒

(6)

𝐼:image

𝑆𝑖 𝜆

9/19

Minkowski family norm Minkowski norm

– Definition of p-norm for 𝑋 = 𝑋1,𝑋2,⋯ ,𝑋𝑁 𝑇

𝑋 𝑝 = � 𝑋𝑖 𝑝𝑁

𝑖=1

1/𝑝

– Example of 2 norm

• Equal to Euclidean distance

𝑋 2 = � 𝑋𝑖 2𝑁

𝑖=1

1/2

= 𝑋12 + 𝑋22 + ⋯+ 𝑋𝑁2

(8)

10/19

Mean of p-norm – Mean of p-norm for 𝑋 = 𝑋1,𝑋2,⋯ ,𝑋𝑁 𝑇

𝜇𝑝 𝑋 =𝑋 𝑝

𝑁1/𝑝 =𝑋1𝑝 + 𝑋2

𝑝 + ⋯+ 𝑋𝑁𝑝

𝑁

𝑝

– Property of Minkowski norm

• Triangular inequality: Equation (8) in this paper • Monotonically increasing sequence

𝑋 𝑝

𝑁1/𝑝 ≤𝑋 𝑞

𝑁1𝑞

, 𝑖𝑖 𝑝 ≤ 𝑞

• Infinity norm

𝑋 ∞ = max

0≤𝑖≤𝑁𝑋𝑖

(11)

11/19

Proposed method Expression of Max-RGB & Gray-world with Minkowski norm

– Max-RGB

𝑅𝑒𝐺𝑒𝐵𝑒

=𝜇∞ 𝑅𝜇∞ 𝐺𝜇∞ 𝐵

– Gray-World

𝑅𝑒𝐺𝑒𝐵𝑒

=𝜇1 𝑅𝜇1 𝐺𝜇1 𝐵

12/19

– Order relationship between Max-RGB and Gray-World

𝜇1 𝑅 ≤ 𝜇2 𝑅 ≤ ⋯ ≤ 𝜇∞ 𝑅

𝜇1 𝐺 ≤ 𝜇2 𝐺 ≤ ⋯ ≤ 𝜇∞ 𝐺

𝜇1 𝐵 ≤ 𝜇2 𝐵 ≤ ⋯ ≤ 𝜇∞ 𝐵

– Proposed method: (Shade of grey algorithm) • Assuming that the average of pixels raised to the power of p is gray

𝑅𝑝,𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝑅 𝜆 𝑑𝜆𝜔

𝑝

= � 𝐸 𝜆 𝑝 𝑆𝑖 𝜆 𝑝𝑅 𝜆 𝑑𝜆𝜔

= � 𝐸𝑝 𝜆 𝜎𝑖 𝜆 𝑅 𝜆 𝑑𝜆𝜔

= 𝑅𝑖𝑝

Max-RGB Gray-World

(15)

13/19

– Extension of this formula to R,G,B

𝑅𝑝,𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝑅 𝜆 𝑑𝜆𝜔

𝑝

= � 𝐸𝑝 𝜆 𝜎𝑖 𝜆 𝑅 𝜆 𝑑𝜆𝜔

= 𝑅𝑖𝑝

𝐺𝑝,𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝐺 𝜆 𝑑𝜆𝜔

𝑝

= � 𝐸𝑝 𝜆 𝜎𝑖 𝜆 𝐺 𝜆 𝑑𝜆𝜔

= 𝐺𝑖𝑝

𝐵𝑝,𝑖 = � 𝐸 𝜆 𝑆𝑖 𝜆 𝐵 𝜆 𝑑𝜆𝜔

𝑝

= � 𝐸𝑝 𝜆 𝜎𝑖 𝜆 𝐵 𝜆 𝑑𝜆𝜔

= 𝐵𝑖𝑝

– Shade of grey algorithm • Assumption

𝜇𝑝 𝑆 𝜆 = �𝑆𝑖 𝜆 𝑝

𝑁

𝑁

𝑖=1

1/𝑝

= 𝑘𝑝

(16)

𝐼:image

𝑆𝑖 𝜆

14/19

𝜇𝑝 𝑅𝑝 = � 𝐸𝑝 𝜆 �𝑆𝑖 𝜆 𝑝

𝑁

𝑁

𝑖=1

𝑅 𝜆 𝑑𝜆𝜔

1/𝑝

= 𝑘𝑝𝑅𝑒

𝜇𝑝 𝐺𝑝 = � 𝐸𝑝 𝜆 �𝑆𝑖 𝜆 𝑝

𝑁

𝑁

𝑖=1

𝐺 𝜆 𝑑𝜆𝜔

1/𝑝

= 𝑘𝑝𝐺𝑒

𝜇𝑝 𝐵𝑝 = � 𝐸𝑝 𝜆 �𝑆𝑖 𝜆 𝑝

𝑁

𝑁

𝑖=1

𝐵 𝜆 𝑑𝜆𝜔

1/𝑝

= 𝑘𝑝𝐵𝑒

where 𝑅𝑝 = 𝑅1𝑝,𝑅2

𝑝,⋯ ,𝑅𝑁𝑝 𝑇

, 𝐺𝑝 = 𝐺1𝑝,𝐺2

𝑝,⋯ ,𝐺𝑁𝑝 𝑇

, 𝐵𝑝 = 𝐵1𝑝,𝐵2

𝑝,⋯ ,𝐵𝑁𝑝 𝑇

15/19

Experimental evaluation Evaluation by using angular error

– Using two databases • Data set suggested Barnard et al. • One consisting of 321 images of a variety of 32 scenes • Another of 220 images of a variety of 22 scenes • Both groups taken under 11 coloured illuminant\ • Comparison measure

− Angular error: Equation (18) in this paper − Distance error in the chromaticity space: Equation (19) in this paper

16/19

– 𝐿6 norm: Working best overall

17/19

Fig. 2. The figure shows the angular error of the group A images for 30 values of p

Fig. 3. The figure shows the angular error of the group B images for 30 values of p

Table 1. Results for the p shade of grey algorithm on two databases considered: the firsts two columns are the mean of angular errors and the lasts two report the distance error in the chromaticities space.

18/19

Conclusion Shade of grey algorithm

– Performance • 𝐿6 norm: Working best overall • Comparable to many advanced colour constancy algorithm for the

norm 6 algorithm • But, significant computational cost

19/19