analysis of colour channel coupling from a physics-based viewpoint: application to colour edge...

ARTICLE IN PRESS

Pattern Recognition 43 (2010) 2507–2520

Contents lists available at ScienceDirect

Pattern Recognition

0031-32

doi:10.1

� Corr

E-m

URL

journal homepage: www.elsevier.com/locate/pr

Analysis of colour channel coupling from a physics-based viewpoint:Application to colour edge detection

Alberto Ortiz �, Gabriel Oliver

University of the Balearic Islands, Department of Mathematics and Computer Science, Edificio Anselm Turmeda - Campus UIB, Cra. Valldemossa, km 7.5,

07071 Palma de Mallorca, Spain

a r t i c l e i n f o

Article history:

Received 6 August 2007

Received in revised form

29 January 2010

Accepted 10 February 2010

Keywords:

Physics-based vision

Colour

Edge detection

03/$ - see front matter & 2010 Elsevier Ltd. A

016/j.patcog.2010.02.004

esponding author. Tel.: +34 971 172992; fax

ail addresses: [email protected] (A. Ortiz), g

: http://dmi.uib.es/aortiz (A. Ortiz).

a b s t r a c t

The Dichromatic Reflection Model (DRM) introduced by Shafer in 1985 is surely the most referenced

physics-based model of image formation. A preliminary analysis of this model derives in the conclusion

that colour channels remain coupled by the reflectance of objects surface material. Taking this

observation as a basis, this paper shows that this coupling manifests itself at the signal level in the form

of a set of properties only satisfied in uniform reflectance areas. Such properties are deeply analysed

and formalized throughout the paper, and, eventually, they are stated geometrically. As will be seen, a

compatibility relationship stems from this geometric interpretation, which allows checking at a very

low cost whether two pixels correspond to the same scene reflectance or not. Finally, an edge detector

based on the use of the previous relationship is presented as an example of application. This edge

detector inherits all the properties of the compatibility relationship: simplicity, low computational

power requirements, sensor noise adaptivity and insensitivity to image intensity changes due to scene

objects curvature.

& 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Undoubtedly, the isolation of scene objects in images has beenthe main concern of a vast part of the computer vision literatureproduced since this discipline emerged. Its importance as a priorstep for many vision tasks within a scene understanding frame-work is beyond doubt. Apart from other approaches based onother visual clues (e.g. motion, texture, depth, etc.), many of thesolutions proposed look for abrupt changes in image intensity inmonochrome or colour images (see several recent surveys fromwithin the context of image segmentation [1,2] and about edgedetection [3,4]). Although they are increasingly more sophisti-cated, irrespective of the underlying technique adopted (e.g.based on uni- or multi-dimensional differential calculus [4], usingparametric fitting models [5], based on variational calculus [6],adopting a morphological approach [7], based on graphs [8], usingscale space theory [9,10] or integrating multiple cues [11–13], toname but a few), they commonly assume that images are noisypiecewise constant bidimensional functions (and, therefore, someincorporate a more or less complex statistical treatment of imagedata). However, when curved objects are imaged or some opticalphenomena appear in the image, such as specularities or shadows,

ll rights reserved.

: +34 971 173003.

[email protected] (G. Oliver).

this model of image formation is not accurate enough so as toensure an adequate performance. The direct consequence is thatthe aforementioned approaches fail because scene curvature,among others, gives rise to noticeable changes in image intensitynot necessarily related to the transition between objects.

Solutions accounting for the aforementioned optical phenom-ena assume a more complex image formation model and, in amore or less explicit way, are devised around the concept ofsurface reflectance, looking for image areas homogeneous in thisrespect. Among the several methods proposed throughout theyears, photometric invariants and the analysis of cluster struc-tures in RGB or alternative colour spaces have been the two moststudied approaches (see [14–21], just to cite some of them). Imagenoise, however, deforms clusters and makes photometric invar-iants to be not as invariant as expected, which leads to visionalgorithms whose performance critically depends on the accuracyof the estimation of the cluster parameters or the invariant values.Several strategies have been reported in the related literature inorder to counteract those undesired effects. For instance, Ohtaet al. [22] and Healey [23] only consider, when computingphotometric invariants, RGB values above a certain threshold,which is determined adaptively in the case of Healey. On the otherhand, Klinker et al. [15] allow clusters of a certain thickness inaccordance with sensor noise, while Yuan et al. [24] define a fuzzymeasure to refine colour coarse clusters obtained in a firstsegmentation stage. Some others resort to more elaboratedmethods, such as the case of Gevers and Stokman [25], which

www.elsevier.com/pr

dx.doi.org/10.1016/j.patcog.2010.02.004

mailto:[email protected]

mailto:[email protected]

mailto:http://dmi.uib.es/~aortiz.3d

ARTICLE IN PRESS

A. Ortiz, G. Oliver / Pattern Recognition 43 (2010) 2507–25202508

make use of variable kernel density estimation and propagation ofsensor noise to construct robust colour invariant histograms, orthe case of van de Weijer et al. [26], who introduce robust quasi-invariants to avoid the instabilities of full standard photometricinvariants.

Overcoming those problems and providing a novel solution isthe main motivation of this work. The new approach derives fromthe coupling which can be observed among colour channels inareas of uniform reflectance. In other words, in such areas, colourchannels are not free to take any intensity value, but they dependon the values taken by the other colour channels. As will be seenin the following sections, this paper presents a formal and deepanalysis of colour channel behaviour in those areas. Such analysishas culminated in the formulation of three properties whichformalize several aspects of the coupling.

Moreover, the coupling properties admit a geometrical inter-pretation through the space of possible intensity values attainableby any combination of colour channels. As a result, a relation ofcompatibility between pairs of pixels C has been derived from thethree properties. C is defined such that, if any of the threeproperties fails to be met by a certain couple of pixels for anycombination of colour channels, then both pixels are notC- related because it is impossible that they can correspond tothe same scene material. Therefore, all three properties result tobe necessary conditions for searching areas of uniform reflectancein the image. In a noiseless environment, checking C is as simpleas determining the orientation of a vector in the aforementionedspace. To cover noisy environments, C is redefined by means ofthe incorporation of intensity uncertainties derived from a noisemodel related to the operation of real imaging sensors. Despitethe reformulation, however, the complexity of checking C doesnot significantly increase. Finally, the limitations of the couplinganalysis are also discussed. To sum up, the main contribution ofthis work is the colour channel coupling analysis described in thepaper and ultimately the formal derivation of relation C.

As will be seen, for the formalization of the coupling analysiswe make use of the Dichromatic Reflection Model (DRM) [27], thephysics-based model of image formation that is both referred toand used most often due to its general validity. Furthermore, inorder to widen its applicability, the usual expression of the DRMhas been extended with a term accounting for ambient illumina-tion, what is ignored in most publications concerned, in one wayor another, with the physics of image formation. Finally, theformalization here presented is independent of particular expres-sions of the different (spectral and non-spectral) terms of theDRM, so that other more elaborate models of light reflection canbe covered without any change.

Although the applicability of the results of the previousanalysis is obvious for edge detection and image segmentation

tasks, they can also turn out to be useful whenever uniformreflectance areas of the image must be identified, such as, forinstance, for shape from shading tasks. Nevertheless, by way ofillustration, the paper finishes with the proposal of a novel edgedetection technique which inherits all the features of thecompatibility relationship C: simplicity, low computational powerrequirements, sensor noise adaptivity and insensitivity to imageintensity changes due to scene objects curvature.

The remainder of the paper is organized as follows: first,Section 2 revises the most relevant features of the DRM anddescribes the model of image formation which has been assumedin this work; next, Sections 3–5 investigate the properties ofuniform reflectance areas according to the image formation modelpresented in Section 2; Section 6 enunciates a compatibilityrelationship among pixels based on the colour channel couplinganalysis proposed in the previous sections; Section 7 discusses thecoverage of reflectance transitions achieved by analysis of colour

channels coupling; Section 8 relates colour channel couplinganalysis to similar work; Section 9 comments on the use of all theaforementioned to compute an edge map representing thereflectance changes of the imaged scene; finally, Section 10concludes the paper. A list with the symbols used in the paper canbe found in the appendix.

2. Physics-based models of image formation

2.1. Global overview

Generally speaking, irradiance values measured at pixellocations are a combination of several scene factors whichinteract with each other: the illumination distribution, thereflection properties of scene objects—which in particulardetermine the colour (red, green, light blue, etc.) by which weperceive them—the objects geometry, both at the macroscopicand microscopic level (surface roughness), the propagationmedium and the performance of the imaging sensor. The finalpixel value is eventually determined by the (physical) laws thatgovern the processes of light propagation, reflection and imageacquisition. Researchers using these models to develop visionalgorithms have mostly focused their attention on the light-matter interaction part of the image formation process. In thisrespect, a lot of knowledge has been gained over the years withcontributions coming from fields such as radiant heat transfer,energy conservation, computer graphics and vision itself [28–30].Of particular importance to the computer vision community arethe developments of several models of reflection manageableenough so as to be effectively incorporated in machine visionalgorithms at the cost of some loss of physical accuracy.

One of those models is the often-cited Dichromatic ReflectionModel (DRM) proposed by Shafer [27]. The DRM expresses theradiance at a given scene point x for a certain wavelength l as asum of two reflection components, one for body reflection (alsoreferred to as the diffuse term) and the other one for interfacereflection (also referred to as the specular term), as indicated inthe following equation:

Lðx; lÞ ¼mbðxÞ½LdðlÞrbðx; lÞ�zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{Lbðx;lÞ

þmiðxÞ½LdðlÞriðx; lÞ�zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{Liðx;lÞ

: ð1Þ

As can be seen, in the DRM, each reflection component Lj ðjAfb; igÞis modeled as the product of two terms: (1) Cjðx; lÞ ¼ LdðlÞrjðx; lÞ,dependent only on wavelength and not on local geometry, andexpressing the fraction of the incoming (directional) light LdðlÞwhich is conveyed by that reflection component due to thecorresponding (body or interface) reflectance 0rrjðx;lÞr1; and(2) mjðxÞA ½0;1�, which is a geometrical factor which only dependson the local surface geometry at point x. One important propertyof the DRM is this separability of the spectral and the non-spectral(geometric) factors. According to Shafer, the separation of theinterface reflection component into a geometrical and a spectralterm has an effect no greater than 2% in pixel-value errors, andthis error is attained only if the angle of incidence of the light islarger than 603 and the viewing angle is nearly the perfectspecular direction. As for the body reflection component, there isan interdependence between the geometry and the spectralfactors only if Ldðx;lÞriðx; lÞ is not constant; since it generallydoes not vary much with wavelength, the effect of thisinterdependence is negligible.

The validity of the DRM, as a whole or parts of it, has beenexperimentally checked by several researchers for a large numberof surface materials (see [31–35], among others). Besides,although there are other models of image formation, they can

ARTICLE IN PRESS

A. Ortiz, G. Oliver / Pattern Recognition 43 (2010) 2507–2520 2509

all be assimilated into the general structure of the DRM or parts ofit, since they mainly provide more or less sophisticated, butseparable, expressions for Lb and Li (see [36] for more accuratemodels of body reflection, alternative to the usually employedLambertian model, [33,37] for more or less detailed descriptionsof interface reflection terms, and [38,39] for models not foundedon a sound physical basis but providing a surprisingly remarkablerealism). Summing up, the DRM has proved to be adequate formost materials. It is because of this general application that thestudy presented in this paper has focused on the DRM and itsproperties.

2.2. Model of image formation assumed in this work

The DRM as presented earlier depends on the assumption thatthe illumination at any point comes from a single light source.A further improvement to the model can be made by incorporat-ing an ambient lighting term Laðx; lÞ ¼ LaðlÞraðx; lÞ, representinglight coming from all directions in equal amounts, LaðlÞ, whichincreases the radiance from scene surfaces independently of localsurface geometry. The reflectance term in this case, ra, isgenerally expressed as a linear combination of the body andinterface reflectances since the light reflected contains a part dueto interface reflection and a part due to body reflection [27]. Afteradding the ambient term, the DRM turns out to be

Lðx; lÞ ¼ LaðlÞraðx;lÞzfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflffl{Laðx;lÞ

þmbðxÞ½LdðlÞrbðx; lÞ�zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{Lbðx;lÞ

þmiðxÞ½LdðlÞriðx; lÞ�zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{Liðx;lÞ

: ð2Þ

As pointed out by Shafer, this model is a better approximation ofdaylight—which contains directional light from a point lightsource (Ld terms), the sun, and ambient light from the sky(La term)—and of light in a room—which comes from (directional)light fixtures (Ld terms) and from inter-reflections on walls andother objects (La term).

Assume a non-attenuating propagation medium and ignorethe blurring and low-pass filtering effects of the point-spreadfunction of the optics in a properly focused camera. After thephoton-to-charge conversion and the charge integration pro-cesses performed by current imaging sensors, the final digitalvalue corresponding to the radiance Lðx; lÞ at the scene points x

optically reachable from a sensor cell (i,j) can be described by thefollowing compact and useful expression [40]:

Dcði; jÞ ¼ Ccaði; jÞþmbði; jÞC

cbði; jÞþmiði; jÞC

ci ði; jÞ; ð3Þ

where Cca, Cc

b and Cci are, respectively, the so-called ambient,

body and interface composite reflectances for colour channel c,representing the joint contribution of lighting and materialreflectance to the corresponding reflection component for thatcolour band.

3. General properties of uniformly coloured image areas

Within an image area whose pixels correspond to scene pointsbelonging to the same material, the ambient, body and interfacecolours are constant (i.e. rbði; j; lÞ ¼ rbðlÞ and riði; j; lÞ ¼ riðlÞ and,by extension, also raði; j; lÞ ¼ raðlÞ because of its dependence onrb and ri). In a noiseless environment, changes in the pixel valuesbetween image locations are, thus, only due to changes in thegeometrical factors mb and mi. As a consequence, in areas ofuniform reflectance, colour channels remain coupled, in the sensethat they are not free to take any intensity value, but they dependon the values taken by the other colour channels through thebody and interface reflectances and the way in which they arecombined in the image formation model. The study of this

coupling has allowed us to identify up to three behaviours for thecolour signal within areas of uniform reflectance. Such behaviourscan be summarized as follows:

B1.
Colour channels do not cross one another. B2. Colour channels vary in a coordinated way: when one
changes, so do the others, and in the same sense, all increaseor all decrease.

B3.
As the intensity in one channel decreases, so does thedifference between colour channel intensities; the oppositehappens when the intensity in one channel increases.
Behaviours B1–B3 derive in fact from three necessary condi-tions of compatibility between pixels sharing the same scenematerial. Taking into account the following notation:

�
p and q represent different image locations, e.g. (i1, j1) and(i2, j2), � sign½x� ¼ x=jxj if xa0; sign½0� ¼ 0, and � sign½x� ¼ �sign½y�3sign½x� ¼ sign½y� or sign½x� ¼ 0 or sign½y� ¼ 0,
the compatibility conditions can be formally stated in the form ofthe properties P1–P3 below:

Property P1. Given any two image locations p and q corresponding

to the same scene material, and any two colour channels c1 and c2,then:

sign½Dc1 ðpÞK1�Dc2 ðpÞK2� ¼�sign½Dc1 ðqÞK1�Dc2 ðqÞK2�;

where K1;K2AR depend on the image formation model.

Property P2. Given any image location p not involved in a material

change along a specific direction x over the image plane, and any two

colour channels c1 and c2, then:

signdDc1 ðpÞ

dx

� �¼ �sign

dDc2 ðpÞ

dx

� �:

Property P3. Given any two image locations p and q corresponding

to the same scene material, and two colour channels c1 and c2 such

that Dc1 ðpÞZDc2 ðpÞ, then:

sign½Dc1 ðpÞ�Dc1 ðqÞ� ¼ �sign½ðDc1 ðpÞ�Dc2 ðpÞÞ�ðDc1 ðqÞ�Dc2 ðqÞÞ�:

To finish this section, the following comments provide addi-tional insight into the colour channel coupling properties:

�
Property P1 expresses the fact that colour channels do notcross one another in areas of uniform reflectance. This is moreevident when one sets K1=K2; otherwise, this behaviour arisesfor scaled versions of the colour planes, i.e. Dc1 K1 and Dc2 K2, forcertain K1 and K2. As will be discussed later, the values for K1
and K2 depend on the chosen particular instance of the imageformation model.Observe that this property predicts that colour channels (orscaled versions of them) are ordered in regions of uniformreflectance. That is to say, if for pixel p the intensity for channelc1 is ‘‘above’’ the intensity of channel c2 (i.e. the intensity inchannel c1 is larger than in channel c2), then for any pixel q inthe same region, colour channel c1 will also be ‘‘above’’ colourchannel c2.
� Regarding Property P2, an alternative formulation follows:
‘‘Given any two image locations p and q corresponding to thesame scene material, and any two colour channels c1 and c2,sign½Dc1 ðpÞ�Dc1 ðqÞ� ¼ �sign½Dc2 ðpÞ�Dc2 ðqÞ�.’’
� Finally, note that, although Property P3 imposes a specific role
on channels c1 and c2, it indirectly takes into consideration thefour cases that are possible: one channel is ‘‘above’’/‘‘below’’

ARTICLE IN PRESS


the other channel, and intensity decreases/increases between p

and q.

4. Particularization of the general properties

The fulfillment of Properties P1–P3 depends on the particularinstantiation of the image formation model expressed by Eq. (3).Four instances are distinguished:

�
Case 1: non-glossy pixels/without ambient lighting � Case 2: non-glossy pixels/with ambient lighting � Case 3: glossy pixels/without ambient lighting � Case 4: glossy pixels/with ambient lighting
All these combinations are revised and analysed along thefollowing sections, which, in turn, formalize the results obtainedas a set of propositions.

4.1. Case 1: non-glossy pixels/without ambient lighting

If interface reflection and ambient lighting are not taken intoaccount, the intensity of every colour channel is reduced to thefollowing expression:

Dcði; jÞ ¼mbði; jÞCcbði; jÞ: ð4Þ

In such a case, the three properties are met. This fact is stated inthe following proposition:

Proposition 1. Consider a pair of colour channels c1 and c2, two

image locations p and q sharing the same scene material, and a scene

for which directional illumination is uniform. If ambient lighting is

negligible in the scene and interface reflection is negligible at p and q,then:

�
Property P1 holds for any K1=K2, and
�
Properties P2 and P3 hold.
In other words, assuming the model of Eq. (4), in uniformreflectance areas, colour channels do not cross one another(behaviour B1), they vary coordinately (behaviour B2), and thedifference between colour channel intensities decreases—resp.increases—as the intensity in anyone of them decreases—resp.increases—(behaviour B3).

4.2. Case 2: non-glossy pixels/with ambient lighting

The image formation model for this case is given by


cbði; jÞ: ð5Þ

For this instance, Properties P1 and P3 require, to beapplicable, the additional hypothesis of the proportionality

Fig. 1. Four possible configurations of the

between ambient and directional illumination, i.e. LaðlÞ ¼aLdðlÞ ½aZ0�; 8l; Property P2 is however met in general. All thesefacts are stated in the following proposition:



for which directional illumination is uniform. If interface reflection is

negligible at p and q, then:

�

stra

Property P1 holds if ambient illumination is proportional to

directional illumination and for any K1=K2;
� Property P2 holds in any case; and
�
Property P3 holds if ambient illumination is proportional to
directional illumination.

In this case, with the introduction of the ambient reflectionterm, the sign of Dc1 ðpÞ�Dc2 ðpÞ does not determine, in the generalcase, the sign of the difference for any other image point for the
same scene material. In effect, for all the points p=(i, j) belongingto the same scene material, the differences between colourchannels c1 and c2, DDðpÞ ¼Dc1 ðpÞ�Dc2 ðpÞ ¼ ðCc1
a �Cc2a ÞþmbðpÞ

ðCc1

b �Cc2

b Þ ¼DCaþmbðpÞDCb, lie in a straight line with variablesDD and mb and parameters DCa (intercept with DD�axis) and DCb

(slope with mb-axis). This straight line can adopt any of the fourconfigurations shown in Fig. 1. In cases (a) and (b), the signs ofDCa and DCb are the same and, thus, coincide with the sign of DD

for any value of mb. However, in cases (c) and (d), the signs of DCa

and DCb are different and only if mb(p) and mb(q) are both aboveor both below P¼Q ¼�DCa=DCb the signs of the differencesDc1 ðpÞ�Dc2 ðpÞ and Dc1 ðqÞ�Dc2 ðqÞ are the same. The constraintrelating ambient lighting to directional illumination through thescale factor aZ0 in Proposition 2 forces situations (a) and(b) everywhere in the scene.

4.3. Case 3: glossy pixels/without ambient lighting

In this case, the intensity of every colour channel is given bythe following expression:

Dcði; jÞ ¼mbði; jÞCcbði; jÞþmiði; jÞC

ci ði; jÞ: ð6Þ

When interface reflection is introduced in the formationmodel, Property P1 is satisfied, for certain values of K1 and K2

and the additional hypothesis of the Neutral Interface Reflection

model (NIR), i.e. riðlÞ ¼ ri, 8l [32], as is indicated in the followingproposition:



for which directional illumination is uniform. If ambient lighting is

negligible in the scene and the NIR formation model is assumed, then

Property P1 holds for K1 ¼ Lc2

d and K2 ¼ Lc1

d , where Ldc is the (scaled)

radiance of the directional light source for channel c.

ight line DDðpÞ ¼DCaþmbðpÞDCb .

ARTICLE IN PRESS

20 40 60 80 100 1200

50

100

150

200

250row 64

20 40 60 80 100 1200

50

100

150

200

250

20 40 60 80 100 1200

50

100

150

200

250

Fig. 2. (a) A synthetic image without ambient lighting, (b) intensity profile for row 64 (in yellow in (a)) for every colour channel, (c) body component profiles, and

(d) interface component profiles. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Table 1Fulfillment of Properties P1–P3 depending on the particular instantiation of the

image formation model.

Case Properties fulfilled and conditions

(1) Matte pixels, no

ambient lighting

Property P1 for K1=K2

Properties P2 and P3 always

(2) Matte pixels,

ambient lighting

Property P1 for K1=K2 and La proportional to Ld

Property P2 always

Property P3 for La proportional to Ld

(3) Glossy pixels, no

ambient lightingProperty P1 for K1 ¼ Lc2

d, K2 ¼ Lc1

dunder the NIR

model

(4) Glossy pixels,

ambient lightingProperty P1 for K1 ¼ Lc2

d, K2 ¼ Lc1

d, under the NIR

model and La proportional to Ld


In order for the previous proposition to be applicable, Ldc must

thus be estimated for every colour channel c for the particularscene. Any of the methods presented in [41, Chapter 5] or in anyother publication regarding the estimation of the lightingparameters can be used for this purpose.

Regarding Property P2, the incorporation of interface reflectioninto the formation model cancels its general satisfiability. This isbecause the derivative of the intensity in every channel not onlydepends now on the sign of the derivative of mb but also on thederivative of mi (Cb and Ci are constant in a uniform reflectancearea). For p:

dDc1 ðpÞ

dx

� �¼m0bðpÞC

c1

b þm0iðpÞCc1

i ;dDc2 ðpÞ

dx

� �¼m0bðpÞC

c2

b þm0iðpÞCc2

i ;

where m0bðpÞ and m0iðpÞ are the first-order derivatives alongdirection x of mb(p) and mi(p), respectively. As a consequence, ingeneral:

m0bðpÞCc1

b þm0iðpÞCc1

i b0Wm0bðpÞCc2

b þm0iðpÞCc2

i b0

Observe that the derivatives of both channels c1 and c2 will havethe same sign if m0b and m0i are both positive or both negative, or,

should their signs do not coincide, if the derivative of the samereflection component dominates the sum in both cases (i.e.

jm0bðpÞCc1

b jbjm0iðpÞC

c1

i j and jm0bðpÞCc2

b jbjm0iðpÞC

c2

i j). By way of exam-

ple, in the intensity profiles of Fig. 2, the dotted lines enclose anarea where m0b and m0i have different signs, but the derivatives of

every colour channel all have the same sign.Finally, as for Property P3, Fig. 2 shows precisely a case where

it is not met. In effect, the intensity levels of the green and bluechannels within the area enclosed by the dotted lines are suchthat, when the intensity increases in both, the difference betweenboth channels does not increase, but in fact decreases.

4.4. Case 4: glossy pixels/with ambient lighting

In the general case:


cbði; jÞþmiði; jÞC

ci ði; jÞ: ð7Þ

With the incorporation of ambient lighting and the interfacereflection term, Property P1 still holds, for certain values of K1 andK2 and the additional hypotheses of the proportionality betweenambient and directional lighting, and the NIR formation model.The following proposition states the aforementioned in a formalway:



for which directional illumination is uniform. If ambient illumination

is proportional to directional illumination and if the NIR formation

model is assumed, then Property P1 holds for K1 ¼ Lc2

d and K2 ¼ Lc1

d ,

where Ldc is the (scaled) radiance of the directional light source for

channel c.

The same remarks made for case 3 apply in this case withrespect to Properties P2 and P3.

4.5. Summary of the applicability of the properties

To recapitulate on the previous sections, Table 1 summarizesthe dependence of the fulfillment of Properties P1–P3 onthe particular instantiation of the image formation model. Someconclusions regarding the applicability of those properties can bedrawn now that the four cases have been revised:

�
When the interface reflection term is introduced, only PropertyP1 is satisfied in general within uniform reflectance areas. Thismeans that false material changes can be detected sometimesinside or near specularities. As has been discussed earlier, itdepends on whether the sign of m0b or m0i are the same or not,and on which term dominates the value of the derivative: thebody/ambient reflection terms or the interface reflection term.Therefore, specularities will not always give rise to falsematerial changes. � Properties P1 and P3 require that La is proportional to Ld in
order for them to be applicable when ambient lighting is notnegligible. As stated in Section 2, the ambient lighting comes,in an outdoor environment, from the sky through scattering inthe atmosphere, or, in a room, from inter-reflections on thewalls and other objects. Given these facts, it is obvious thatsome relationship can be expected between ambient lightingand directional lighting. Unfortunately, no physical evidenceregarding the proportionality relationship has been found inthe related literature, apart from other authors also making

ARTICLE IN PRESS


use of it, be it directly [16] or indirectly by assuming whiteillumination [14,42]. However, the experiments performedhave shown that, in general, violations of Properties P1 and P3are closely related to reflectance transitions.

5. Unfulfillment of the properties and reflectance changes

Failure to meet any of the coupling properties for anycombination of colour channels indicates a difference in reflec-tance between the two image points considered and, therefore,the impossibility of sharing the same scene material. Thefollowing proposition describes which sort of differences arise.


image locations p and q, and a scene for which directional

illumination is uniform. Under the assumption of ambient illumina-

tion proportional to directional illumination (and the NIR formation

model for case (a.2)):

(a.1)

Fig. 3.met. (

Proper

if interface reflection is negligible at p and q, then Property P1(K1=K2) does not hold iff

sign½Cc1

b ðpÞ�Cc2

b ðpÞ�asign½Cc1

b ðqÞ�Cc2

b ðqÞ�

if p and q do show interface reflection, then Property P1
(a.2) ðK1 ¼ Lc2
d ;K2 ¼ Lc1

d Þ does not hold iff

sign½Cc1

b ðpÞK1�Cc2

b ðpÞK2�asign½Cc1

b ðqÞK1�Cc2

b ðqÞK2�

if Property P2 does not hold then:
(b)
Cc2

b ðpÞ

Cc1

b ðpÞa

Cc2

b ðqÞ

Cc1

b ðqÞ

if Property P3 does not hold then:
(c)
sign½Cc1

b ðpÞ�Cc1

b ðqÞ� ¼ sign½Cc2

b ðpÞ�Cc2

b ðqÞ�a0

The following observations apply for Proposition 5:

�
First, Proposition 5 relates the unfulfillment of PropertiesP1–P3 to changes in the composite body reflectance betweenimage locations p and q. Observe that, if lighting is uniformacross the scene, a change in the composite body reflectanceentails a change in the body reflectance (see [41]). Henceforth,every reflectance transition will be referred to as: (a.1) and(a.2) channel crossing (CHC) transition, (b) non-coinciding
derivative (NCD) transition and (c) non-decreasing difference

(NDD) transition.
� Second, note that Proposition 5 states that, for cases (a.1) and
(a.2), if the reflectance transition takes place, then Property P1is violated, and vice versa. However, cases (b) and (c) are notdouble implications and, therefore, not all body reflectancetransitions of the types indicated will necessarily be perceivedas violations of Properties P2–P3. Fig. 3(a) and (b) illustratethese facts.

(a) A case where possibly Cc2

b ðpÞ=Cc1

b ðpÞaCc2

b ðqÞ=Cc1

b ðqÞ, but Property P2 is

b) A case where possibly signðCc1

b ðpÞ�Cc1

b ðqÞÞ ¼ signðCc2

b ðpÞ�Cc2

b ðqÞÞa0 and

ty P3 still holds. [p=(i1,j1), q=(i2,j2)].

6. Formulation of a compatibility relationship between pixels

Properties P1–P3 can be said to lead to a set of necessarycompatibility conditions between pixels. In effect, according toProposition 5, if two pixels fail to satisfy any of the threeproperties for any of the two-by-two combinations betweencolour channels, then both pixels cannot correspond to the samescene material. In this section, now that the usefulness of theaforementioned properties has been discussed, all three will beunified in a single compatibility relationship between imagepixels by means of their geometric interpretation. To simplifythe explanation, noiseless irradiance measurements will beconsidered first. Next, the relationship will be extended to realcameras.

For a start, let us thus consider a noiseless sensor and a certaincombination of colour channels, say c1 and c2. Now, PropertiesP1–P3 will be stated geometrically. To this end, consider the spaceof intensity values taken by both channels and the set of intensitypairs ðDc1 ðqÞ;Dc2 ðqÞÞ which are compatible with a certain intensitypair ðDc1 ðpÞ;Dc2 ðpÞÞ, in the sense of the satisfaction of the couplingproperties. The shaded areas of Fig. 4(a) and (c) are then thecompatibility zones for case Dc1 ðpÞZDc2 ðpÞ and, respectively,Properties P1 and P3; for case Dc1 ðpÞrDc2 ðpÞ, the compatibilityzones would then be the complement of the shaded areas. Thedependence on Dc1 ðpÞxDc2 ðpÞ does not exist for Property P2 and,therefore, the compatibility zone is just the shaded area ofFig. 4(b). For Properties P1 and P3, if Dc1 ðpÞ ¼Dc2 ðpÞ, thecompatibility zone is the union of the compatibility zones forcases Dc1 ðpÞrDc2 ðpÞ and Dc1 ðpÞZDc2 ðpÞ.

Finally, if for a given ðDc1 ðpÞ;Dc2 ðpÞÞ, the pair ðDc1 ðqÞ;Dc2 ðqÞÞ isnot inside the compatibility zones for all properties, then it isimpossible that p and q share the same scene material. Fig. 4(d)shows the compatibility zone for all three properties simulta-neously (triangles correspond to the case Dc1 ðpÞrDc2 ðpÞ, whilecircles are for case Dc1 ðpÞZDc2 ðpÞ).

Fig. 4. Geometrical interpretation of Properties P1–P3. (In the figures, Dnk is an

abbreviation of Dcn ðkÞ.)

ARTICLE IN PRESS

Fig. 5. Redefinition of the C compatibility relation. (The uncertainty areas have

been magnified for illustration purposes. In the figure, Dnk is an abbreviation of

Dcn ðkÞ.)


In view of these figures, a compatibility relation C can bewritten as:

ðDc1 ðqÞ;Dc2 ðqÞÞ is C- compatible with ðDc1 ðpÞ;Dc2 ðpÞÞ if:

(i)

1

when Dc1 ðpÞZDc2 ðpÞ, the orientation of the vector joining bothlies within ½03;453

�; and
(ii) when Dc1 ðpÞrDc2 ðpÞ, the orientation of the vector joining both
lies within ½453;903�.

This formulation of C assumes that the origin of the vector joiningðDc1 ðpÞ;Dc2 ðpÞÞ and ðDc1 ðqÞ;Dc2 ðqÞÞ is chosen so that its orientationlies between �903 and 903 (see Fig. 4(e) for the angle convention).

In order to counteract sensor noise in an adaptive way whenapplying relation C, intensity uncertainties are considered. That isto say, for every colour channel c, every digital noisy level Dc

produced by the camera is associated to an interval½Dc�dðDcÞ;DcþdðDcÞ�. For determining dðDcÞ, we suggest usingalgorithm R2CIU [43], which computes sensor uncertainties forevery noisy intensity level Dc on the basis of the camera noisemodel by Healey and Kondepudy [40]. With the introduction ofthese uncertainties, for a given ðDc1 ðpÞ;Dc2 ðpÞÞ, a rectangularuncertainty area around ðDc1 ðpÞ;Dc2 ðpÞÞ covering tC uncertaintiesis defined, as depicted in Fig. 5.C is then reformulated as:

ðDc1 ðqÞ;Dc2 ðqÞÞ is C- compatible with ðDc1 ðpÞ;Dc2 ðpÞÞ if any of thepoints belonging to the uncertainty area of ðDc1 ðqÞ;Dc2 ðqÞÞ fallswithin the union of the compatibility zones of the pointsbelonging to the uncertainty area of ðDc1 ðpÞ;Dc2 ðpÞÞ (shadedarea of Fig. 5).

The incompatibility between ðDc1 ðpÞ;Dc2 ðpÞÞ and ðDc1 ðqÞ; Dc2 ðqÞÞ

stems, thus, from the fact that a very low probability exists thatany pair of intensity levels which could lead, through noise, toðDc1 ðpÞ;Dc2 ðpÞÞ was compatible, in the sense of Properties P1–P3,with any pair of intensity levels leading, through noise, toðDc1 ðqÞ;Dc2 ðqÞÞ.

Finally, being dðDcÞ defined as indicated before, then, resortingto the Chebyshev inequality,1 observe that tC=2,3 and 4 representthat Dc will lie inside ½Dc�tCdðDcÞ;DcþtCdðDcÞ� with, respectively,probabilities of, at least, 75.00%, 88.89% and 93.75%. If, besides,the different sensor noise sources are assumed to obey Gaussiandistributions, the noise distribution turns out to be normal, and,therefore, the previous probabilities for tC=2,3 and 4 increase,respectively, up to 95.45%, 99.73% and 99.99%.

For a random variable Xðm;sÞ, PðjX�mjZksÞr1=k2, k40.

7. Coverage of reflectance transitions

The compatibility relationship C formulated in Section 6 coversa broad spectrum of body reflectance changes. For instance, byonly taking into account CHC reflectance transitions, C achievesthe detection of 83.33% of all possible transitions (see [41,Appendix C] for the derivation of this percentage). By way ofillustration, Fig. 6 shows the C- incompatibilities found for severalreal images. These maps were built by checking, at every pixel p,whether for any of its eight neighbours q and at least onecombination of colour channels, both pixels were not compatiblewith one another in the sense of relation C. As can be observedfrom the maps, C is able to capture a considerable amount ofreflectance transitions as predicted above (this is particularly truefor images (a)–(c)), and, what is also relevant in many visionapplications, at a very low cost regarding both time andcomputation resources. In addition, observe that C allows thedetection of such an amount of reflectance changes withoutresorting to explicitly computing the reflectance of the underlyingsurfaces.

However, despite this important coverage of reflectancetransitions, some others cannot be detected from the unfulfill-ment of Properties P1–P3, despite a reflectance change is actuallyarising there. Therefore, from the point of view of C, thesereflectance changes can be thought of as if the pixels involved allcorresponded to the same scene material. Fig. 3(a) and (b) areexamples of this sort of reflectance transitions.

To investigate further this point, the situation of Fig. 3(a) isconsidered next. As can be observed, all three properties are metin this case. More precisely, given p and q:

Dc1 ðpÞoDc1 ðqÞ; Dc2 ðpÞoDc2 ðqÞ

Dc2 ðpÞ4Dc1 ðpÞ½) Dc2 ðpÞ�Dc1 ðpÞ ¼ ðaþmbðpÞÞðCc2

b ðpÞ�Cc1

b ðpÞÞ40�

Dc2 ðqÞ4Dc1 ðqÞ½) Dc2 ðqÞ�Dc1 ðqÞ ¼ ðaþmbðqÞÞðCc2

b ðqÞ�Cc1

b ðqÞÞ40�

Dc2 ðpÞ�Dc1 ðpÞoDc2 ðqÞ�Dc1 ðqÞ

) ðaþmbðpÞÞðCc2

b ðpÞ�Cc1

b ðpÞÞ|fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}40

o ðaþmbðqÞÞðCc2

b ðqÞ�Cc1

b ðqÞÞ|fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}40

This last inequality can be satisfied under, among others, the twofollowing sets of circumstances:

�
mb(p)=mb(q) and Cc2
b ðpÞ�Cc1

b ðpÞoCc2

b ðqÞ�Cc1

b ðqÞ, which meansthe changes in intensity Dc1 ðpÞ�Dc1 ðqÞ and Dc2 ðpÞ�Dc2 ðqÞ aredue to a change in Cb from p to q; or
� mbðpÞombðqÞ, Cc2
b ðpÞ ¼ Cc2

b ðqÞ and Cc1

b ðpÞ ¼ Cc1

b ðqÞ, which meansthe changes in intensity Dc1 ðpÞ�Dc1 ðqÞ and Dc2 ðpÞ�Dc2 ðqÞ aredue to a change in mb from p to q.

Consequently, in this case, there is no way to decide whetherthere is a change in reflectance or merely a change in the localsurface geometry, and, thus, an ambiguity results. In other words,the interrelation between colour channels is not powerful enoughin this case so as to disambiguate the source of the intensityvariation. For instance, the reflectance transition between thepink and brown toys of image (d) of Fig. 6 corresponds to the casedescribed.

Several other facts can also be observed from the images ofFig. 6. On the one hand, when one of the objects intervening in thetransition is dark or a shadow is involved, then the transition isnot detected by C (see the backpack of image (e), as well as thereflectance transitions involving shadows). This is because thearea of compatibility is large for low intensity values and, thus, itis unlikely to find violations of Properties P1–P3.

On the other hand, as shown in Fig. 7, in real images, for agiven reflectance transition, the image area violating any of

ARTICLE IN PRESS

Fig. 6. Example of detection of C- incompatibilities for real images: (1st row) original images; (2nd row) CHC/NCD/NDD transitions in black (tC=3).

20 40 60 80 100 120 140 160 180 200 2200

50

100

150

200

250INTENSITY

Fig. 7. Examples of reflectance transitions in a real image: intensity profiles for RGB colour channels, for row 50 (in yellow) of leftmost image (shadowed areas correspond

to reflectance transition locations). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 8. Only the LOG zero-crossing behind the circle is considered relevant by C3E.


Properties P1–P3 can comprise more than two pixels because ofthe round-off and between-channel asynchrony2 effectsintroduced by camera aliasing (in noiseless images, reflectancetransitions just involve two pixels). Consequently, ‘‘thickdetections’’ of incompatibility are obtained and small details canbe missed (see, for instance, the text on the right of image (b) ofFig. 6).

From a global perspective, taking into account the previousimages and many others, the incidence of the above-mentionedmisbehaviours tends to be low, which proves that C is effective fordetecting reflectance transitions in many situations at a very lowcost. To cover all the possibilities, however, it is necessary tosupplement colour channel coupling analysis through C withadditional detection means. Section 9 deals with this problem inorder to develop an edge detector.

8. Relation with similar work

The coupling between colour channels which has beenintroduced in the preceding sections derives from the shape ofthe cluster constituted by the set of pixels corresponding to auniformly coloured object in colour space. The DRM predicts that,in general, these pixels are confined within the hyperparallelo-gram defined by the ambient Ca, body Cb and interface Ci colourvectors of Eq. (3); in case of a non-glossy object, the hyperpar-

2 Due to aliasing, the process of change from one material to another can

involve more pixels for colour channel c1 than for colour channel c2.

allelogram reduces into the straight line defined by the ambientand the body colours. The coupling manifests, therefore, in theform of this confinement; that is to say, during the transitionbetween neighbouring pixels, ‘‘jumping’’ outside the cluster is notallowed.

This fact, although not explicitly mentioned, is exploited bymany physics-based segmentation algorithms, either by theanalysis of clusters in colour space or by means of photometricinvariants. Most of those algorithms, however, do not approachthe problem by means of necessary conditions but can be said tomake use of sufficient conditions of compatibility. In effect,belonging to the same cluster in colour space or yielding the sameinvariant value can each be considered a sufficient (andnecessary) condition between pixels at the reflectance level.Since image noise deforms clusters and makes photometricinvariants to be not as invariant as expected, the performance ofthe aforementioned algorithms critically depends on the accuracyof the estimation of the cluster parameters or the invariant values.Our approach, based on necessary conditions, which do not needany of such estimates, must therefore be, in this regard, simpler.

ARTICLE IN PRESS


9. Application to colour edge detection

Colour channel coupling analysis is useful whenever uniformreflectance areas of the image must be identified, such as for edge

detection, image segmentation or shape from shading tasks.Although this paper is not about colour edge detection, since thisis the most obvious application of the compatibility relation C, this

Table 2Some measures for evaluating the performance of edge detection algorithms.

Measure Formulation Typicalvalues

Pratt’s FOMFOMðaÞ ¼ 1

maxfnðAÞ;nðBÞg

PxAB

1

1þadðx;AÞ2a¼ 1,

a¼ 1=9

Discrepancy

percentageD¼

nðADBÞ

nðIÞ

—

BaddeleyDp

wðA;BÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

nðIÞ

PxA I

jwðdðx;AÞÞ�wðdðx;BÞÞjpp

sp=2, c=5

wðtÞ ¼minft; cg

A is the set of image locations corresponding to edges in the reference edge map;

B is the set of image locations corresponding to edges in the edge map under

evaluation; I is the set of all image locations; n(I) is the number of elements of

set I; ADB¼ ðA\BÞSðB\AÞ (set difference operator or exclusive-or for sets);

B\A¼ fxABjx=2Ag (set minus operator); d(x,Y) is the distance between image

location x and the nearest edge in Y.

0 0.2 0C3ES&GC3ES&GC3ES&G

C3ES&G

Spheres &

1 2 3 40

1

2

3

4C3E – Baddeley – t = 3

noise conditions

1 2 3 40

1

2

3


noise conditions

avg ± s

avg ± s

Fig. 9. Comparison between C3E and S&G under noisy conditions by means of synth

conditions, the plots show average and standard deviation values for t=4; (bottom, rig

Baddeley’s measure for t=3y5.

section presents the novel edge detector C3E, which inherits allthe features of C: simplicity, low computational power require-ments, sensor noise adaptivity and insensitivity to image intensitychanges due to scene objects curvature.

9.1. Colour channel coupling-based edge detection (C3E)

As the colour channel coupling analysis is performed acrossthe image, an edge map can be built, where a pixel is classified asan edge whenever it is involved in a reflectance change. Never-theless, as has been shown in Section 7, a small amount ofreflectance transitions are not covered by relation C. Therefore, inorder to obtain a complete edge map, it is necessary to enhancethe previous analysis with additional means. For instance, ifobjects with smooth surfaces are expected in the scene, gradientinformation can be considered practical in order to locate theremaining reflectance transitions. Hereafter, they will be referredto as GRD reflectance transitions.

Second-order derivative operators are of common use for edgedetection in piecewise constant images. In those cases, theidealized step/ramp edges give rise to a zero crossing at the edgebecause of the sudden change in the first-order derivative, whichin turn makes the second-order derivative change its sign inorder to return to a situation of nearly zero gradient. In generalimages (i.e. in images where the intensity profiles are no longerpiecewise constant but rather piecewise curved), zero-crossingsoccur at points where the image surface changes from being

.4 0.6 0.8 1

FOM(1/9)

FOM(1)

D

Baddeley

Planes set – t = 4

5 1 2 3 4 50

1

2

3


noise conditions

5 1 2 3 4 50

1

2

3

4S&G – Baddeley

noise conditions

td avg ± std

td avg ± std

etic images: (left) Examples of noisy synthetic images; (top, right) normal noise

ht) increasing noise conditions, the plots show average and standard deviation of

ARTICLE IN PRESS


concave-shaped to be convex-shaped and vice versa. Although theinflection can happen in uniform reflectance areas because of theproperties of the scene surface, it occurs particularly at reflectancetransitions. Therefore, LOG zero-crossings detection can be usedto complement C.

In order to counteract image noise when detecting GRDreflectance transitions, as well as C does, uncertainties fromR2CIU [43] are also introduced. More precisely, whenever a zero-crossing is detected, it is deemed relevant if the positive andnegative peak LOG values along the direction of detectionare larger than tG times the respective uncertainties (see Fig. 8).The uncertainties are computed using standard uncertaintypropagation rules, by which, if the output of the LOG operator isobtained from f ¼

PxDcðxÞmðxÞ, where m(x) are the LOG mask

constants, then [44]

dðf Þ ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiX

x

dðDcðxÞÞ2m2ðxÞr

: ð8Þ

Fig. 10. Edge maps for real images on the left: (2nd col.) C3E

Note that the meaning of tG is equivalent to the one of parametertC introduced in Section 6.

The new colour edge detector C3E (colour channel coupling

based edge detection) makes use of jointly the relation C and theaforementioned strategy for detecting relevant LOG zero-cross-ings. The following procedure is applied to every image location p:

EdgeMap(p) :¼NON_EDGE

IF p is C- incompatible with any of its eight neighbours THENEdgeMap(p) :¼EDGE

ELSE IF p is involved in a relevant LOG zero-crossing THENEdgeMap(p) :¼EDGE

ENDIF

In this way, for every pixel, C3E first verifies whether thereis physical evidence of a reflectance transition arising thereand only resorts to gradient information if there is no suchconfirmation.

and t=4; (3rd col.) S&G; (4th col.) M&G; (5th col.) CAN.

ARTICLE IN PRESS


9.2. Experimental results

To prove experimentally the usefulness of C3E, and byextension of colour channel coupling analysis, this sectionpresents and discusses the results of several experiments withsynthetic and real images. Furthermore, for comparison purposes,results are also provided for the photometric-invariant-basededge detection algorithm by Stokman and Gevers [45] (S&G fromnow on), the well known and most popular non-physics-basededge detector by Canny [46] (CAN from now on) and the moresophisticated non-physics-based algorithm by Meer and Geor-gescu [47] (M&G from now on).

The parameter values selected for the different edge detectorsin the different experiments performed, as well as the comparisonmetrics used, are indicated in the following:

�

Table 3Comparison between C3E, S&G, M&G and CAN with the real images of Fig. 10.

C3E S&G M&G CAN

t=3 t=4 t=5

In the case of C3E, the only relevant parameters are the numberof uncertainties to be used when looking forC�incompatibilities, tC, and the number of uncertaintiesassociated with GRD edges, tG. The same value has been usedfor both, so that only one parameter results, which will bereferenced as t. The value given to t is indicated for everyexperiment.On the other hand, the value of the standard deviation for theLOG operator used for detecting GRD edges, sLOG, was set to 1.0at all time. The size of the LOG mask was then determined as2� d3sLOGeþ1.To finish with the particularities of the experiments involvingC3E, the edge maps are shown thinned. An edge thinningstrategy based on the watershed transform has been used (see[41] for the details).
�
(a)

F1=9 0.68 0.65 0.62 0.56 0.58 0.47

F1 0.62 0.59 0.57 0.51 0.51 0.41

%D 6.21 6.15 6.20 6.78 7.79 8.32

B 0.75 0.75 0.78 0.82 1.13 1.15

Referring to S&G, it is a parameter-free algorithm which alsouses intensity uncertainties to find local thresholds, so that noparameter needs to be set up. Besides, non-maxima suppres-sion is used to thin the corresponding edge maps (as proposedby the authors).
� Regarding M&G, the default parameter values set by the
application offered in their web page are used.
(b)
�
F1=9 0.73 0.71 0.69 0.59 0.64 0.52
F1 0.55 0.54 0.53 0.42 0.46 0.37

%D 5.17 5.03 4.87 5.47 5.83 5.59

As for CAN, we have used the implementation included in theImage Processing Toolbox for Matlab, allowing the correspond-ing function to choose automatically the parameter values.

B 0.97 0.90 0.85 1.08 1.22 1.21
�
(c)

F1=9 0.69 0.70 0.72 0.67 0.64 0.54

F1 0.52 0.53 0.56 0.47 0.48 0.41

%D 12.67 12.23 11.50 14.04 12.28 11.68

B 1.69 1.64 1.56 1.73 1.78 1.66

(d)

F1=9 0.52 0.56 0.60 0.44 0.47 0.47

F1 0.38 0.42 0.46 0.28 0.33 0.33

%D 13.88 12.20 10.99 18.57 13.82 12.78

B 2.22 2.03 1.89 2.73 2.32 2.23

(e)

F1=9 0.88 0.88 0.86 0.80 0.70 0.67

F1 0.70 0.73 0.71 0.58 0.58 0.52

%D 6.59 6.05 6.05 8.80 6.02 7.58

B 0.80 0.66 0.71 1.15 0.78 0.96

(f)

F1=9 0.47 0.53 0.56 0.46 0.54 0.56

F1 0.35 0.41 0.44 0.30 0.43 0.45

%D 12.67 10.56 9.42 13.71 9.04 8.37

B 2.30 2.01 1.86 2.48 1.87 1.73

In the table, F1=9 � FOMð1=9Þ, F1 � FOMð1Þ, %D�D� 100, B� Baddeley.

Finally, up to four empirical discrepancy evaluation measuresare provided in the comparison (see Table 2): the often-citedPratt’s figure of merit (FOM) [48] with (1) a¼ 1

9 and (2) a¼ 1;(3) the discrepancy percentage (D), considered in Zhang’ssurvey as one of the best discrepancy measures [49,50]; and(4) the Baddeley measure [51], deemed the best evaluationtechnique in a recent survey [52]. All of them handle measuresof discrepancy between a reference edge map and the outputof the edge detector, although, while FOM and D yield valuesbetween 0 and 1, Baddeley’s measure belongs to [0,c].Furthermore, while Pratt’s FOM scores good edge maps withlarger values than bad edge maps, the percentage ofdiscrepancy and Baddeley’s measure both attain lower valueswith better edge maps.

9.2.1. Experiments with synthetic images

In the experiments with synthetic images, C3E and S&G werefaced against a set of 100 synthetic noisy images. Noise was addedaccording to the noise model by Healey and Kondepudy [40] withthe parameters estimated for a real camera using algorithm R2CIU[43]. Throughout the comparison, the level of noise correspondingto the real camera is referred to as normal noise conditions, whileincreasing noise conditions means the standard deviations of thenoise sources involved in the model are multiplied by a factor

k=1y5. Furthermore, the set consisted of scenes involvingspheres and planes of different reflectances (see Fig. 9(left)).

Fig. 9 provides the results of the comparison. As can beobserved, Baddeley’s measure indicates C3E edge maps are betterthan the ones by S&G, under normal noise conditions(Fig. 9(top,right)), while FOM and D measures are quite similarfor both algorithms. The source of this discrepancy is likely to layin the lack of sensitiveness to false edge positives of FOM [51]; asfor the discrepancy percentage D, it should be taken into accountthat the location of misclassified pixels is not considered by thismeasure. Furthermore, the larger standard deviation of Baddeley’smeasure for S&G indicates a bigger instability of S&G against C3E,although for some images S&G could have produced better edgemaps than C3E. As for increasing noise conditions, C3E obtainsbetter scorings than S&G and, besides, C3E shows a strongertolerance to noise, with a performance similar to the one achievedunder normal noise conditions (Fig. 9(bottom, right)).

9.2.2. Experiments with real images

This section presents experimental results for C3E, S&G, M&Gand CAN for a varied set of real scenes, including artificial andnatural objects and different surface materials for both cases. Theset of images contains ‘‘proprietary’’ images taken so as to ensurea linear relationship between image intensity and scene radiance,

ARTICLE IN PRESS


as implicitly expressed in Eq. (3) (this is required by physics-based vision algorithms in general). Additionally, images typicallyused for testing edge detection algorithms, i.e. standard images,have also been included in the comparison. Observe that they arelikely to violate the linearity constraint. The aim is, thus, toobserve the dependence of C3E on the satisfaction of such arequirement.

The comparison can be found, at the qualitative level, in Fig. 10and, quantitatively, in Table 3. The ground truth for thequantitative comparison was generated manually. As can beobserved, C3E outperforms S&G numerically in all cases,sometimes with a larger difference (Fig. 10(d)) and other timesonly very slightly (Fig. 10(c)). As for M&G and CAN, C3E also tendsto produce better edge maps, although, on some occasions, theperformance metric values are almost the same. The largerdifferences in performance between physics and non-physics-based algorithms appear, as could easily be deduced, for images

Fig. 11. Edge maps for real images on the left: (2nd col.) C3E

containing curved objects (Fig. 10(a) and (b)). Observe that M&Gand CAN produce edges related to scene curvature while C3E doesnot, due to it being based on the physics of image formation. Forexample, observe the green cup in the image in Fig. 10(a).However, in the same image, it can also be detected that C3E doesnot deal with the small details as well as M&G does, as can beobserved in the ornamentation of the upper part of the flowerpotof Fig. 10(a). Clearly, the contours are better delineated by M&Gand CAN. This behaviour is related to the fact that C3E labels asedges all the pixels involved in a reflectance transition andsometimes the thick edges that are formed are not properlyhandled during the thinning stage. The obvious solution for thesecases is, of course, to increase image resolution. Finally, note that,for some images, M&G produces better edge maps than S&G.However, this happens more predominantly with the standardimages, for which, surely, gamma correction was not turned off,what can affect S&G (and also C3E).

and t=4; (3rd col.) S&G; (4th col.) M&G; (5th col.) CAN.

ARTICLE IN PRESS


To conclude this section, Fig. 11 shows edge maps for anotherset of real images, for which no ground truth was available and,therefore, the analysis must lie at a qualitative level. As before,some of the images conform strictly to the assumed model ofimage formation, while others do not. In the case of image (a), thebest edge maps are produced by C3E and M&G, with a very similarbehaviour, although the latter misses several useful edges that theformer is able to catch. CAN in this case produces a lot of edges notrelated to material transitions. In the cases of images (b)–(d), M&Gfails to detect some edges which C3E manages to capture. As for thestandard image (e), S&G produces the best edge map if spuriousedges are not taken into account (observe that, for this set ofimages, S&G generates less spurious edges than for the previousset). Regarding image (f), the best edge map is probably the one byCAN, although, once more, the result of S&G is not bad if thespurious edges are ignored. Finally, note that small details, such asthe chain of image (d) or the eyes of Lena in image (e), are not wellsolved by C3E, as already observed in the previous set of images.

10. Conclusions

This paper has investigated colour channel coupling from aphysics-based viewpoint. It has been shown that colour channelsremain coupled in uniform reflectance areas, while the coupling isbroken in a number of ways at pixels involved in reflectancetransitions. This coupling has been expressed in the form of a setof properties which are fulfilled by all the pixels corresponding tothe same surface material (i.e. same reflectance). In other words,when any of them is not satisfied, the only possible cause of theunfulfillment reduces to a change in reflectance. The satisfactionof those properties has been studied for different instantiations ofthe image formation model, what has allowed determining, froma theoretical point of view, the constraints to be met by the sceneso that these properties can be effectively used to locate reflectancetransitions. An important aspect of the analysis proposed is that itdoes not depend on the particular expression selected for thedifferent terms of the DRM: mb and mi—geometrical terms —

and LaðlÞraðx;lÞ, LdðlÞrbðx; lÞ and LdðlÞriðx; lÞFspectral terms.Additionally, a very-easy-to-check compatibility relationshipC has been defined through the geometrical interpretation ofthe aforementioned properties. C allows determining at a verylow cost whether two pixels correspond to the same reflectanceor not.

As an example of application of the proposed colour channelcoupling analysis, a very low-cost new edge detector named C3E(colour channel coupling-based edge detection) based on C has beenpresented. The following conclusions can be drawn from thedifferent experiments performed, demonstrating the power of thecoupling analysis for locating reflectance transitions in generalscenes, i.e. scenes containing curved objects and different surfacematerials:

�
The scene constraints concerning the fulfillment of thecoupling properties have not been a serious problem forobtaining sound edge maps. � C3E does not show important differences between scenes
where objects shading is noticeable with regard to sceneswhere it is not. Three or four uncertainties are enough toproduce adequate edge maps on most occasions.
� C3E has exhibited a good tolerance to noise. � Finally, C3E has compared well to the photometric-invariant-
based edge detection algorithm by Stokman and Gevers [45](S&G) and to the recognized non-physics-based edge detectionalgorithms by Meer and Georgescu [47] (M&G) and Canny [46](CAN).

For a more detailed formalization and discussion on thecoupling properties and their implications, as well as the proofs
of the different propositions, the reader is referred to [41, Chapter8 and Appendix B].
To finish, the C source code for the C3E edge detector utilizedduring the experiments discussed in Section 9.2 can be foundpublicly available at http://dmi.uib.es/aortiz/C3E-BC50-source-code.zip.

Acknowledgements

This study has been partially supported by project CICYT-DPI2008-06548-C03-02 and FEDER funds. We thank the refereesfor their helpful comments which have undoubtedly improvedthe quality and the readability of the paper.

Appendix A. Symbols and notation

This is the list of the principal symbols used throughout thepaper, together with their default interpretation:

l
wavelength
x
a point of the scene L (generic) radiance La ambient radiance Ld radiance of a directional light source Ld
c
(scaled) radiance of a directional light source
for colour channel c: Lcd ¼ qc

0

RLdðlÞtcðlÞsðlÞdl,

where tc is the colour filter transmittance, s isthe camera responsivity and q0

c representscamera gain, exposure time, etc.

ra, rb, ri
ambient, body and interface reflectances
mb, mi
geometric factors for body and interfacereflectance
La, Lb, Li
ambient, body and interface components ofreflection
Cac, Cb

c, Cic
ambient, body and interface composite
reflectances for colour channel c

Dc
intensity level produced by the camera forcolour channel c
p, q
image locations, i.e. abbreviation of image cellcoordinates (i, j)
References

[1] H. Cheng, X. Jiang, Y. Sun, J. Wang, Color image segmentation: advances andprospects, Pattern Recognition 34 (2001) 2259–2281.

[2] L. Lucchese, S. Mitra, Color image segmentation: a state-of-the-art survey, in:Proceedings of the Indian National Science Academy, vol. 67-A-2, 2001,pp. 207–221.

[3] D. Ziou, S. Tabbone, Edge detection techniques—an overview, TechnicalReport TR-195, Departement de Math et Informatique, Universite deSherbrooke, Quebec, Canada, 1997.

[4] A. Koschan, M. Abidi, Detection and classification of edges in color images,IEEE Signal Processing Magazine (2005) 64–73.

[5] J. Ye, G. Fu, U. Poudel, High-accuracy edge detection with blurred edge model,Image and Vision Computing 23 (5) (2005) 453–467.

[6] A. Brook, R. Kimmel, N.A. Sochen, Variational restoration and edge detectionfor color images, Journal of Mathematical Imaging and Vision 18 (2003)247–268.

[7] A.N. Evans, X.U. Liu, A morphological gradient approach to color edgedetection, IEEE Transactions on Image Processing 15 (6) (2006) 1454–1463.

[8] Y. Haxhimusa, A. Ion, W.G. Kropatsch, Evaluating hierarchical graph-basedsegmentation, in: Proceedings of the IEEE International Conference on PatternRecognition, 2006, pp. 195–198.

[9] S. Konishi, A.L. Yuille, J.M. Coughlan, A statistical approach to multi-scale edgedetection, Image and Vision Computing 21 (1) (2003) 37–48.

http://dmi.uib.es/aortiz/C3E-BC50-source-code.zip

http://dmi.uib.es/aortiz/C3E-BC50-source-code.zip

ARTICLE IN PRESS


[10] G. Gilboa, Nonlinear scale space with spatially varying stopping time, IEEETransactions on Pattern Analysis and Machine Intelligence 30 (12) (2008)2175–2187.

[11] X. Munoz, J. Freixenet, X. Cufı, J. Martı, Strategies for image segmentationcombining region and boundary information, Pattern Recognition Letters 24(1–3) (2003) 375–392.

[12] A. Dupuis, P. Vasseur, Image segmentation by cue selection and integration,Image and Vision Computing 24 (10) (2006) 1053–1064.

[13] C. Zhou, B.W. Mel, Cue combination and color edge detection in naturalscenes, Journal of Vision 8 (4) (2008) 1–25.

[14] T. Gevers, Adaptive image segmentation by combining photometric invariantregion and edge information, IEEE Transactions on Pattern Analysis andMachine Intelligence 24 (6) (2002) 848–852.

[15] G.J. Klinker, S.A. Shafer, T. Kanade, A physical approach to colorimage understanding, International Journal of Computer Vision 4 (1) (1990)7–38.

[16] F. Perez, C. Koch, Toward color image segmentation in analog VLSI:algorithm and hardware, International Journal of Computer Vision 12 (1)(1994) 17–42.

[17] C.-K. Ong, T. Matsuyama, Robust color segmentation using the dichromaticreflection model, in: Proceedings of the IEEE International Conference onPattern Recognition, 1998, pp. 780–784.

[18] C.M. Onyango, J.A. Marchant, Physics-based colour image segmentation forscenes containing vegetation and soil, Image and Vision Computing 19 (8)(2001) 523–538.

[19] J.B. Park, A.C. Kak, A truncated least squares approach to the detection ofspecular highlights in color images, in: Proceedings of the IEEE InternationalConference on Robotics and Automation, 2003, pp. 1397–1403.

[20] J.-M. Geusebroek, R. Boomgaard, A. Smeulders, H. Geerts, Color invariance,IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (12)(2001) 1338–1350.

[21] E. Vazquez, J. van de Weijer, R. Baldrich, Image segmentation in the presenceof shadows and highlights, in: Proceedings of the European Conference onComputer Vision, 2008, pp. 1–14.

[22] Y. Ohta, T. Kanade, T. Sakai, Color information for region segmentation,Computer Graphics and Image Processing 13 (3) (1980) 222–241.

[23] G.E. Healey, Segmenting images using normalized color, IEEE Transactions onSystems, Man and Cybernetics 22 (1) (1992) 64–73.

[24] X. Yuan, D. Goldman, A. Moghaddamzadeh, N. Bourbakis, Segmentation ofcolour images with highlights and shadows using fuzzy-like reasoning,Pattern Analysis and Applications 4 (4) (2001) 272–282.

[25] T. Gevers, H. Stokman, Robust histogram construction from color invariantsfor object recognition, IEEE Transactions on Pattern Analysis and MachineIntelligence 26 (1) (2004) 113–117.

[26] J. Weijer, T. Gevers, J.-M. Geusebroek, Edge and corner detection byphotometric quasi-invariants, IEEE Transactions on Pattern Analysis andMachine Intelligence 27 (4) (2005) 625–630.

[27] S.A. Shafer, Using color to separate reflection components, COLOR Researchand Application 10 (4) (1985) 210–218.

[28] R. Hall, Illumination and Color in Computer Generated Imagery, Springer,Berlin, 1989.

[29] J. Birn, Digital Lighting and Rendering, New Riders Publishing, 2000.[30] H.W. Jensen, Realistic Image Synthesis using Photon Mapping, AK Peters,

2001.[31] S. Tominaga, Surface identification using the dichromatic reflection model,

IEEE Transactions on Pattern Analysis and Machine Intelligence 13 (7) (1991)658–670.

[32] H.-C. Lee, E.J. Breneman, C.P. Schulte, Modeling light reflection for computercolor vision, IEEE Transactions on Pattern Analysis and Machine Intelligence12 (4) (1990) 402–409.

[33] G.E. Healey, Using color for geometry-insensitive segmentation, Journal ofthe Optical Society of America 6 (6) (1989) 920–937.

[34] T. Hashimoto, H. Kato, K. Sato, S. Inokuchi, K. Moriwaki, Recognition ofmaterial types and interreflection using color images, Systems andComputers in Japan 23 (12) (1992) 727–735.

[35] S. Tominaga, N. Tanaka, Estimating reflection parameters from a single colorimage, IEEE Computer Graphics and Applications 20 (5) (2000) 58–66.

[36] L.B. Wolff, S.K. Nayar, M. Oren, Improved diffuse reflection modelsfor computer vision, International Journal of Computer Vision 30 (1) (1998)55–71.

[37] H. Tagare, R. de Figueiredo, A framework for the construction of reflectancemaps for machine vision, Computer Vision, Graphics, and Image Processing57 (3) (1993) 265–282.

[38] B. Phong, Illumination for computer generated pictures, Communications ofthe ACM 18 (1975) 311–317.

[39] G. Ward, Measuring and modeling anisotropic reflection, in: Proceedings ofSIGGraph, 1992, pp. 265–272.

[40] G.E. Healey, R. Kondepudy, Radiometric CCD camera calibration and noiseestimation, IEEE Transactions on Pattern Analysis and Machine Intelligence16 (3) (1994) 267–276.

[41] A. Ortiz, New segmentation and edge detection methods using physics-basedmodels of image formation, Ph.D. Thesis, University of the Balearic Islands,(Available online at /http://dmi.uib.es/aortiz/AORthesis.zipS), 2005.

[42] W.H. Tsang, P.W.M. Tsang, Suppression of false edge detection due tospecular reflection in color images, Pattern Recognition Letters 18 (2) (1997)165–171.

[43] A. Ortiz, G. Oliver, Radiometric calibration of vision cameras and intensityuncertainty estimation, Image and Vision Computing 24 (10) (2006)1137–1145 (Available online 30 June 2006).

[44] J.R. Taylor, An Introduction to Error Analysis, second ed., University ScienceBooks, Mill Valley, 1997.

[45] T. Gevers, H. Stokman, Classifying color edges in video into shadow-geometry, highlight, or material transitions, IEEE Transactions on Multimedia5 (2) (2003) 237–243.

[46] F.J. Canny, A computational approach to edge detection, IEEE Transactions onPattern Analysis and Machine Intelligence 8 (6) (1986) 679–698.

[47] P. Meer, B. Georgescu, Edge detection with embedded confidence, IEEETransactions on Pattern Analysis and Machine Intelligence 23 (12) (2001)1351–1365 (The code is publicly available at /http://www.caip.rutgers.edu/riul/research/code.htmlS).

[48] W. Pratt, Digital Image Processing, second ed., Wiley, New York, 1991.[49] Y. Zhang, A survey on evaluation methods for image segmentation, Pattern

Recognition 29 (8) (1996) 1335–1346.[50] R. Roman-Roldan, J. Gomez-Lopera, C. Atae-Allah, J. Martinez-Aroza, P. Luque-

Escamilla, A measure of quality for evaluating methods of segmentation andedge detection, Pattern Recognition 34 (5) (2001) 969–980.

[51] A. Baddeley, An error metric for binary images, in: W. Forstner, S. Ruwiedel(Eds.), Robust Computer Vision: Quality of Vision Algorithms, Wichmann,Karlsruhe, 1992, pp. 59–78.

[52] N. Fernandez-Garcia, R. Medina-Carnicer, A. Carmona-Poyato, F. Madrid-Cuevas, M. Prieto-Villegas, Characterization of empirical discrepancy evalua-tion measures, Pattern Recognition Letters 25 (1) (2004) 35–47.

About the Author—ALBERTO ORTIZ received the PhD degree in Computer Science from the University of the Balearic Islands (UIB), Spain, in 2005. He is currently amember of the Systems, Robotics and Vision (SRV) group of the UIB, and an Associate Professor at the Department of Mathematics and Computer Science of the UIB. Hispresent research interests include physics-based vision, visual tracking and visual navigation systems.

About the Author—GABRIEL OLIVER received the PhD in Computer Science from the Polytechnic University of Catalonia, Spain, in 1993. His major research interests arereal-time vision and control architectures for autonomous mobile robots. He has been the leading researcher of projects granted by the local administration and theSpanish Scientific Council (CICYT), both being related to underwater robotics. He is currently Associate Professor at the Department of Mathematics and Computer Scienceat the University of the Balearic Islands where he is the leader of the Systems, Robotics and Computer Vision (SRV) Group.

http://dmi.uib.es/&sim;aortiz/AORthesis.zip

http://www.caip.rutgers.edu/riul/research/code.html

http://www.caip.rutgers.edu/riul/research/code.html

analysis of colour channel coupling from a physics-based viewpoint: application to colour edge...

Documents