Chapter 6: C l P Color Image Processing

Lecturer: Jianbing ShenLecturer Jianbing ShenEmail : shenjianbing@bit.edu.cn

Office room : 841http://cs bit edu cn/shenjianbinghttp://cs.bit.edu.cn/shenjianbing

It is only after years of preparation that the young artist should touch color- not color used descriptively that is but ascolor used descriptively, that is, but as a means of personal expression.

– Henri Matisse

For along time I limited myself to one g ycolor – as a form of discipline.

Pablo Picasso– Pablo Picasso

6.1 Color Fundamentals Two principal motivating factors for

i l i i iusing color in image processing: Color is a powerful descriptor that p p

often simplifies object identification and extraction from a scene.

Humans can discern thousands of color shades and intensities but just only shades and intensities, but just only two dozen of shades of grays.

3

Color image processing includes Color image processing includes three major areasj

1. Full-color: the images in question typically i d i h f ll l h are acquired with a full-color sensor, such as a

color TV camera or scanner. 2. false-color: processing intentionally the

natural color images or multi-spectral sensor g pimages into false color images.

3. pesudo-color processing: assigning a color to . p u r pr ng a gn ng a r a particular monochrome intensity or range of intensities.

4

.

Six bands in color spectrum: Violet blue green yellow orange and red each Violet, blue, green, yellow, orange and red, each

colors blends smoothly into next.

The colors of object are often determined by j ythe nature of the light reflected from the object, for example, green objects reflect light with wavelengths primarily in the 500 to 700 nm with wavelengths primarily in the 500 to 700 nm range.

The electromagnetic spectrum of chromatic light ranges from about 400 to 700nm three basic features used to describe theabout 400 to 700nm, three basic features used to describe the quality of a chromatic light: radiance, luminance, and brightness.Radiance: the amount of energy that flows from the light source, measured in watts (W);Luminance: a measure of the amount of energy an observerLuminance: a measure of the amount of energy an observer perceives from a light source, measured in lumens (lm);Brightness: a subjective descriptor practically impossible toBrightness: a subjective descriptor practically impossible to measure.

6.1 Color Fundamentals

7

Three primary colors designated by visual characteristicsRed (R), green (G) and blue (B), the wavelength specified by CIE is 700nm, 546.1nm and 435.8nm respectively.Absorption experiment of color light of cones in human eye:Absorption experiment of color light of cones in human eye:

Blue Green Red

Blu

Pu Blu

Blu

Gre

Yell

Yel

Yel

Red

Red

400 450 500 550 600 650 700nm

uish purple

rplish blue

ue

ue green

een

lowish gree

low

low orange

dish orang

d

e en

e ge

6.1 Color Fundamentals It is important to keep in mind that

having three specific primary color wavelengths for the purpose of wavelengths for the purpose of standardization does not mean that these three fixed RGB components these three fixed RGB components acting alone can generate all spectrum

lcolors.

9

Secondary colors (the i l f i t

Additive nature

primary color of pigments or colorants) : magenta (red + bl ) ( bl ) blue), cyan (green + blue), and yellow (red + green)

Mixing the three primaries, or a secondary with its opposite primary will produces white light.

Mixing the three pigment primaries, or a secondary p ywith its opposite pigment primary will produces blacksubtractive nature

10

p y plight.subtractive nature

brightness, hue, saturationThree characteristics generally used to distinguish one color from

h b i h h d i

g

another are brightness, hue, and saturation.Brightness embodies the achromatic notion of intensity.Hue is an attribute associated with the dominant wavelength in aHue is an attribute associated with the dominant wavelength in a mixture of light waves and represents dominant color as perceived by an observer.Saturation refers to the relative purity or the mount of white light (inversely proportional) mixed with a hue.H d t ti t k t th ll d h ti itHue and saturation taken together are called chromaticity.Tristimulus value the amounts of red, green and blue needed to form any particular color denoted by X Y and Z respectively

11

form any particular color, denoted by X, Y and Z, respectively.

Approaches for specifying colorsx (red) : 62%

( ) 25%Approaches for specifying colors

1. Trichromatic

y (green) : 25%z (blue): z=1-x-y=13%

Xx

coefficients

xX Y Z

YyX Y Z

yX Y Z

ZzX Y Z

:1

X Y Zobviouslyx y z

1x y z

2. CIE chromaticity diagram ( h l iti(shows color composition as a function of x (red) and y (green) :

Color gamut of RGB monitor and color printing deviceColor gamut of RGB monitor and color printing device

The color gamut of RGB monitor

The color gamut of high g gquality color printing devices

From triangle with three fixed colors as vertices we can observe that not all colors can be obtained with th fi d i i

13

three fixed primaries.

6.2 Color Models Purpose

To facilitate the specification of colors to To facilitate the specification of colors, to quantitatively describe colors, usually can be classified as:

RGB model Most commonly used modely

CMY (CMYK: cyan, magenta, yellow and black) model Color printing

HSI model

14

Reflects the way humans perceive colors

Hardware-oriented: RGB: for color monitors and a broad class of RGB: for color monitors and a broad class of

color video cameras; CMY (or CMYK: cyan magenta yellow and black): CMY (or CMYK: cyan, magenta, yellow and black):

color printing;Application-oriented:Application oriented: HSI (hue, saturation and intensity): corresponds

closely with the way humans describe and closely with the way humans describe and interpret color. It also decouples the color and gray-scale information, making it suitable for

15

g y f m , m g fmany of the gray-scale technique.

6.2.1 RGB color model

Bit depth (or pixel depth): the number of bits used to represent each pixel in RGB space.

16

Full color: 24-bit RGB color image. The total of colors is (28)3= 16,777,216.

Generating the RGB image of the cross sectional color planethe cross-sectional color plane (127, G, B)

Three hidden surface planes in the left color cube

17

Organized in descending RGB values.

For most graphics images used for Internet applications, a set of g p g pp216 colors has been selected to represent "safe colors" which should be reliably displayed on computer monitors.

It is a hollow cube, that is to say, it has valid colors only on the surface planescolors only on the surface planes.

19

Color Models -- CMY and Color Models -- CMY and CMYK Models

An RGB to CMY conversion An RGB to CMY conversion

RC 1

BGR

YMC

111

CMYK cyan magenta yellow and black

BY 1

CMYK – cyan, magenta, yellow, and black. This model is used in connection with generating hardcopy output

20

generating hardcopy output.

HSI (hue saturation and HSI (hue, saturation, and intensity) Modely

HSI model decouples the intensity component from the color-carrying information (hue and saturation). It is an ideal tool for color description that are natural and i i i hintuitive to humans. H: Hue

l b h d b l A color attribute that describes a pure color S: Saturation

l l A measure of degree to which a pure color is diluted by white light

I: Intensity21

I: Intensity

Relationships Between RGB and Relationships Between RGB and HSI Models

22

Relationships Between RGB and Relationships Between RGB and HSI Models

Several other formsSeveral other forms of HSI Models

23

24Several other forms of HSI Models

25

Manipulating HSI

d i

components imagesHue component and change to 0 the pixels corresponding to the

saturation component and reduce by half the saturation of the

blue and green region in Fig.6.16(b)

cyan region in Fig.6.16(c)

Intensity component and reduce by half the

The modified result imagea d educe by a e

intensity of the central white region in Fig 6 16(d)

image

Fig.6.16(d)

Conversions From RGB To HSI if GB

H

)]()[(2/1

if 360

1 BRGR

GBH

)])(()[()]()[(2/1cos 2/12

1

BGBRGRBRGR

)],,[min(31 BGRBGR

S

)(31 BGRI

R, G, B , S, I values have been normalized to the range [0, 1]. H can be normalized to [0 1] by dividing by 360° and the angle

27

[0, 1] by dividing by 360 , and the angle theta is measured with respect to the red axis of the HSI space.

Conversions From HSI To RGB RG sector (0<=H<120°)

]cos1[

)1(HSIR

SIB

)(3

])60cos(

1[

BRIGH

IR o

GB sector (120°<=H<240°) ]

)60(cos1[

)1(

HHSIG

SIR

o

(120 <=H<240 ) H=H-120°

)1( SIG

)(3

])60cos(

[

GRIBHo

BR sector ( ° °)

])60cos(

cos1[

)1(

HHSIB

SIG

o

28

(240°<=H<360°) H=H-240°

)(3)60cos(

BGIRH

Pseudocolor Image Processingg g Concerns the assignment of colors to gray

valuesvalues Purpose: Better human visualization and

interpretationinterpretation Why: Humans can discern thousands of color shades

compared to only two dozen or so shades of grayp y g y Intensity slicing Gray level to color transformationsGray level to color transformations (generate color map based HSI model and

covert it to RGB and display the pseudocolor

29

covert t to RGB and d splay the pseudocolor image)

Intensity Slicingy g Treat an image as a 3D function

30

Pseudocolor Image Processing-m g gIntensity Slicing and color coding

The pseudo- (or false-) colors used to l i l l i d t colorize a gray level image do not

represent the original, true colors if h i i ll l i O there were originally a color image. One

of the principal motivations for using f l l h f h h false-color is the fact that humans can discern thousands of color shades and intensities, compared to only two dozen or so shaded of gray.

31

g y

Assign one color to the brightest gray-level and

th t ll thanother to all other gray levels.

Pseudocolor Image Processing for the visual inspectionPseudocolor Image Processing for the visual inspection of weld crack, the human error rates would be lower.

Pseudocolor Image Processing Pseudocolor Image Processing Intensity Slicing y g

33

Pseudocolor Image Processing Pseudocolor Image Processing for rainfall detection

Each pixel in these images represents a physical land p p yarea whose size depends on the resolution of the sensors.

34

Gray Level to Color TransformationsyGray level to color transformation for achieving a wider range of pseudocolor enhancement results.

Through three independent transformations

range of pseudocolor enhancement results.

35

X-ray scanning imagesTwo types of transformation functions y g gfunctions

36

Gray level to color transformation used y f min multi-spectral image processing

It is often of interest to combine several monochrome images (which are formed by different spectrum band) into a single color composite.

(a) (b)

(a)-(d) show four spectral satellite images: red green

(a) (b)

satellite images: red, green, blue, and near-infrared. (e) is the full-color image gobtained by combining the first three images into a RBG image. (f) is the combination of (a), (b) and (e) Notation: near infrared

(c) (d)

(e). Notation: near-infrared band is strongly responsive to the biomass components pof a scene.

P Ri(e) (f)

Potomac River

The type in Fig 6.27(f) processing j ill d i i f l ijust illustrated is quite powerful in helping visualize events of interest in complex images especiallyin complex images, especially when those events are beyond our normal sensing capabilities.g p

Red: newly ejected materialsYellow: elder sulfur deposits.

39

Full-Color Image Processingg g

Color Transformations Color Transformations Color Complements Tone Correction

Gray level transformations Image negatives

Color Correction Histogram Processing

Log, power law,… Histogram equalization

Spatial processing Spatial processing

For individual color component

Spatial processing Spatial filtering, edge

detection, segmentation…component Color vector processing

, g

40

Full-Color Image Processing

Two processing methods:

g g

Two processing methods: (1) process each channel (or color component)

separately, as if the color image were three gray scale images;

(2) work with color pixels directly represented as a vector and process all p pchannels with each pixel.

41

Full-Color Image Processing

In order for per-color-component and

g g

In order for per-color-component and vector-based processing to be equivalent, two conditions have to be satisfiedtwo conditions have to be satisfied The process has to be applicable to both

d lvectors and scalars. The operation on each component of a vector p p

must be independent of the other components.

42

Full-Color Image Processingg g

),(),( yxRyxcR

)(),(

)(),(

)( SyxHyxcH

),(),(

),(),(),(

yxByxG

yxcyxcyx

B

Gc

),(),(

),(),(),(

yxIyxS

yxcyxcyx

I

Sc

43

Color Transformations As apposed to gray-level transformation General expression

nirrrTs nii ,...,2 ,1 ),,...,,( 21 Selection of color model and color

component(s) to work on

nii 21

component(s) to work on For modifying intensity, HSI model is

preferredp For HSI model, only one component (I) is to be

modified F RGB d l ll t t b difi d

45

For RGB model, all components are to be modified

Color TransformationsRGB<->HSI<->CMYK

Color TransformationsRGB< >HSI< >CMYK

46

Color Complements Transferp

To the human visual system colors have To the human visual system, colors have complements.Complements are basically given by subtracting Complements are basically given by subtracting one color from white, or by changing a hue by 180 degrees 180 degrees.

Complements are useful for enhancing detail h i b dd d i d k i f l that is embedded in dark regions of a color

image.

47

Color Complements Transferp Implementation using RGB model

BGRirrrrTs )1()(

Implementation using HSI model

BGRirrrrTs iBGRii ,, ),1(),,(

)(S ,1mod)5.0(),,( HISHH rrrrHs

,1),,( IISHI rrrrIs

,),,( SISHS rrrrSs

48

Color Slicing Color slicing: highlighting a specific range of colors in

an image Two methods: cube- and sphere-based

g

an image. Two methods: cube and sphere based methods.

50

Tone and Color CorrectionsTone and Color CorrectionsTone Corrections

Device-independent color model CIE L* a * b * model (CIELAB): colorimetric, perceptually uniform, and device independent.

L* represents brightness a* for red min s green and b* for– L* represents brightness, a* for red minus green, and b* for green minus blue;– CIE model is useful in both image manipulation (tone andCIE model is useful in both image manipulation (tone and contrast editing) and image compression applications.

Tonal range, also called its key type, refers to its general distribution of color intensities. The correction of tonal and color imbalances.

51

l The highlight and shadow area were lightened and darkened respectivelyTonal Correctionlightened and darkened, respectively

It corresponds to th h t f the enhancement of gray-level images

i l l using gray level transformations.

Modifying image tones is done by adjusting the image’s brightness

52

and contrast.

Color Correction

Used to correct any ycolor imbalance

Done experimentally by adjusting each of by adjusting each of the color componentsp Analyzing a known

color using a color spectrometerp

Visually assess skin color

53

Histogram processingg p g

Histogram processing is performed on H stogram process ng s performed on the intensity component only (HSI model)model) O.W. the colors are altered

Histogram processing is often followed by saturation adjustment (increasing y j m ( gsaturation).

54

Spatial Filtering of Color Imagesp g g

Linear filtering Linear filtering technique can be applied on individual applied on individual color component or all of the color components at a time.

Usually is performed on all of the three RGB components. Smoothing by neighborhood averaging can be carried out on a per-color-plane

basis

56

basis. Result is similar to performing on the intensity component only using HSI

model.

Color Image Smoothing g gand Sharpening

57

Color Image Smoothing and SharpeningColor Image Smoothing and Sharpening

58

Color Image Smoothing and SharpeningColor Image Smoothing and Sharpening

Laplacian method: see Section 3.7.2 (Pages 128-131)

59

6.8 Color Edge Detectiong Many edge detection operators are derived from

approximating gradients like Sobel and Prewitt approximating gradients like Sobel and Prewitt operators.

One way to perform color edge detection is to apply One way to perform color edge detection is to apply edge detection operators on individual color plane and combine the results.

Alternatively, we can generalize the idea of gradient to vectors and derive similar operation for vector pvalued functions (color image)

In general, the edge detection is more stronger and l h l

60

complete in the latter case

6.8 color edge detectiong Edge detection is a important tool for image

i Th di di d i segmentation. The gradient discussed in Section 3.7.3 is a method of edge detection f l i b b d fi d f of gray-scale image, but can’t be defined for

vector quantities. Thus, computing the di i di id l i d h i gradient on individual image and then using

the result to form a color image will lead to Ob i l d erroneous. Obviously we need a new

definition of the gradient applicable to t titi

61

vector quantities.

Generalization of Gradient to Generalization of Gradient to Vector Valued Functions

Let r, g, b be unit vectors along the Let r, g, b be un t vectors along the R, G and B axis of RGB color space, and define the following two vectors: and define the following two vectors:

bBGR bgruxB

xG

xR

bgrvyB

yG

yR

62

yyy

Generalization of Gradient to Generalization of Gradient to Vector Valued Functions

Let the quantities gxx, gyy, and gxy be d f d f h d d f

yy ydefined in terms of the dot product of these vectors, as follows (Di Zenzo[1986]) :

BGRg T

uuuu

222

BGR

xxxg

T

xx

uuuu

222

BBGGRRyB

yG

yRg T

yy

vvvv

63yB

xB

yG

xG

yR

xRg T

xy

vuvu

Generalization of Gradient to Generalization of Gradient to Vector Valued Functions

It can be shown that the direction of maximum rate of change of c(x y)=(R(x y) G(x y) B(x y)) is given by the change of c(x,y)=(R(x,y), G(x,y), B(x,y)) is given by the angle

]2

[tan1 1 xyg

and the value of the rate of change at (x y) in the

][tan2 yyxx gg

and the value of the rate of change at (x, y) in the direction of Ө is given by

2/1 2/1

)]2sin(2)2cos()()[(21)(

xyyyxxyyxx gggggF

64

Color Edge Detectiong

65

66

Color Segmentationg Color segmentation can be done in either

RGB model or HSI modelRGB model or HSI model The goal is to segment regions with similar

colors defined by the usercolors defined by the user. The general approach is the following

Choose a prototypical color c with components Choose a prototypical color c with components (c1,…, cn )

Define a distance measure D between two colors d ba and b.

If D(c, a)<T, keep the original color, o.w, assign a nonprominent neutral color T is a predefined

67

nonprominent neutral color. T is a predefined threshold.

Color Segmentation in Color Segmentation in RGB Vector Spacep

The prototypical color c can be determined p ypby a set of sample color points representative of a color of interest through the mean gvector (m) of the samples.

If the Euclidean distance is used, we have 2/1)]()[()( TD

2/1222

2/1

])()()[(

)]()[(),(

BBGGRR

T

mzmzmz

D

mzmzmzmz

68

Fig6.44b Yield much more t lt d taccurate results compared to

HIS (correspond much more closely with what we wouldclosely with what we would define as “reddish”

Color Segmentation in Color Segmentation in RGB Vector Space

Th l h f diff The poor results shown were a consequence of different variances of the R, G, B, values of the sample points.

The Euclidean distance can be generalized as The Euclidean distance can be generalized as (Mahalanobis distance)

)]()[(),( 2/11 mzCmzmz D C is a symmetric matrix that controls the shape of the

enclosed region, z and m are vectors.

)]()[(),(

C is often chosen as the covariance matrix of the samples that are representatives of the color to be segmented segmented.

The locus of points such that D(z, m)<T describes a solid 3D elliptical body with the property that its p y p p yprincipal axes are oriented in the direction of maximum data spread (variance).

Color Segmentation in Color Segmentation in HSI Vector Space p

If HSI model is used, the definition of D should involve H component only.

Saturation is often used as a masking Saturation is often used as a masking image to isolate further regions of int st in th hu iminterest in the hue image Pixels with the same hue but very low

saturation appear to be quite different in color (whitish)

71

The noise in color images

Different noise levels are more likely to ybe caused by differences in the relative strength of illumination available to strength of illumination available to each if the color channels CCD sensors are noisier at low level of illuminationare noisier at low level of illumination

Intensity component is slightly smoother than any of the three noisy images (because intensity image is the

73

images (because intensity image is the average of the RGB images)

74

The noise in color images

Fig. 6.48(d)

75

Noise in Color images

The noise of one RGB h l b channel can be

spread to all HSI channel HSI channel when converting the RBG m d l t RBG model to HSI model.

Color image compressiong p

The data that are the object of any compression are the components of compression are the components of each color pixelC i i th f d i Compression is the process of reducing and eliminating redundant and/or girrelevant data

77

Color image compression

Using JPEG2000,

g p

Using JPEG2000,The compression is 230:1

78

Summaryy color fundamentalsf m Color models

P d l d f ll l i Pseudo-color and full-color processing Color vector space Color vector space Color edge detection

79