2-0 2.1. image representations, types and concepts 2.2. image digitization 2.3. digital image...

2-1

2.1. Image representations, types

and concepts

2.2. Image digitization

2.3. Digital image properties

2.4. Color images

2.5. Cameras

Ch. 2 – The image, its representation,

type, and properties

2-2

○ Function

2.1. Image representations, types and concepts

• Continuous – A, B: continuous• Discrete – A: discrete, B: continuous• Digital – A, B: discrete

Digital image f(x,y): a 2-D digital function

○ Scene g(x,y,z): a 3-D continuous function

Pixel: picture elementGray level: pixel value

Origin

: , where : domain, : rangef A B A B

○

2-3

Pin-hole camera

○ Image Formation

2-4

○ Perspective Projection

2-5

Drawbacks of pinhole camera: (i) Big pinhole - Averaging rays blurs image

Small pinhole - Diffraction effect blurs image

2 mm 1 mm 0.35 mm 0.15 mm 0.07 mm

(ii) Images are relatively dark

Solution : ＊ Pinholes Lenses Lenses: gather light sharp focus

2-6

Intensity (Gray Scale) Image

○ Types of ImagesBinary Image

2-7

Color Image

Indexed (or Palette) Color Image

2-8

Multispectral Image

2-9

X-Ray Image Gamma-Ray Image

2-10

Ultraviolet Image

Radio Image

2-11

Ultrasound Image

2-12

X-Ray Transmission Computerized Tomography Image

2-13

Range Image

2-14

Moire Image

2-15

○ Biological Visual Systems

humancattle, horsebird, chicken, duckbatsnakeinsect, fly, bee

2-16

Camera

Image array

2.2. Image Digitization

Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time).

Sensor array

2-17

2.2.1. Spatial Resolution

2-18

False contours

2.2.2. Radiometric Resolution

2-19

Non-uniform resolution -- to reduce storage while preserve as much information as possible

＊ Radiometric resolution (quantization): Busy area – fine, dense Inactive region – coarse, sparse

＊ Spatial resolution (sampling): Sharp region – fine, dense Smooth area – coarse, sparse

Non-uniform

＊ Spectral resolution (quantization):＊ Time resolution (sampling):

2-20

Uniform sampling

Non-uniform sampling

Non-Uniform Sampling

128 by 128

64 by 64

2-21

Adaptive mesh

Adapted mesh

Gradient image

Adaptive sampling

Input image

Spring modelSampled image

2-22

2.3. Digital Image Properties ○ Metric Properties

( , ) 0,D p q ( , ) ( , ),D Dp q q p( , ) ( , ) ( , )D D D p z p q q z

2 2( , ) ( ) ( )ED x s y t p q

4 ( , ) | | | |D x s y t p q

8 ( , ) max{| |,| |}D x s y t p q

Euclidean City-block

Chess-board

Quasi-Euclidean( 2 1)

( , )( 2 1) otherwise

x s y t x s y tD

x s y t

p q

( , ), ( , )x y s t p q (i) Distances

2-23

(ii) Adjacency

4-neighbors 8-neighbors

(iii) Distance Transform (Chamfering)

2-24

2-25

2-26

(i) Crack edges: each pixel has 4 crack edges

(ii) Inner, outer borders

(iii) Convex, concave

(vi) Convex hull: the smallest convex region containing the input region

○ Geometrical Properties

2-27

○ Topological properties(i) Rubber Sheet Transform (RST): preserves the contiguity of object parts and the number of holes in regions (no cut and joint). e.g., Imagine a rubber balloon with an object painted on it

(ii) Euler-Poincare Characteristic: Euler number (#regions - #holes) is invariant under RST

(iii) Topological components:

2-28

2.3.2. HistogramsHistogram

Probability distribution

2-29

Information theory -- amount of informationPhysical system -- measure of disorder Confidence theory -- amount of uncertaintyImage -- degree of redundancy

1

20

( ) ( ) log ( )L

i ii

H X k p x p x

2.3.3. Entropy

2-30

2.3.4. Visual Perception

(i) Perceptual Grouping (Organization)

2-31

2-32

(iii) Overshoot and Undershoot

(iv) Subjective Objects

(ii) Simultaneous Contrast

Mack band pattern

The human brain tricks us whenever

it can!

(v) Optical Illusion and Visual Phenomena

2-33

Are the lines parallel or not?

2-34

Coil or circle?

If something‘s rotating – go home, you need a break!

2-35

It doesn‘t move!

2-36

It doesn‘t move too!

2-37

It doesn‘t move too!

2-38

Concentrate on the cross in the middle, after a while you will notice that this moving purple dot will turn green!

Look at the cross a bit longer and you‘ll notice that all dots except the green one will disappear.

2-39

2-40

2.3.5. Image Quality

Image degradation results from (i) noise, (ii) error, (iii) distortion, (iv) blurring

where g(x,y): input image, f(x,y): degraded image H: degradation process, n(x,y): noise

Degradation model:

( , ) [ ( , )] ( , )f x y H g x y n x y

2-41

Linearity:

Additivity:

Homogeneity:

Position (or space) invariance:

1 1 2 2 1 1 2 2[ ( , ) ( , )] [ ( , )] [ ( , )]H k g x y k g x y k H g x y k H g x y

1 2 1 2[ ( , ) ( , )] [ ( , )] [ ( , )]H g x y g x y H g x y H g x y

1 2, : constantsk k

[ ( , )] [ ( , )]H kg x y kH g x y

( , ) [ ( , )]f x y H g x y

Degradation processes H

2-42

Image:

If H is linear,

If H is homogeneous,

( , ) ( , ) ( , )g x y g x y d d

( , ) [ ( , )] [ ( , ) ( , ) ]f x y H g x y H g x y d d

( , ) [ ( , ) ( , )]f x y H g x y d d

( , ) ( , ) [ ( , )]f x y g H x y d d

Let

[ ( , )]H x y

( , , , ) [ ( , )]h x y H x y

: impulse response of H

: point spread function (PSF) ( , ) ( , ) ( , , , )f x y g h x y d d Then,

Degraded image:

2-43

If H is position invariant,

( , ) ( , ) ( , )f x y g h x y d d Then,

( , ) ( , ) ( , )f x y g x y h x y

Simplified degradation model:

( , ) ( , ) ( , ) ( , )f x y g x y h x y n x y

( , , , ) [ ( , )] ( , )h x y H x y h x y

In frequency domain,

( , ) ( , ) ( , ) ( , )F u v G u v H u v N u v

2.3.6. Noise

-- Originating from image acquisition, digitization,

or transmission

○ White noise: the noise whose Fourier spectrum is constant

( , ) ( , ) ( , )f x y g x y n x y

( , ) ( , ) ( , )f x y g x y n x y

e.g., Television raster, film material

e.g., Physical processes

FT

( , )n x y

( , )n x y ( , )N u v2-44

Additive:

Multiplicative:

2-45

○ Random noise:

Objectives: i) examine the performance of a developed algorithm ii) help to remove noise from an image

(i) Salt-and-pepper (impulse) noise

for

( ) for

0 otherwise

a

b

P x a

p x P x b

2-46

(ii) Uniform noise

1if

( ) ,0 otherwise

a x bp x b a

22 ( )

[ ] , [ ]2 12

a b b aE x Var x

2 2 2

1 1[ ] ( )

1 1 1 1 ( )

2 2 2

b b

a a

E x xp x dx x dx xdxb a b a

b a bx b a

ab a b a

2-47

2 2

2 2

2 2

2 2 2 2

2 2 2

[ ] [( [ ]) ]

[ 2 [ ] ( [ ]) ]

[ ] 2 [ ] [ ] ( [ ])

[ ] ( [ ]) ( ) ( )2

1 ( ) ( ) /12

2

b

a

Var x E x E x

E x xE x E x

E x E x E x E x

a bE x E x x p x dx

a bx dx a b

b a

2-48

(iii) Rayleigh noise2( ) /2

( ) for ( )

0 for

x a bx a e x ap x b

x a

2

/ 4

(4 )

4

a b

b

(iv) Erlang (gamma) noise1

for 0( ) ( 1)!

0 for 0

b baxa x

e xp x b

x

2 2

/

/

b a

b a

2-49

(v) Exponential noise

for 0( )

0 for 0

axae xp x

x

2 2

1/

1/

a

a

(vi) Gaussian noise2

2

( )

21( ) ,

2

x

p x e

Method 1:

2-50

2-51

Method 2:

pdf

cdf

inverse

-20 -10 0 10 200

0.02

0.04

0.06

0.08normal distribution pdf, =0, =5

-20 -10 0 10 200

0.2

0.4

0.6

0.8

1normal distribution CDF, =0, =5

2

2

2exp

2

1),;(

x

xf

1( ) 1

2 2

xF x erf

1 1( ) 2 2 1 , ( )F p erf p p F x

若 p 為均勻分佈，則 F-1(p) 為常態分佈

Method3:

Definition:

By Taylor expansion:

Error Function Erf(x)

2-54

◎ Estimation of noise parameters

Gaussian Rayleigh Uniform

Image segmentation (i) manually or (ii) automatically

2-55

Steps: 1. Segment the image 2. Choose a uniform image region 3. Compute its histogram 4. Compute the mean and variance of the histogram

5. Determine the probability distribution from the shape of the histogram 6. Estimate the parameters of the probability distribution of the histogram

( )ih x

2 2( ), ( ) ( )i i i ix h x x h x

( )ih x

2,

2-56

Example: Rayleigh noise:

24 (4 ),b

/ 4, / 4a b a b

From 2 (4 ) / 4,b 24 /(4 )b

From

24

4(4 ) 4a

2( ) /2( ) for

( )0 for

x a bx a e x ap x b

x a

2-57

1. Signal-to-Noise Ratio (SNR)2 2

( , ) ( , )

( , ), ( , ),x y x y

E n x y F f x y /SNR F E

1010logdBSNR SNR (logarithmic scale)

2. Peak Signal-to-Noise Ratio (PSNR)

Measures of Image Quality

where I: input image, K: output image

2.4. Color Images2.4.1. Physics of Color

-- The world is colorless. Human color perception is carried out by the nervous system, which interprets differently to distinct electromagnetic wavelengths.

-- Electromagnetic spectrum

2-58

2-59

-- Observed color of objects Surface reflection rebounds incoming energy, whose spectrum remains the same as that of the light source Body reflection: The incoming energy diffuses into surface and reflects from its internal pigment

-- Visible spectral colors

2-60

2.4.2. Human Color Perception

Human Eye FOV: Field Of View (FOV)

width × height = 160 deg × 135 deg

2-61

Eyeball

Camera

2-62

Retina is composed of photoreceptors

retina

photoreceptors

neural fiber

light

2 types of photo-receptors: rods and cones

2-63

Rods : sensitive to intensity

Cones: sensitive to color

Types of cones: L(red), M(green), S(blue)

2-64

○ RGB Color Space

1 2 3( ) ( ) ( ) ( )C w R w G w B

1 2 3( ), ( ), ( )w w w : color matching functions may be negative

2.4.3. Color spaces – in which colors are made up

R, G, B: primary colors, real

of different amounts of primary colors

2-65

○ CIE XYZ Color Space

CIE (Commission Internationale d’Eclairage):

an organization responsible for color standard

X,Y,Z: not real primaries, Y: luminance

。 The volume of visible

colors in CIE XYZ space

is a cone

Their color matching functions are positive everywhere

2-66

。 The relationship between RGB and XYZ

0.431 0.342 0.178

0.222 0.707 0.071

0.02 0.130 0.939

X R

Y G

Z B

3.063 1.393 0.476

0.969 1.876 0.042

0.068 0.229 1.069

R X

G Y

B Z

2-67

○ CIE xy Color Space -- The section intersects

the XYZ space with the plane 1X Y Z

, , X Y Z

x y zX Y Z X Y Z X Y Z

Since x + y + z = 1, a color can be specifiedby x and y alone.

Chromaticity Diagram

2-68

。 Chromaticity Diagram(i) Spectral locus: the curved boundary along which colors of single wavelengths are viewed

(ii) Hue changes as one moves around the spectral locus

(iv) Colors that lie farther away from the neutral point are more saturated

(iii) Neutral point: the color whose weights are equal for all three primaries

2-69

。 RGB Gamut – The colors correspond to

positive matching values

2-70

Gamut which can be displayed by display devices

CRT monitor

Color film

Color printer

2-71

○ HSV (Hue, Saturation, Value) Color Space

Hue: varies from red greenSaturation: varies from red pinkValue: varies from black white

2-72

○ (i) RGB HSV

If R = V, then

If G = V, then

If B = V, then

If H ends up being negative, add 1

If (R,G,B) = (0,0,0), then (H,S,V) = (0,0,0)

max{ , , },

min{ , , }, /

V R G B

V R G B S V

1

6

G BH

1

26

B RH

1

46

R GH

2-73

。 Example: (R, G, B) = (0.2, 0.4, 0.6)

max{ , , } max{0.2,0.4,0.6} 0.6

min{ , , } 0.6 min{0.2,0.4,0.6} 0.4

/ 0.4 / 0.6 0.6667

V R G B

V R G B

S V

Since 0.6,

1 1 0.2 0.44 4 0.5833

6 6 0.4

B V

R GH

2-74

(ii) HSV RGB

6

6

(1 )

(1 )

[1 (1 )]

H H

F H H

P V S

Q V SF

T V S F

0

1

2

3

4

5

H R G B

V T P

Q V P

P V T

P Q V

T P V

V P Q

2-75

。 Example: (H, S, V) = (0.5833, 0.6667, 0.6)

6 6(0.5833) 3

6 6(0.5833) 3 0.5

(1 ) 0.6(1 0.6667) 0.2

(1 ) 0.6(1 0.6667 0.5) 0.4

[1 (1 )] 0.6[1 0.6667

(1 0.5)] 0.4

H H

F H H

P V S

Q V SF

T V S F

Since 3,

( , , ) ( , , ) (0.2, 0.4, 0.6)

H

R G B P Q V

2-76

○ YIQ Color Space – Used for TV and video

0.299 0.587 0.114

0.596 0.274 0.322

0.211 0.523 0.312

Y R

I G

Q B

1.0 0.956 0.621

1.0 0.272 0.647

1.0 1.106 1.703

R Y

G I

B Q

Y : luminance information I, Q : color information

2-77

○ Uniform Color Space

。 Macadam ellipse -- the noticeable difference of a color forms the boundary of the color in a color space and can be fitted with an ellipse

。 Noticeable difference – the difference when modifying a color until one can tell it has changed

2-78

The color difference in CIE xy space is poor:

The distance in the space is a guide to color difference

(a) the ellipses at the top are larger than those at the bottom

(b) the ellipses rotate as they move up

2-79

○ CIE u’v’ Color Space – a more uniform

space than the CIE xy color space 4 9

( , ) ( , )15 3 15 3

X Yu v

X Y Z X Y Z

2-80

○ CIE L*a*b* Color Space

– another substantial uniform space

* 1/3

* 1/3 1/3

* 1/3 1/3

116( ) 16

500[( ) ( ) ]

200[( ) ( ) ]

n

n n

n n

YL

Y

X Ya

X Y

Y Zb

Y Z

where , ,n n nX Y Z : the XYZ coordinates of a reference white patch

2-81

○ Summary

○ Secondary colors (primaries of pigments):

Magenta (purple) = R + B = W - G

Cyan = G + B = W - R

Yellow = R + G = W - B

2.4.5. Color and Lightness ConstanciesSurfaces have different colors and brightness when viewed under different lights

2-82

2-83

Color constancy – Intensity-independent description of color

Lightness constancy – Color-independent description of brightness

2-84

2.5. Cameras2.5.1. Photosensitive SensorsTypes: CCD (charge-coupled devices) CMOS (complementary metal oxide semiconductor)

CCD uses a rectangular

grid of electron collection

sites laid over a silicon

wafer.

○ Color CCD: uses the same chip as black and white CCD except that successive rows or columns are made sensitive to R, G or B light using a filter coating that blocks the complementary light.

Bayer pattern: a filter pattern including a mosaics of 2 by 2 blocks form by two G, one R, and one B receptors

2-85

2-86

• Perspective Projection

Summary

• Biological images in addition to engineering images

• Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time).

2-87

Euclidean City-block

Chess-board

2 2( , ) ( ) ( )ED x s y t p q

4 ( , ) | | | |D x s y t p q

8 ( , ) max{| |,| |}D x s y t p q

( , ), ( , )x y s t p q• Metric functions

• Non-uniform resolution -- to reduce storage while preserve as much information as possible

2-88

• Image degradation model:

( , ) [ ( , )] ( , )f x y H g x y n x y

• Chamfer distance

Simplified model:

( , ) ( , ) ( , ) ( , )f x y g x y h x y n x y

In frequency domain,

( , ) ( , ) ( , ) ( , )F u v G u v H u v N u v

• Bayer pattern:

2-89

• Color spaces:

• Color and brightness constancies:

• Noise generation and estimation