Spatial Domain Processing and Image Enhancement

Lecture 4, Feb 16th, 2009

Lexing Xie

EE4830 Digital Image Processing http://www.ee.columbia.edu/~xlx/ee4830/

thanks to Shahram Ebadollahi and Min Wu for slides and materials

-2-

announcements

Today HW1 dueHW2 out

-3-

recap

-4-

why spatial processing

http://flickr.com/photos/alliwalk/3284897415/

?

-5-

roadmap for today

)((.) fTgf NTΝ=⎯⎯→⎯

NyMxyxf ≤≤≤≤ 1,1,),( NyMxyxg ≤≤≤≤ 1,1,),(

(.)NT : Spatial operator defined on a neighborhood N of a given pixel

),(0 yxN ),(4 yxN ),(8 yxN

point processing mask/kernel processing

Application

Method

-6-

outline

What and whySpatial domain processing for image enhancement

Intensity TransformationSpatial Filtering

-7-

intensity transformation / point operationMap a given gray or color level u to a new level v

Memory-less, direction-less operationoutput at (x, y) only depend on the input intensity at the same pointPixels of the same intensity gets the same transformation

Does not bring in new information, may cause loss of informationBut can improve visual appearance or make features easier to detect

input gray level u

outp

ut g

ray

leve

l

v

-8-

intensity transformation / point operation

Two examples we already saw

Color space transformationScalar quantization

-9-

image negatives

the appearance of photographic negatives

Enhance white or gray detail on dark regions, esp. when black areas are dominant in size

-10-

basic intensity transform functions

monotonic, reversiblecompress or stretch certain range of gray-levels

-11-

log transform

im = imread(‘lena.png’)a = abs(fftshift(fft2(double(im))));c = log(1+double(im)); c = range_normalize(c);b = log(1+a); b=b/max(b(:));

lena

FFT(lena)stretch:

u ∈ [0, .5] v ∈ [0, .59]

compress:u ∈ [.5, 1] v ∈ [.59, 1]

-12-

-13-

power-law transformation

power-law response functions in practice

CRT Intensity-to-voltage function hasγ ≈ 1.8~2.5Camera capturing distortion with γc = 1.0-1.7 Similar device curves in scanners, printers, …

power-law transformations are also useful for general purpose contrast manipulation

-14-

gamma correctionmake linear input appear linear on displaysmethod: calibration pattern + interactive adjustment

example calibration chart

-15-

effect of gamma on consumer photosL0

2.2 L01/2.2L0

-16-

what gamma to use?

γ >1

γ <1 ?

-17-

more intensity transformlog, gamma … closed-form functions on [0,1]can be more flexible

contrast stretching

-18-

intensity slicing

-19-

image bit-planes

-20-

slicing bitplanesDepend on relative importance of bitsHow much to slice depend on image contentUseful in image compression, e.g. JPEG2000

-21-

outline

What and whyImage enhancementSpatial domain processing

Intensity TransformationIntensity transformation functions (negative, log, gamma), intensity and bit-place slicing, contrast stretchingHistograms: equalization, matching, local processing

Spatial FilteringFiltering basics, smoothing filters, sharpening filters, unsharp masking, laplacian

Combining spatial operations

-22-

gray-level image histogramRepresents the relative frequency of occurrence of the various gray levels in the image

For each gray level, count the number of pixels having that levelCan group nearby levels to form a big bin & count #pixels in it

-23-

interpretations of histogramif pixel values are i.i.d random variables histogram is an estimate of the probability distribution of the r.v.“unbalanced” histograms do not fully utilize the dynamic range

Low contrast image: narrow luminance rangeUnder-exposed image: concentrating on the dark sideOver-exposed image: concentrating on the bright side

“balanced” histogram gives more pleasant look and reveals rich details

-24-

contrast stretching

α

)(αβ T=

2α1α

2β

1β0 L-1

1s

2s

3sL-1

Stretch the over-concentrated gray-levelsPiece-wise linear function, where the slope in the stretching region is greater than 1.

-25-

… in practice

intuition about a “good” image:a “uniform” histogram spanning a large variety of gray tones

can we figure out a stretching function automatically?

-26-

histogram equalizationgoal: map the each luminance level to a new value such that the output image has approximately uniform distribution of gray levelstwo desired properties

monotonic (non-decreasing) function: no value reversals[0,1] [0,1] : the output range being the same as the input range

pdf

cdf

o

1

1o

1

1

-27-

histogram equalizationmake

show that v follows uniform distribution, i.e.

o

1

1

-28-

implementing histogram equalization

1-L ..., 0, ifor )(

)()( 1

0

==

∑−

=

L

ii

iiu

xn

xnxp

∑=

−=u

xiu

i

xpLv0

)()1(

)(' vroundv =

∑≤

=ux

iui

xpv )( Rounding or Uniform

quantization

u v v’

pu(xi)

Only depend on the input image histogramFast to implementFor u in discrete prob. distribution, the output vwill be approximately uniform

compute histogram

equalize

round the output

∑=

−=

u

xi

i

xnMNLv

0)(1or

-29-

a toy example ∑=

−=u

xiu

i

xpLv0

)()1(

-30-

a toy example

1.333.084.555.676.236.656.867.00

13566777

1.331.751.471.020.560.420.210.14

-31-

histogram equalization example

-32-

contrast-stretching vs. histogram equalization

function formreversible? loss of information?input/output?automatic/interactive?

input gray level u

outp

ut g

ray

leve

lv

a bo

α

β

γ

-33-

histogram matchingHistogram matching/specification

Want output v with specified p.d.f. pV(v)Use a uniformly distributed random vairableW as an intermediate step

W = FU(u) = FV(v) V = F-1V (FU(u) )

Approximation in the intermediate step needed for discrete r.v.

W1 = FU(u) , W2 = FV(v) take v s.t. its w2 is equal to or just above w1

-34-

histogram matching example

Histogram equalized

-35-

local histogram processingproblem: global spatial processing not always desirablesolution: apply point-operations to a pixel neighborhood with a sliding window

-36-

histogram equalization not always desirable

-37-

outline

What and whyImage enhancementSpatial domain processing

Intensity TransformationIntensity transformation functions (negative, log, gamma), intensity and bit-place slicing, contrast stretchingHistograms: equalization, matching, local processing

Spatial FilteringFiltering basics, smoothing filters, sharpening filters, unsharp masking, laplacian

Combining spatial operations (sec. 3.7)

-38-

spatial filtering in image neighborhoods

-39-

kernel operator / filter masks

Spatial Filtering

f g

(.)(.) wTN =

∑∑−= −=

++=a

ai

b

bjjnimfjiwnmg ),(),(),(

NnMm

≤≤≤≤

11

kernel

-40-

smoothing: image averaging

Low-pass filter, leads to softened edges

smoothing operator

-41-

spatial averaging can suppress noiseimage with iid noise y(m,n) = x(m,n) + N(m,n)averagingv(m,n) = (1/Nw) Σ x(m-k, n-l) + (1/Nw) Σ N(m-k, n-l)

Nw: number of pixels in the averaging windowNoise variance reduced by a factor of Nw

SNR improved by a factor of Nw

Window size is limited to avoid excessive blurring

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

©20

02)

-42-

smoothing operator of different sizes

3x3

5x5

15x15

9x9

35x35

original

-43-

directional smoothingProblems with simple spatial averaging mask

Edges get blurredImprovement

Restrict smoothing to along edge directionAvoid filtering across edges

Directional smoothingCompute spatial average along several directionsTake the result from the direction giving the smallest changes before & after filtering

Other solutionsUse more explicit edge detection and adapt filtering accordingly

θ

Wθ

UM

CP

EN

EE

408G

Slid

es (c

reat

ed b

y M

.Wu

& R

.Liu

©20

02)

-44-

non-linear smoothing operatorMedian filtering

median value ξ over a small window of size Nw

nonlinearmedian{ x(m) + y(m) } ≠ median{x(m)} + median{y(m)}

odd window size is commonly used3x3, 5x5, 7x7

5-pixel “+”-shaped window

for even-sized windows take the average of two middle values as output

Other order statistics: min, max, x-percentile …

-45-

median filter example

iid noise

more at lecture 7, “image restoration”

Median filteringresilient to statistical outliersincurs less blurringsimple to implement

-46-

image derivative and sharpening

-47-

edge and the first derivativeEdge: pixel locations of abrupt luminance change

Spatial luminance gradient vector a vector consists of partial derivatives along two orthogonal directionsgradient gives the direction with highest rate of luminance changes

Representing edge: edge intensity + directionsDetection Methods

prepare edge examples (templates) of different intensities and directions, then find the best matchmeasure transitions along 2 orthogonal directions

-48-

edge detection operators

Robert’s operator

Sobel’soperator

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

∂∂∂∂

=⎥⎦

⎤⎢⎣

⎡=∇

yfxf

GG

fy

x

Image gradient:

yx GGf +≈∇

-49-

edge detection examples

http://flickr.com/photos/reneemarie11/97326485http://flickr.com/photos/valkyrie2112/2829322895

Roberts

Sobel

-50-

second derivative in 2D

Image Laplacian:

-51-

laplacian of roman ruins

http://flickr.com/photos/starfish235/388557119/

-52-

unsharp maskingUnsharp masking is an image manipulation technique for increasing the apparent sharpness of photographic images. The "unsharp" of the name derives from the fact that the technique uses a blurred, or "unsharp", positive to create a "mask" of the original image. The unsharped mask is then combined with the negative, creating a resulting image sharper than the original.

StepsBlur the imageSubtract the blurred version from the original (this is called the mask)Add the “mask” to the original

-53-

high-boost filtering

Avg.

+

+

-

),(),(),( yxfyxAfyxf lphb −=

Unsharp mask:

high-boost with A=1

-54-

unsharp mask example

-55-

unsharp mask example

http://flickr.com/photos/dpgnashua/2274968238/

-56-

summary

Spatial transformation and filtering are popular methods for image enhancementIntensity Transformation

Intensity transformation functions (negative, log, gamma), intensity and bit-place slicing, contrast stretchingHistograms: equalization, matching, local processing

Spatial Filteringsmoothing filters, sharpening filters, unsharp masking, laplacian

Combining spatial operations (sec. 3.7)

-57-

sharpen !

http://flickr.com/photos/t_schnitzlein/87607390/

-58-