Suetens 1

Digital Image Processing

Ho Kyung Kim

Pusan National University

Introduction to Medical Engineering (Medical Imaging)

Motivation

2

• Introduce a number of basic mathematical operations on images

– Image enhancement

– Image analysis

– Visualization

• Provide the clinician with some means to: (perceive better all the relevant diagnostic information

present)

– Enhance contrast of local features

– Remove noise and other artifacts

– Enhance edges and boundaries

– Composite multiple images for a more comprehensive view

• Two basis operations

– Global operations

• Operate on the entire set of pixels at once

• e.g., Brightness and contrast enhancement

– Local operations

• Operate only on a subset of pixels (in a pixel neighborhood)

• e.g., Edge detection, contouring, image sharpening, blurring

Gray level transformations

3

• To increase the contrast in some regions of the image

enhance the dark area

(slope > 1)suppress the bright area

(slope < 1)

Original Enhanced (or transformed)

�� �, � = � �(�, �)

Thresholding, level/windowing

4

• Thresholding

– �� � = 0for� ≤ ��– �� � = �for� > ��

• � = the largest gray level

• �� = the threshold

– Very useful for images with a bimodal histogram

window width

level

threshold

• Window/level operation

– ��,� � = 0for� < � − ��

– ��,� � = �� � − � + �

� for� − �� ≤ � ≤ � +

��

– ��,� � = �for� > � + ��

• � = level

• � = window width

– Lost contrast outside the window

– Stretched contrast inside the window

5

Original

Bone window Lung window

(Bimodal histogram)

Multi-image operations: Add/subtraction

6

• Get rid of the background in two similar images

– �� �, � = �� �, � + �� �, �– � �, � = �� �, � − �� �, �– e.g., Blood vessel imaging (angiography): images with and without a contrast agent

After injection Before injection (mask image) After subtraction

Multi-image operations: Averaging

7

• �!"#(�, �) = �$ �� �, � + ⋯+ �$(�, �)

• Useful to decrease the noise in a sequence of images (of a motionless object)

• Averaged the random noise out but leaving the object unchanged

Original After averaging 16 images

Geometric operations

8

• Image-to-patient registration for image-guided surgery

• Registration of images from different modalities (image fusion)

3D CT 3D MR CT + MR

Common 2D transformations

9

• Scaling (zooming)�′'′1

=)* 0 00 )+ 00 0 1

�'1

• Translation�′'′1

=1 0 ,*0 1 ,+0 0 1

�'1

• Shear�′'′1

=1 -* 0-+ 1 00 0 1

�'1

• Rotation �′'′1

=cos 0 − sin 0 0sin 0 cos 0 00 0 1

�'1

• General affine�′'′1

=3�� 3�� ,*3�� 3�� ,+0 0 1

�'1

Filters

10

– 4 �, � = 5 6(�, �) called the kernel or filter

– Linear transformation on � is the discrete convolution with its kernel 4

From linear-systems theory: � �, � = ∑ �(8, �)6(� − 8, � − �)9,�

For a linear shift-invariant (LSI) transformation :

5 �(�, �) =:�(8, �)5 6(� − 8, � − �)9,�

=:�(8, �)4(� − 8, � − �)9,�

=:4 8, � �(� − 8, � − �)9,�

= 4 �, � ∗ �(�, �)

11

– This transformation on � is the discrete cross-correlation of ℎ and 4• Aka, ℎ is called an image template or mask

– If the filter is symmetric, the cross-correlation and convolution are identical

Applying the same operation with the flipped kernel: ℎ �, � = 4(−�, −�)

5 �(�, �) = 4 �, � ∗ � �, � = :4 8, � �(� − 8, � − �)9,�

=:ℎ 8, � �(� + 8, � + �)9,�

= ℎ �, � ⊗ �(�, �)

• Filtering operation

① Superimpose the center of the mask ℎ(0,0) onto an image pixel (�, �)② Multiply the values of the mask and image that correspond to the same position

③ Sum and replace the value of pixel (�, �) by the summed value

④ Move to the next pixel and repeat

12

• Averaging filter

– Making the image smoother and removing some noise

– Giving the same weight to the center pixel as to its neighbors

Taken from R. C. Gonzalez & R. C. Woods, Digital Imaging Processing (2002)

3×3

5×5 9×9

15×15 35×35

13

• Low-pass filter

– Averaging filters

– To smoothen and/or reduce noise

• High-pass filter

– To enhance small-scale variations

– To extract edges and fine structures

Gaussian filter

(20 x 20 pixels, σ = 15)

Original

Original – LPF'd image

Gaussian filter: to give high

weight to the center pixel and

less weight to distant pixels

- Convolution vs. multiplication

- Acting as LPF

- Then, how to construct HPF?

� > = 12@A� B

CD/�FD ℱ � > =?

14

• Gaussian filters in space and frequency domains

Taken from R. C. Gonzalez & R. C. Woods, Digital Imaging Processing (2002)

15

• Differential operator

– Gradient, Laplacian

– Approached by interpolating a discrete function with a differentiable function

•II* � �, � ≈

II* ∑ � 8, � 4 � − 8, ' − �9,� *KL,+KM = ∑ IN L 9,M �

I* � 8, �9,�

• An approximate derivative by a convolution with a filter that is the sampled derivative of some

differentiable interpolation function

– This procedure can be used to

• O� = O4 ∗ � gradient

• O�� = O�4 ∗ � Laplacian

• Using the Gaussian function for 4:

– O� > = − �FD � > · >

– O�� > = �FQ (R� − 2A�) · �(>)

– For A = 0.5;

0.01 0.08 0.01

0.08 0.64 0.08

0.01 0.08 0.01

0.05 0 -0.05

0.34 0 -0.34

0.05 0 -0.05

0.05 0.34 0.05

0 0 0

-0.05 -0.34 -0.05

0.3 0.7 0.3

0.7 -4 0.7

0.3 0.7 0.3

Gaussian

Note that the Laplacian is superior in enhancing fine detail, but which causes noisier results than the gradient.

S/S� S/S' O�

16

1 0 -1

2 0 -2

1 0 -1

1 1 1

1 -8 1

1 1 1

Approximate Laplacian of Gaussian

““““SobelSobelSobelSobel””””

for the first derivative

“average “average “average “average ---- δδδδ””””

for the Lapalacian

Gaussian function Derivative in �

Derivative in ' Laplacian

Note that integration of a Gaussian over the whole spatial domain

must be 1, and for the gradient and Laplacian must be 0.

R�AT +

2A� � > − 4

A� � >

17

• Unsharp masking

– To enhance edge by emphasizing the high-frequency part and assigning it a higher weight

Original � � ∗ � (3 x 3)

V– X ∗ V α = 5

- α (> 0) controls the strength

of the enhancement, and σ(of �) is responsible for the

size of the frequency band

that is enhanced.

- The smaller σ, the more

unsharp masking focuses on

the finest details.

� = � ∗ � + (� − � ∗ �)

�′ = � ∗ � + (1 + Z)(� − � ∗ �)= � + Z(� − � ∗ �)= 1 + Z � − Z� ∗ �

Nonlinear filters

18

• The averaging filter removes noise. In addition, edges are also smeared out.

• Better to calculate the median instead of the mean value in small window around each pixel.

Original chromosome image Gaussian filter Median filter

19

• Note that the “median filtering” is a non-linear process capable of removing image features, and which

is unacceptable in medical imaging processing

Taken from R. C. Gonzalez & R. C. Woods, Digital Imaging Processing (2002)

3×3 averaging filter 3×3 median filterSalt-and-pepper

noise

20

Taken from R. C. Gonzalez & R. C. Woods, Digital Imaging Processing (2002)

1) Original 2) Laplacian of 1)

3) Sharpened by

adding 1) & 2) 4) Sobel of 1)

5) Smoothed by

taking a 5×5

averaging filter to 4)

6) Mask [3) × 5)]

7) Sharpened by

adding 1) & 6)

8) Power-law

transformation of 7)

Laplacian to highlight fine detail

Gradient to enhance prominent edges

Transformation to increase dynamic range

Effect of the filter size in unsharp masking

21

Original

Unsharp; size 10 Unsharp; size 30

Unsharp; size 60 Unsharp; size 125

Enhanced fine details, but reduction in

contrast

Enhanced large-scale variations (lung

& mediastinum), but suppressed small

details

Multiscale image representation

22

• Reduce an image by smoothing and subsampling (with a factor of 2)

– ℛ � =↓ (� ∗ �)• Pyramid of images

– �L�� = ℛ �L =↓ (� ∗ �L) for � = 0,… , ^ − 1 and �_ = �• Expand an image by upsampling and interpolation

– ℰ � = 4� ∗(↑ �)• Approximate Laplacian operator:

– b � = � − ℰℛ � = 1 − ℰℛ �• Laplacian pyramid: bL � = 1 − ℰℛ �L = �L − ℰ(�L��)

• Multiscale representation

– {b_, b�, … , bd �, �d}⇒ reconstruction by �L = bL + ℰ(�L��)– The edges or details at the different resolution levels together with the residual image �d– A pyramid of detail images

– A finer-scale image �L can be obtained from the coarser-scale image �L�� by adding the finer-scale details bL to it

Image enhancement by nonlinear mapping

23

• MUltiScale Image Contrast Amplification

– Convert � to its multiscale representation {b_, b�, … , bd �, �d}– Enhance the contrast of each detailed image by the non-linear gray scale transformation, hence

{b′_, b′�, … , b′d �}– Reconstruct the enhanced image from {b′_, b′�, … , b′d �} and the residual image �d

Original Edge enhancement Window/level MUSICA