2102676 digital image processing 7.pdfdigital image processing lecture 7 image segmentation razi...

Digital Image Processing

Lecture 7Image Segmentation

Razi University

School of Electrical Engineering &

Computer Science

Spring Semester 2020

Content and Figures adapted from Gonzales and et al. , “Digital Image Processing. 1

Segmentation

Types of segmentation

Point, line and edge detection

Edge linking

– local and global methods

– Hough transform

Thresholding

– global and local, optimal methods

– multivariable thresholding

2

Image Segmentation

Process of dividing an image into two or more regions

– regions of interest (ROI)

– background / foreground

– clustering, boundary detection

Further shift towards image analysis / machine vision

One of the most difficult tasks of machine vision

3

Segmentation Properties

Accuracy – good boundary localisation

Robustness to noise, false boundaries

– sensor types can help this

Fixed/variable number of clusters?

Region or boundary based?

– similarity or discontinuity measures

Connectivity

4

Discontinuity Detection

Process of finding points, lines and edges

Detects significant local variations in image statistics

– grey levels, local mean/variance, texture features, etc.

General process:

– filter and threshold

Filter kernel determined by object to be detected

– gradient filter, Laplacian filter, and others.

5

Point Detection

Detection of single isolated

points

Laplacian filter can detect

such locations

Threshold is applied to find

likely points of image

Can be generalised to larger points by rescaling

image (shrinking) or increasing filter size

1 1 1

1 -8 1

1 1 1

yxr ,

6

Line Detection

Filters can be designed to detect lines of different directions

– generally 1 pixel wide

– can be adjusted by same techniques as points

Again, spatial filtering with thresholding is used to detect likely line regions

Filter kernel sums to zero to ensure no response in ‘flat’ image regions.

7

Line Detection (Cont’d)

8

Let R1, R2, R3, and R4 denote the responses of the masks in the figure

below, from left to right.

Suppose that all masks are run through an image.

If, at a certain point in the image, |Ri| > |Rj|, for all j = i, that point

is said to be more likely associated with a line in the direction of mask

i.

Horizontal + 45O Vertical - 45O

-1 -1 -1

2

-1-1-1

2 2

-1 -1 2

-1

-1-12

-1 2

-1 2 -1

-1

-12-1

-1 2

2 -1 -1

-1

2-1-1

-1 2

8

Line Detection Example

9

Edge Detection

Formally defined as a boundary between

two regions of an image

– difficult to model mathematically

– often does not correspond to grey-level changes

10

Edge Detection (Cont’d)

Requires local measurable property

– grey level values, average, std. deviation, texture feature,

colour, etc..

Areas of strong change in property are considered

edges

– can be measured using first derivative

– values above threshold are considered edge pixels (edgels)

– second derivative is also useful for locating edges

– Note:

11

Edge Profile

12

13

Edge detectionby derivativeoperator

Edge Profile

Noisy Edges

14

Eliminating Effects of Noise

Edges require smoothing to remove noise

– low-pass filtering

Ringing would have extremely adverse effect

– ideal, most Butterworth filters are unacceptable

– Gaussian filter is ideal in this situation

Cutoff frequency (sigma) is dependent upon estimated noise levels of image

15

Gradient Operators

1st derivative is not defined for 2 dimensions

– require approximation using gradient operators

Gradient vector:

Gradient magnitude and direction are also important properties

y

fx

f

G

G

y

xf

16

22yx GGmagf )( f

yx GGf – The approximation is:

17

Gradient Operators (Cont’d)

The direction of the gradient vector also is an important quantity. The

direction angle of the vector at (x,y) is

Because derivatives enhance noise, the smoothing effect is a

particularly attractive feature of the Sobel operators which its masks

are

And

f

x

y

G

Gtanyx 1),(

Gx = (z7 + 2z8 + z9) – (z1 + 2z2 + z3)

Gy = (z3 + 2z6 + z9) – (z1 + 2z4 + z7)

Gradient Filters

-1 0 1

-2 0 2

-1 0 1

-1 -2 -1

0 0 0

1 2 1

0 -1

1 0

-1 -1 -1

0 0 0

1 1 1

-1 0

0 1

-1 0 1

-1 0 1

-1 0 1Roberts

Prewitt

Sobel18

Edge Localisation

Threshold is applied to gradient magnitude

– sum of magnitudes often used as approximation

– values over threshold considered edges

– may result in ‘thick’ edges

– valid edges can be missed if gradient is not high enough

– value of threshold depends on image content, and

desired error tradeoff

19

Gradient Examples

Original

No

smoothing

Smoothed(

5x5 avg)

20

21

Laplacian

The Laplacian of a 2D function, f(x,y), is a 2nd-

order derivative defined as:

For a 3x3 region, the form most frequently used in

practice is:

2

2

2

22

y

f

x

ff

)( 864252 4 zzzzzf

22

The Laplacian responds to transitions on intensity.

As a 2nd-order derivative, the Laplacian typically is sensitive to noise.

The Laplacian produces double edges and is unable to detect edge direction.

The Laplacian usually plays the secondary role of detector for establishing whether a pixel is on the dark or light side of an edge.

A more general and reliably use of the Laplacian is in finding the “location” of edges using its zero-crossing property:

– This concept is based on convolving an image with the Laplacian of a 2D Gaussian function of the form: h(x,y)=exp[-(x2+y2)/2s2].

Where s is the standard deviation (Marr and Hildreth [1980]).

– Let r2 = x2 + y2. Then 2h = [(r2 – s2)/s2] exp[-r2/2s2]

– The result of convolving an image will blur that image. The degree of blurring is proportional to s.

Laplacian (Cont’d)

23

The zero-crossing works well in cases when edges are

blurry or when a high noise content is present.

The zero-crossings offer reliable edge detection.

0 -1 0-10-10

-1 4

Mask usedto computethe Laplacian

Cross section of 2h. 2h shown as an intensity function.(From Marr [1982].)

Laplacian (Cont’d)

24

(a) Original image;

(b) Result of convolving (a) with 2h;

(c) Result of making (b) binary to simplify detection of zero-crossings;(d) Zero-crossing. (From Marr [1982].)

Laplacian (Cont’d)

Laplacian for Edge Detection

Zero crossings of Laplacian represent edges

– very sensitive to noise

– smoothing even more critical than for gradient

Gaussian and Laplacian filters can be combined (via

convolution) to form single kernel

Laplacian of Gaussian (LoG):

2224

222 ,

2

2

yxrer

rh

r

s

s

s

25

Laplacian of Gaussian

26

Detecting Zero-Crossings

Basic approach

– threshold with value of 0

– perform morphological boundary detection

Problems

– all closed loops

– small zero-crossings are not always desired

Applying a non-zero (positive and negative) threshold is a possible solution

27

LoG Example

28

LoG Failure

29

Edge Linking

Gradient-based edge detection often leads to

incomplete boundaries

Linking used to bridge gaps and prune false

edges

Two basic approaches:

– local techniques

– global techniques

30

Local Edge Linking

Examine local neighbourhood of each edgel

– size of neighbourhood dependent on application

– ‘similar’ points are linked, forming boundary

Similarity measure can be:

– strength of gradient

– direction of gradient

Additional constraints can be applied for specific problems

– direction/magnitude of gradient in certain ranges

31

Local Edge Linking

32

Hough Transform

Global approach to edge linking

Transforms image into Hough (line) domain

– lines represented by angle and position

– significantly fast than line searching approach

Each edge pixel could be part of infinite number of lines y = ax + b

– each with unique angle(a) and location(b)

Line in image represents point in HT

Point in image represents locus in HT

33

Hough Transform (Cont’d)

y = ax+b is not the best representation

– a approaches infinity for vertical lines

Better:

– theta ranges from –90 to 90

– rho ranges from 0 to diagonal width of image

sincos yx

34

Hough Transform Example

35

Hough Transform Edge

Linking Lines in edgemap will have high point values in

Hough transform

– points over a threshold can be considered full lines and

linked in edge image

– gaps in line are filled if required

Hough transform can be easily extended to arbitrary

shapes

– suitable parameter space is required

– higher dimensionality -> more computations

36

Image Thresholding

Used to separate objects from background

– form of image segmentation

Can operate on any local statistic

– grey levels only considered here

Can be applied successfully when an image

has two (or more) dominant modes

– can also be applied to smaller image regions

37

Image Thresholding

Threshold can be generalised as a function

yxfyxpyxTT ,,,,,

38

Thresholding Function

Global thresholds

– depend only on f(x,y)

Local thresholds

– also depend on p(x,y)

Adaptive thresholds

– also dependent on x,y

39

Illumination Effects

Non-uniform illumination can cause serious problems for thresholding algorithms

– causes histogram ‘blurring’

40

Simple Global Thresholding

Single value of T is used for entire image

– works only in well controlled conditions

– copes poorly with illumination effects

– often fails in presence of shadows

Selection of T

– median or average grey-level value

– (max+min)/2

– iterative refinement approach

41

Global Thresholding Example

42

Adaptive Thresholding

Simple approach

– divide image into regions

– use separate threshold value for each region

Not all regions require thresholding

– need property to determine this – standard deviation works

well in many cases

Regions with small white or black areas are often

thresholded incorrectly

– further splitting of region can overcome this

43

Adaptive Thresholding

44

Optimal Thresholding

Determining the ‘best’ threshold for a given region is

a non-trivial problem

– may contain < 2 or >2 modes

– modes may not be well separated in histogram

Optimal thresholds are designed to produce the

minimum segmentation error

– by some pre-defined criteria

– often makes some assumptions regarding the PDF’s of the

region

45

Optimal Thresholding

Assume histogram is comprised of two

separate PDF’s

– prior probabilities may or may not be equal

46

Optimal Thresholding

Full PDF is then given by:

Given a threshold T, the overall error is

defined as

Minimised when,

zpPzpPzp 2211

dzzpPdzzpPTET

T

1122

TpPTpP 2211

47

Optimal Thresholding

PDF’s of two distributions are rarely known

Often estimated with two Gaussians, with

parameters

Assuming equal variances, T can be

obtained by:

For P1=P2, T is the average of means

2211 ,,, ss

2

1

21

2

21 ln2 P

PT

s

48

Appendix A: Hough Transform

49

Appendix A: Hough Transform

50

Appendix A: Hough Transform

51

Appendix A: Hough Transform

52