10-2 - morphological image processing

8/4/2019 10-2 - Morphological Image Processing

1/24

4/29/

MorphologicalImageProcessing:BasicAlgorithms

Spring2009 ELEN4304/5365DIP 1

byGlebV.Tcheslavski:[email protected]://ee.lamar.edu/gleb/dip/index.htm

PreliminariesWhen dealing with binary images, one of the principal

applications of morphology is extracting image components

that are useful in the re resentation and descri tion of

shape.

We consider morphological algorithms for extracting

boundaries, connected components, the convex hull, and the

skeleton of a region. We also develop methods (region

fillin thinnin thickenin and runin that are


frequently used in conjunction with these algorithms as pre-

or post-processing steps.


2/24

4/29/

BoundaryExtractionTheboundary of a setA, denoted as (A), can be obtained by first

erodingA byB and then performing the set difference betweenA andits erosion as follows:

( ) ( ) A A A B = whereB is a suitable structuring element.

SetA Structuring

elementB


Erosion ofAbyB

Boundary: set

differencebetweenA and

its erosion

BoundaryExtraction


A binary image Boundary extracted using a 3x3

structuring element of ones

Size of structuring element defines the boundary being 1 pixel thick.


3/24

4/29/

HoleFillingAhole may be defined as a background region surrounded by a

connected border of foreground pixels.

Let denote b A a set whose elements are 8-connected boundaries,

each boundary enclosing a background region (a hole). Given a point

in each hole, the objective is to fill all the holes with ones (for binary

images).

We start from forming an arrayX0 of zeros (the same size as the array

containingA), except at the locations inX0 corresponding to the given


, . ,

fills all the holes with ones:

( )1 1,2,3,...c

k k X X B A k = =whereB is the symmetric structuring element.

HoleFillingThe algorithm terminates at the iteration step kifXk= Xk-1.

The setXkthen contains all the filled holes; the union ofXkand A contains all the filled holes and their boundaries.

The dilation would fill the entire area if left unchecked.

However, the intersection at each step with the complement

ofA limits the result to inside the region of interest. This is

an example of how a morphological process can be

conditionedto meet a desired ro ert . In the current


application, it can be called aconditional dilation.


4/24

4/29/

HoleFillingSetA

Complement ofA

tructur ng e ement

SetX0; a grey dot

indicates location

of a hole


A set with a filled

hole

HoleFillingAn image that could result from thresholding to 2 levels a scenecontaining polished spheres (ball bearings). Dark spots could be

results of reflections. The objective is to eliminate reflections by hole

ng

A (white) point selected

inside one sphere

Result of filling that

component

Result of filling all

the spheres



5/24

4/29/

ExtractionofconnectedcomponentsWe discussed concepts of connectivity and connected components

earlier. Extraction of connected components from a binary image isimportant for many automated image analysis applications.

LetA be a set containing one or more connected components. We

form an arrayX0 (of the same size as the array containingA), whose

elements are zeros (background values), except at each location

known to correspond to a point in each connected component inA,

which we set to one (foreground value). The objective is to start with

X0 and find all the connected components by the following iterative


proce ure:

( )1 1,2,3,...k k X X B A k = =

whereB is a suitable structuring element. The procedure terminates

whenXk=Xk-1 withXkcontaining all the connected components ofA.

ExtractionofconnectedcomponentsStructuring elementbased on 8-connectivity



6/24

4/29/

ExtractionofconnectedcomponentsConnected components are often

used for automated inspection.

X-ray image of chicken filet with

bone fragments

Thresholded image


Image eroded with a 5x5 structuring

element

ExtractionofconnectedcomponentsIt is of interest to be able to detect foreign fragments in processedfood before packaging or shipping.

In this case, the density of the bones is such that their intensity values

are different from background. After thresholding, we observe that the

points that remain are clustered into objects (bones). Therefore, we

can make sure that only objects of significant size remain by

eroding the thresholded image. for erosion, a 5x5 structuring element

was selected.

Next, we analyze the size of objects that remain. We identify them by


extracting the connected components in the image. As a result, 15

connected components were found with 3 of them being dominant in

size (133, 674, and 743 pixels). This is good enough to determine that

significant undesirable objects are contained in the image.


7/24

4/29/

ConvexHullA setA is said to beconvex if the straight line segment joining any

two points inA lies entirely withinA. Theconvex hull Hof anarbitrary set S is the smallest convex set containing S.

The differenceH S is called theconvex deficiency ofS. The convex

hull and convex deficiency are useful for object description.

LetBi, i=1,2,3,4 represent the 4 structuring elements. The procedure

consists of implementing the equation:

( )1 1,2,3,4 1,2,3,...i ik k X X B A i k = = =


0

iwith X A=

When the procedure converges (Xik= Xik-1), we letD

i = Xik. The

convex hull ofA is then

( )4

1

i

i

C A D=

=

ConvexHullTherefore, the method consists of iteratively applying the

hit-or-miss transform toA withB1; when no further

changes occur, we perform the union withA and call the

resultD1. The procedure is repeated withB2( applied toA)

until no further changes occur, and so on the union of

the four resultingDs is the convex hull ofA.



8/24

4/29/

ConvexHullStructuring elements: x indicates

do not care conditions

A set A

Results of convergence with

structural elements


Convex hull showing the contributions

of each structuring element

ConvexHullOne shortcoming of the procedure is that the convex hull can growbeyond the minimum dimensions required to guarantee convexity.

One simple approach to reduce this effect is to limit growth such that

it does not extend past the vertical and horizontal dimensions of the

original set.

The result of this limitation on the

previous image: a convex hull limited to


the dimensions of the original set.


9/24

4/29/

ThinningThethinning of a setA by a structuring elementB is defined in terms

of the hit-or-miss transform:c

A B A A B A A B= =

So far, we were interested only in pattern matching with the

structuring elements, so no background operation is required in the

hit-or-miss transform. A more useful expression for thinningA

symmetrically is based on a sequence of structuring elements:

1 2 ... n B B B B=


whereBi are rotated versions ofBi-1. The thinning by a sequence of

SEs: { } ( )( )( )1 2... ... n A B A B B B=

ThinningThe process is to thinA byone pass withB1, then thin

the result with one pass of Sequence of rotated SEsSet, an so on, un s

thinned with one pass ofBn.

The entire process is repeated

until no further changes

occur.

Results of thinning with


eac one a ter anot er

Using 4 first SEs again

Conversion to m-connectivity

Result after convergence


10/24

4/29/

ThickeningThickening is a morphological dual of a thinning and is defined as

( )c

A B A A B=

whereB is a structuring element suitable for thickening. As in

thinning, thickening can be defined as a sequentional operation:

{ } ( )( )( )1 2... ... n A B A B B B= The structuring element used for thickening has the same form as one


.

However, the usual procedure is to thin the background of the set to

be processed and then complement the result. Therefore, to thicken asetA, we form its complement, thin it, and then complement the

result.

ThickeningDepending on the nature ofA, the thickening procedure may result in

disconnected points. Therefore, this method is usually followed by

post-processing to remove disconnected points.

SetA Its

complement

Thinned

complement

Complement

of thinning


ofA complement...

Result of thickening with

no disconnected points


11/24

4/29/

SkeletonsA skeleton S(A) of a setA can be viewed as:

a) If z is a point ofS(A) and (D)z is the largest disk centered atz and

contained inA, one cannot find a lar er disk not necessaril

centered atz) containing (D)z and included inA. The disk (D)z is

called amaximum disk.

b) The disk (D)z touches the boundary ofA at two or more different

places.

The skeleton ofA can be expressed in terms of erosions and openings:


( ) ( )

( ) ( ) ( )

0

K

k

k

k

S A S A

S A A kB A kB B

=

=

=

ksuccessive erosions ofA

Structuring element

Skeletons( ) ( )( )( )... ... A kB A B B B=

Kis the last iterative step beforeA erodes to an empty set:

( ){ }max |K k A kB= S(A) can be obtained as the union of theskeleton subsets Sk(A). Also,

we can show thatA can be reconstructed from these subsets by:

( )( )

K

k A S A kB=


0k=

ksuccessive dilations ofA

( ) ( )( )( )... ... A kB A B B B =


12/24

4/29/

SkeletonsVarious positionsof maximum

disks with

centers on the

skeleton ofA

Another

maximum disk


on a different

segment of the

skeleton ofA

skeleton ofA

Skeletons

Set A

Openings byBSet differences

between 1st

and 2nd

columns

2 partial skeletons and final

Erosion 1

ofA


Erosion 2

ofA: next

erosion

will be

Reconstruc-

tedA

SE

Skeleton of

A (not connected)


13/24

4/29/

PruningPruning methods are an essential component to thinning and

skeletonizing algorithms since these procedures tend to leave parasiticcomponents that need to be cleaned up by post-processing. We start

w a prun ng pro em an en eve op a morp o og ca so u on.

A common approach in the automated recognition of printed characters

is to analyze the shape of the skeleton of each character. These

skeletons often are characterized by spurs (parasitic components).

Spurs are created during the erosion by non uniformities in the strokes

composing the characters. We develop a morphological technique for


an ling t is pro lem, starting wit t e assumption t at t e lengt of a

parasitic component does not exceed a specific number of pixels.

PruningOriginal image Structuring

elements: x

A parasitic component

a printed a

dont care

3 runs of

thinning:X1

End points:X2


Dilations of end

points

conditioned on

A:X3

Pruned image:

X4


14/24

4/29/

PruningThe solution is based on suppressing a parasitic branch by successively

eliminating its end point. This also shortens (or eliminates) otherbranches in the character. The assumption is that, in the absence of

o er s ruc ura n orma on, any ranc w or ess p xe s s ou e

eliminated. Thinning of an input setA with a sequence of structuring

elements designed to detect only end points achieves the desired result.

Let { }1 X A B= where {B} is the sequence of structuring elements. This sequence

consists of two different structures, each of which is rotated 900 for a


total of 8 elements.

Applying the equation 3 times yields the setX1 and the next step is to

restore the character to its original form but with the parasitic

branches removed.

PruningTo do this, we first form a setX2 containing all end points inX1:( )

8

2 1

1

k

k

X X B=

=

whereBkare the same structuring elements. Next step is dilation of the

end points 3 times using setA as a delimiter:

( )3 2 X X H A=

whereHis a 3x3 structuring element of ones and the intersection with

A as applied after each step. This type of conditional dilation prevents


appearance of non-zero elements outside the region of interest.

Finally, the union ofX3 andX1 yields the desired result:

4 1 3 X X X =


15/24

4/29/

PruningIn more complex situations,X3 sometimes includes the tips of some

parasitic branches. This can occur when the end points of these

branches are near the skeleton.

Although, they may be eliminated inX1, these elements may show up

during the dilation since they are valid points inA. The entire parasitic

elements are rarely picked up again since they are usually short

compared to the valid strokes. Therefore, their detection and

elimination is easy since they are disconnected regions.


Morphologicalreconstruction(MR)Morphological reconstruction is a morphological

transform involving 2 images and a structuring element.

ne mage, emar er, con a ns e s ar ng po n s or e

transformation. The other image, themask, constrains the

transformation. The structuring element is used to define

connectivity.



16/24

4/29/

MR:geodesicdilationanderosionCentral to morphological reconstruction are the concepts of geodesic

dilation and geodesic erosion. Let Fdenote the marker image and Gthe mask image (assuming that both are binary images and that F

. e geo es c a on o s ze o e mar er mage w respec

to the mask is( ) ( ) ( )1GD F F B G=

Here denotes the set intersection and may be interpreted as logicalAND since they are the same for binary sets.

Thegeodesic dilation of size n ofFwith respect to G is


( ) ( ) ( ) ( ) ( )( )1 1n nG G G D F D D F =with ( ) ( )0G D F F =

MR:geodesicdilationanderosionIn the last expression, the set intersection is performed at each step.The intersection operator guarantees that maskG will limit the growth

(dilation) of marker F.

Spring2009 ELEN4304/5365DIP 32Geodesic erosion


17/24

4/29/

MR:geodesicdilationanderosionSimilarly, thegeodesic erosion of size 1 of the marker image Fwith

respect to the maskG is( ) ) )1G E F F B G=

where denotes the union (OR operation). The geodesic erosion ofsize n ofFwith respect to G is defined as

( ) ( ) ( ) ( ) ( )( )1 1n nG G G E F E E F =( ) ( )0G E F F =

with


The union operation is performed at each iterative step and guarantees

that geodesic erosion of an image remains greater then or equal to itsmask image. Geodesic dilation and erosion are duals with respect to

set complementation.

MR:geodesicdilationanderosion

Spring2009 ELEN4304/5365DIP 34Geodesic dilation


18/24

4/29/

MRbydilationanderosionMorphological reconstruction by dilation of a mask image G from a

marker image Fis defined as the geodesic dilation ofFwith respectto G, iterated until stability is achieved:

( ) ) ( )kDG G R F D F =with ksuch thatDG

(k)(F) =DG(k+1)(F).

Considering these

marker, mask, and


structuring

element

MRbydilationanderosionDilation dilated

with SEResult AND G



19/24

4/29/

MRbydilationanderosionThe morphological reconstructed image isDG

(5)(F), which is identical

to the maskFsince Fcontained a single 1-valued pixel (this is

analogous to convolution of an image with an impulse, which simply

copies an image at the location of the impulse).

Themorphological reconstruction by erosion of a maskG from a

marker image Fis defined as the geodesic erosion ofFwith respect to

G, iterated until stability:


( ) ( )EG G R F E F =

with ksuch thatEG

(k)(F) =EG

(k+1)(F).

MR:sampleapplicationsMorphological reconstruction has a broad spectrum of practical

applications, each determined by the selection of the marker and mask

images, by the structuring elements used, and by combinations of the

primitive operations as defined above.

Opening by reconstruction:

In a morphological opening, erosion removes small objects and the

subsequent dilation attempts to restore the shape of objects that

remain. However, the accurac of this restoration is hi hl de endent


on the similarity of the shapes of the objects and the structuring

element used.

Opening by reconstruction restores exactly the shapes of the objects

that remain after erosion.


20/24

4/29/

MR:sampleapplicationsTheopening by reconstruction of size n of an image Fis defined as

the reconstruction by dilation ofFfrom the erosion of size n ofF:

n D

R Fn=

n erosions ofFbyB

We observe that Fis used as the mask in this application.

A similar expression can be written forclosing by reconstruction:


R FC F R F nB=

n dilations ofFbyB

MR:sampleapplicationsText image of size 918x2018;average height of tall characters is 50

Erosion with a SE of size 51x1 pixels


Opening of the image with the same

SE (for reference)

Opening by reconstruction: interest

to extract characters with long

vertical strokes


21/24

4/29/

MR:sampleapplicationsFilling holes:

We develop next a fully automatic procedure for hole filling based onmorphological reconstruction. LetI(x,y) be a binary image, assume

that we form a marker image Fthat is 0 everywhere , except at the

image border, where it is set to 1 I:

( )( ) ( )1 , ,

,0

I x y if x y ison the border of I F x y

otherwise

=

Then

cD

H R F =


cI

Is a binary image equal toIwith all holes filled.

MR:sampleapplications

Original

image

with a

Its

complement Fdilated

by a 3x3

SE of

Intersec-

tion of

dilation

Its comple-

ment Intersec-

tion with

com le-


ho e ones w th

comple-

ment

ment


22/24

4/29/

MR:sampleapplicationsText image of size 918x2018;

average height of tall characters is 50

Its complement


Marker image F Result of hole filling

MR:sampleapplicationsBorder cleaning:The extraction of objects from an image for subsequent shape analysis

is a fundamental task in automated image processing. An algorithm

for removing objects that touch (connected to) the border is useful

since:

1) it can be used to screen the image such that only complete objects

remain;

2) It can be used to detect partial objects that are present in the field


.

We develop a border-cleaning procedure based on morphological

reconstruction.


23/24

4/29/

MR:sampleapplicationsWe use the original image as the mask and the following marker

image:( ) ( ), , I x y if x y is on the border of I

F x

=0 otherwise

The border-cleaning algorithm first computes the morphological

reconstructionRID(F), which extracts the objects touching the border,

and then computes the difference

DX I R F =


To obtain an imageXwith no objects touching the border.

MR:sampleapplicationsFor the same text image, we need to eliminate the incomplete

characters (ones touching the border). This may be used before

automatic character recognition.


Marker image Fobtained using a

3x3 SE of ones

The image with no objects touching

the border


24/24

4/29/

StructuringelementsusedBasic types of structuring elements used in binary morphology:


10-2 - morphological image processing

Documents