optimal multi-thresholding using a hybrid optimization approach

14
Optimal multi-thresholding using a hybrid optimization approach Erwie Zahara a , Shu-Kai S. Fan b, * , Du-Ming Tsai b a Department of Industrial Engineering and Management, St. John’s & St. Mary’s Institute of technology, Tamsui, Taiwan 251, Republic of China b Department of Industrial Engineering and Management, Yuan Ze University, 135 Far East Road, Chung-Li, Taoyuan County, Taiwan 320, Republic of China Received 5 August 2003; received in revised form 27 August 2004 Available online 18 November 2004 Abstract The OtsuÕs method has been proven as an efficient method in image segmentation for bi-level thresholding. However, this method is computationally intensive when extended to multi-level thresholding. In this paper, we present a hybrid optimization scheme for multiple thresholding by the criteria of (1) OtsuÕs minimum within-group variance and (2) Gaussian function fitting. Four example images are used to test and illustrate the three different methods: the OtsuÕs method; the NM–PSO–Otsu method, which is the OtsuÕs method with Nelder–Mead simplex search and particle swarm optimization; the NM–PSO-curve method, which is Gaussian curve fitting by Nelder–Mead simplex search and particle swarm optimization. The experimental results show that the NM–PSO–Otsu could expedite the OtsuÕs method effi- ciently to a great extent in the case of multi-level thresholding, and that the NM–PSO-curve method could provide bet- ter effectiveness than the OtsuÕs method in the context of visualization, object size and image contrast. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Multi-level thresholding; OtsuÕs method; Gaussian curve fitting; Nelder–Mead simplex search method; Particle swarm optimization 1. Introduction Image thresholding has been widely used as a popular tool in image segmentation. It is useful in separating objects from background, or discrimi- nating objects from objects that have distinct gray-levels. Sahoo et al. (1988) have presented a 0167-8655/$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2004.10.003 * Corresponding author. Tel.: +886 3 4638800; fax: +886 3 4638907. E-mail address: [email protected] (S.-K.S. Fan). Pattern Recognition Letters 26 (2005) 1082–1095 www.elsevier.com/locate/patrec

Upload: erwie-zahara

Post on 21-Jun-2016

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Optimal multi-thresholding using a hybrid optimization approach

Pattern Recognition Letters 26 (2005) 1082–1095

www.elsevier.com/locate/patrec

Optimal multi-thresholding using a hybridoptimization approach

Erwie Zahara a, Shu-Kai S. Fan b,*, Du-Ming Tsai b

a Department of Industrial Engineering and Management, St. John’s & St. Mary’s Institute of technology,

Tamsui, Taiwan 251, Republic of Chinab Department of Industrial Engineering and Management, Yuan Ze University, 135 Far East Road, Chung-Li,

Taoyuan County, Taiwan 320, Republic of China

Received 5 August 2003; received in revised form 27 August 2004

Available online 18 November 2004

Abstract

The Otsu�s method has been proven as an efficient method in image segmentation for bi-level thresholding. However,

this method is computationally intensive when extended to multi-level thresholding. In this paper, we present a hybrid

optimization scheme for multiple thresholding by the criteria of (1) Otsu�s minimum within-group variance and (2)

Gaussian function fitting. Four example images are used to test and illustrate the three different methods: the Otsu�smethod; the NM–PSO–Otsu method, which is the Otsu�s method with Nelder–Mead simplex search and particle swarm

optimization; the NM–PSO-curve method, which is Gaussian curve fitting by Nelder–Mead simplex search and particle

swarm optimization. The experimental results show that the NM–PSO–Otsu could expedite the Otsu�s method effi-

ciently to a great extent in the case of multi-level thresholding, and that the NM–PSO-curve method could provide bet-

ter effectiveness than the Otsu�s method in the context of visualization, object size and image contrast.

� 2004 Elsevier B.V. All rights reserved.

Keywords: Multi-level thresholding; Otsu�s method; Gaussian curve fitting; Nelder–Mead simplex search method; Particle swarm

optimization

0167-8655/$ - see front matter � 2004 Elsevier B.V. All rights reserv

doi:10.1016/j.patrec.2004.10.003

* Corresponding author. Tel.: +886 3 4638800; fax: +886 3

4638907.

E-mail address: [email protected] (S.-K.S. Fan).

1. Introduction

Image thresholding has been widely used as a

popular tool in image segmentation. It is useful in

separating objects from background, or discrimi-nating objects from objects that have distinct

gray-levels. Sahoo et al. (1988) have presented a

ed.

Page 2: Optimal multi-thresholding using a hybrid optimization approach

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1083

thorough survey of a variety of thresholding tech-

niques, among which global histogram-based algo-

rithms (Glasbey, 1993) are widely employed to

determine the threshold and can be classified

into parametric and nonparametric approaches.In parametric approaches (Weszka and Rosenfeld,

1979; Synder et al., 1990), the gray-level distribu-

tion of each class has a probability density function

that is assumed to obey a Gaussian distribution.

An attempt to find an estimate of the parameters

of the distribution that will best fit the given histo-

gram data is made by using the least-squares meth-

od. It typically leads to a nonlinear optimizationproblem, of which the solution is computationally

expensive and time-consuming. Nonparametric ap-

proaches find the thresholds that separate the gray-

level regions of an image in an optimum manner

according to some discriminating criteria such as

the between-class variance (Otsu, 1979), entropy

(Kapur et al., 1985) and cross entropy (Li and

Lee, 1993). Compared to the parametric methods,nonparametric approaches are more computation-

ally efficient and simpler to implement in practice.

Both parametric and nonparametric approaches

have one common problem that the computational

complexity becomes exponential when bi-level

thresholding is extended to multi-level threshold-

ing. Synder et al. (1990) presented an alternative

method for fitting curves based on a heuristic meth-od called tree annealing. Yin (1999) proposed a fast

scheme for optimal thresholding using genetic

algorithms. Cheng et al. (2000) applied fuzzy

entropy in image segmentation, used it to select

the fuzzy region of membership function automat-

ically so that an image can be transformed into

fuzzy domain with maximum fuzzy entropy, and

implemented genetic algorithm to find the optimalcombination of the fuzzy parameters. However,

automatic selection of a robust, optimum threshold

has remained a challenge in image segmentation.

To overcome this problem, Yen et al. (1995) pro-

posed a new criterion for multi-level thresholding,

termed Automatic Thresholding Criterion (ATC).

The method uses a criterion function and the inter-

ested histogram region is dichotomized sequen-tially until the cost function reaches its minimum.

There have been numerous applications of fuzzy

entropy in image segmentation.

In this paper, an improvement upon Gaussian

curve fitting and the Otsu�s method is reported that

the effectiveness and the efficiency of these two

optimal thresholding methods could be greatly en-

hanced in the case of multi-level thresholding. Wepresent a hybrid Nelder–Mead simplex search

method and particle swarm optimization (NM–

PSO) to solve the objective functions of Gaussian

curve fitting and the Otsu�s method. The NM–

PSO method is applied to image thresholding with

multi-modal histograms, and the performances of

NM–PSO on Otsu and Gaussian curve fitting are

compared with those of the Otsu�s method viaexhaustive search. We could have also combined

NM–PSO with dichotomization technique to find

the optimal classification number in the Automatic

Thresholding Criterion. Likewise, NM–PSO can

be combined with maximum fuzzy entropy method

to find the optimal combination of all the fuzzy

parameters. The major benefit of NM–PSO meth-

od is to speed up the searching process for theimage segmentation method.

The remainder of this paper is outlined as fol-

lows. Section 2 describes the parametric and non-

parametric objective functions that need to be

solved in image thresholding. In Section 3, the pro-

posed hybrid NM–PSO method is presented.

Section 4 gives the experimental results and com-

pares performances among the methods. Finally,Section 5 concludes this work.

2. Parametric and nonparametric approaches

2.1. Parametric approaches by Gaussian curve

fitting

A properly normalized multi-modal histogram

p(x) of an image, where x represents the gray lev-

els, can be fitted with the sum of d probability den-

sity functions (pdf�s) for finding the optimal

thresholds for use in image segmentation (Synder

et al., 1990). With Gaussian pdf�s, the model has

the following form:

pðxÞ ¼Xd

i¼1

P iffiffiffiffiffiffi2p

pri

exp �ðx� liÞ2

r2i

" #ð1Þ

Page 3: Optimal multi-thresholding using a hybrid optimization approach

1084 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

and

Xd

i¼1

P i ¼ 1 ð2Þ

Here, Pi is the a priori probability, d is the levels of

thresholding, li is themean, and r2i is the variance of

mode i. A pdf model must be fitted to the histogram

data, typically by themaximum likelihood ormean-

square error approach in order to locate the optimalthreshold. Given the histogram data p̂j (observedprobability of gray level j) described below:

p̂ðjÞ ¼ gðjÞPL�1

i¼0 gðiÞð3Þ

where g(j) denotes the occurrence of gray-level j

over a given image range [0,L � 1], and L is the

total number of gray levels. We wish to find a set

of parameters, H, which minimize the fitting error:

Minimize H ¼Xj

½p̂j � pðxj;HÞ�2 ð4Þ

and j ranges over the bins in the measured histo-

gram. Here, H is the objective function to be min-

imized with respect to H, a set of parameters

defining the Gaussian pdf�s and the probabilities,

as given by

H ¼ fP i; li; ri; i ¼ 1; 2; . . . ; dg ð5ÞAfter fitting the multi-modal histogram, the opti-

mal threshold could be determined by minimizing

the overall probability of error, for two adjacent

Gaussian pdf�s, given by

EðT iÞ ¼ P i

Z T i

�1piðxÞdxþ P iþ1

Z 1

T i

piþ1ðxÞdx;

i ¼ 1; 2; . . . ; d � 1 ð6Þ

q1 ¼XT 1

i¼0

p̂ðiÞ; l1 ¼XT 1

i¼0

i� p̂ðiÞ=q1; r21 ¼

XT 1

i¼0

ði� l1Þ2

q2 ¼XT 2

i¼T 1þ1

p̂ðiÞ; l2 ¼XT 2

i¼T 1þ1

i� p̂ðiÞ=q2; r22 ¼

XT 2

i¼T 1þ1

ði�

..

. ... ..

.

qd ¼XL�1

i¼T d�1þ1

p̂ðiÞ; ld ¼XL�1

i¼T d�1þ1

i� p̂ðiÞ=qd ; r2d ¼

XL�1

i¼T d�1

with respect to the threshold Ti, where pi(x) is the

ith pdf (Gonzalez and Woods, 2002). To find the

threshold value for which this error is minimal re-

quires differentiating E(Ti) with respect to Ti (using

Leibniz�s rule) and equating the result to 0. Theresult is

P ipiðT iÞ ¼ P iþ1piþ1ðT iÞ ð7ÞThis equation is solved for Ti to find the optimum

threshold. Using Eq. (1) in the general solution ofEq. (7) results in the following solution for the

threshold Ti:

AT 2i þ BT i þ C ¼ 0 ð8Þ

where

A ¼ r2i � r2

iþ1

B ¼ 2ðlir2iþ1 � liþ1r

2i Þ

C ¼ r2i l

2iþ1 � r2

iþ1l2i þ 4r2

i r2iþ1 lnðriþ1P i=riP iþ1Þ

ð9ÞSince a quadratic equation has two possible solu-

tions, only one of which is a feasible solution.

2.2. Nonparametric approaches by the Otsu’s

method

Developed by Otsu (1979), an optimal thres-

holding criterion was introduced to minimize the

within group variance as follows:

Minimize f ðT 1; T 2; . . . ; T d�1Þ ¼Xd

i¼1

qi � r2i ð10Þ

where

� p̂ðiÞ=q1 ð11Þ

l2Þ2 � p̂ðiÞ=q2 ð12Þ

þ1

ði� ldÞ2 � p̂ðiÞ=qd ð13Þ

Page 4: Optimal multi-thresholding using a hybrid optimization approach

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1085

and

T j 2 ½0; L� 1�; j ¼ 1; 2; . . . ; d � 1 ð14Þwhere p̂ðiÞ is the observed probability of gray-level

i in an image ranging over [0,L � 1], as previously

described in Eq. (3), and Tj is the jth thresholdvalue.

In bi-level thresholding, minimizing the within

group variance is equivalent to maximizing the be-

tween class variance, so the Otsu�s method can find

the optimal threshold by exhaustive search effi-

ciently. However, when this method is extended

to multi-level thresholding, the computation time

grows exponentially with the number of thresh-olds. Specifically, the computational complexity

of exhaustive search is O(Ld�1), where L is the

number of gray levels. Thus, in practical applica-

tions where multi-level thresholding is necessary,

this method is not a logical choice.

A

B CD

G

H

E

J

Fig. 1. NM operations of a two-dimensional case.

3. Hybrid NM–PSO method

In this study, the development of the hybrid

algorithm is intended to improve the performance

of the optimal thresholding techniques currently

used in practice. The goal of integrating Nelder–

Mead (NM) simplex method and particle swarm

optimization (PSO) is to combine their advantages

and avoid disadvantages. For example, NM sim-plex method is a very efficient local search proce-

dure but its convergence is excessively sensitive

to the starting point selected; PSO belongs to the

class of global search procedures but requires

much computational effort. Similar ideas have

been discussed in hybrid methods using GA and

direct search techniques, and they emphasize the

trade-off between solution quality, reliability andcomputation time in global optimization (Renders

and Flasse, 1996; Yen et al., 1998). This section

starts by introducing the procedure of NM and

PSO, followed by a description of our hybrid

method.

3.1. The procedure of NM

The simplex search method, first proposed by

Spendley et al. (1962) and later refined by Nelder

and Mead (1965), is a derivative-free line search

method that was particularly designed for tradi-

tional unconstrained minimization scenarios, such

as the problems of nonlinear least squares, nonlin-

ear simultaneous equations, and other types offunction minimization (see, e.g., Olsson and Nel-

son, 1975). First, function values at the (N + 1)

vertices of an initial simplex are evaluated, which

is a polyhedron in the factor space of N input var-

iables. In the minimization case, the vertex with

the highest function value is replaced by a newly

reflected, better point, which would be approxi-

mately located in the negative gradient direction.Clearly, NM can be deemed as a direct line-search

method of steepest descent kind. The ingredients

of replacement process consist of four basic oper-

ations: reflection, expansion, contraction and

shrinkage. Through these operations, the simplex

can improve itself and come closer and closer to

a local optimum point sequentially. An example

of the function minimization of two variables willillustrate the basic procedure of NM. Starting

point B together with initial step sizes will con-

struct an initial simplex design (shown as A, B

and C), as illustrated in Fig. 1. Suppose f(A) is

the highest of the three function values and is to

be replaced. In this case, a reflection is made

Page 5: Optimal multi-thresholding using a hybrid optimization approach

1086 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

through the centroid of BC (with the midpoint D)

to the point E. Suppose f(C) < f(B) < f(A). At this

stage, three situations can arise.

1. If f(E) < f(C), an extension is made to point J.We then keep E or J as a replacement for A,

depending on which function value is lower.

2. If f(E) > f(C), a contraction is made to point G

or H, depending on whether f(A) or f(E) is

lower.

3. If f(G) or f(H) is larger than f(C), the con-

traction has failed and we then perform a

shrinkage operation. The shrinkage operationreduces the size of the simplex by moving all

but the best point C halfway towards the best

point C.

3.2. The procedure of PSO

Particle swarm optimization (PSO) is one of thelatest evolutionary optimization techniques devel-

oped by Kennedy and Eberhart (1995). PSO con-

cept is based on a metaphor of social interaction

such as bird flocking and fish schooling. Similar

to genetic algorithms, PSO is also population-

based and evolutionary in nature, with one major

difference from genetic algorithms that it does not

implement filtering, i.e., all members in the popu-lation survive through the entire search process.

PSO simulates a commonly observed social behav-

ior, where members of a group tend to follow the

lead of the best of the group. The procedure of

PSO is illustrated as follows.

1. Initialization: Randomly generate a popula-

tion of the potential solutions, called ‘‘particles’’,

and each particle is assigned a randomizedvelocity.

2. Velocity update: The particles then ‘‘fly’’

through hyperspace while updating their own

velocity, which is accomplished by considering its

own past flight and those of its companions. The

particle�s velocity and position are dynamically up-

dated by the following equations:

V Newid ¼ w� V old

id þ c1 � rand� ðpid � xoldid Þþ c2 � rand� ðpgd � xoldid Þ ð15Þ

xNewid ¼ xoldid þ V New

id ð16Þwhere c1 and c2 are two positive constants; w is an

inertia weight and rand is a uniformly generated

random number from the range [0,1] which is pro-

duced every time for each iteration. Eberhart and

Shi (2001) and Hu and Eberhart (2001) suggested

c1 = c2 = 2 and w = [0.5 + rand/2.0)]. Eq. (15)

shows that in calculating the new velocity for aparticle, the previous velocity of the particle

(Vid), their own best location that the particles

have discovered previously (pid) and the global

best location (pgd) all contribute some influence

on the outcome of velocity update. The global best

location (pgd) is identified, based on its fitness, as

the best particle in a population. All particles are

then accelerated towards the global best particleas well as in the directions of their own best solu-

tions that have been discovered previously. While

approaching the current best particle from differ-

ent directions in the search space, all particles

may encounter by chance even better particles en

route, and the global best solution will eventually

emerge. Particles� velocities on each dimension

are clamped to a maximum velocity Vmax, whichis confined to the range of the search space in each

dimension. Eq. (16) shows how each particle�sposition (xid) is updated in the search of solution

space.

3.3. Hybrid NM–PSO

The population size of this hybrid NM–PSOapproach is set at 3N + 1 when solving an N-

dimensional problem. Note that, in Gaussian

curve fitting we need to estimate 3d � 1 parame-

ters, so N is 3d � 1; in the Otsu�s method we need

to estimate d � 1 parameters, so N is d � 1. For

example, in bi-level thresholding, five parameters,

H = {P1,l1,r1,l2,r2} and P2 = 1 � P1, for Gaus-

sian curve fitting and 1 parameter, T, for theOtsu�s method can be solved by NM–PSO. The

initial population is created in two steps: using a

predetermined starting point, N particles are

spawned with a positive step size of 1.0 in each

coordinate direction, and the other 2N particles

are randomly generated. A total of 3N + 1 parti-

cles are sorted by the fitness, and the top N + 1

Page 6: Optimal multi-thresholding using a hybrid optimization approach

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1087

particles are then fed into the simplex search

method to improve the (N + 1)th particle. The

other 2N particles are adjusted by the PSO method

by taking into account the positions of the N + 1

best particles. This procedure for adjusting the2N particles involves selection of the global best

particle, selection of the neighborhood best parti-

cles, and finally velocity updates. The global best

particle of the population is determined according

to the sorted fitness values. The neighborhood best

particles are selected by first evenly dividing 2N

particles into N neighborhoods and denoting the

particle with the better fitness value in each neigh-borhood as the neighborhood best particle. By

Eqs. (15) and (16), a velocity update for each of

the 2N particles is then carried out. The 3N + 1

particles are sorted in preparation for repeating

the entire run. The process terminates when a cer-

tain convergence criterion is satisfied. In this case,

the hybrid NM–PSO method is not an exhaustive

search and we cannot determine its computationalcomplexity. Fig. 2 summarizes the algorithm of

NM–PSO. For further details of the hybrid NM–

PSO, see Fan and Zahara (2002).

In Section 2, we have described the parametric

approaches by Gaussian curve fitting and nonpar-

ametric approaches by the Otsu�s method. Both

objective functions (4) and (10) are now subjects

to being optimized by NM–PSO. For example, intri-level thresholding, eight parameters, H =

1. Initialization. Generate a population of siz

Repeat

2. Evaluation & Ranking. Evaluate the fitnes

Rank them based on the fitness.

3. Simplex Method . Apply NM operator to t

th)1( +N particle with the update.

4 . PSO Method. Apply PSO operator for upd

Selection. From the population select the g

best particles.

Velocity Update. Apply velocity update

according equations (12) and (13).

Until a termination condition is met.

Fig. 2. The hybrid NM

{P1,P2,l1,r1,l2,r2,l3,r3} and P3 = 1 � P1 � P2,

for Gaussian curve fitting and two parameter,

{T1,T2}, for the Otsu�s method can be solved by

NM–PSO.

4. Experimental results

In this section, we evaluate the performances of

the following methods: the Otsu�s method with

exhaustive search, and the proposed NM–PSO

algorithm implemented on the Otsu�s method

and Gaussian curve fitting as objective functions,which we shall refer to as NM–PSO–Otsu and

NM–PSO-curve, respectively. The test images are

of size 256 · 256 pixels with 256 gray-levels, taken

under natural room lighting without the support

of any special light source. The algorithms are

implemented on a Athlon XP 2200 + (166 · 11)

with 1GB RAM using Matlab. A stopping crite-

rion used for the study is 10 · N iterations whensolving an N-dimensional problem. To account

for both efficiency and effectiveness, the criterion

10 · N has been tested with much success from

previous experiments. Initially, parameters Pi for

NM–PSO-curve are set at Pi = 1/d, li�s are selectedfrom the d possible peaks of the histogram, and

ri�s are all set to 1.00. Starting values of the T

parameters for NM–PSO–Otsu are randomlygenerated from Uniform(0,255). The maximum

e 13 +N .

s of each particle.

he top 1+N particles and replace the

ating N2 particles with worst fitness.

lobal best particle and the neigborhood

to the 2N particles with worst fitness

–PSO algorithm.

Page 7: Optimal multi-thresholding using a hybrid optimization approach

Fig. 3. Standard bi-level thresholding image (image 1): (a) original image, (b) Otsu or NM–PSO–Otsu, (c) NM–PSO-curve, (d) original

histogram of (a), (e) bi-level threshold of (b), (f) bi-level threshold of (c).

1088 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

Page 8: Optimal multi-thresholding using a hybrid optimization approach

Fig. 4. Standard tri-level thresholding image (image 2): (a) original image, (b) Otsu or NM–PSO–Otsu, (c) NM–PSO-curve, (d)

original histogram of (a), (e) tri-level threshold of (b), (f) tri-level threshold of (c).

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1089

Page 9: Optimal multi-thresholding using a hybrid optimization approach

Table 1

Comparison results by using Otsu and NM–PSO–Otsu methods

Images Level Optimal thresholds

for Otsu and NM–PSO–Otsu

Optimal objective values

derived from (10)

CPU times (s)

Otsu NM–PSO–Otsu Iteration

1 2 133 29.85 0.000 0.000 10

2 3 111, 146 54.11 0.281 0.015 20

3 2 210 90.05 0.000 0.000 10

4 3 83, 175 145.58 0.282 0.016 20

Table 2

Experimental result for image 1 by NM–PSO-curve

Image Level Starting point Number of iteration

1 2 (0.50,99.00,166.00,1.00,1.00) 50

Method (P1,l1,l2,r1,r2) H (fitting error) Time (s)

NM–PSO-curve (0.21,98.70,165.43,2.38,5.72) 0.0007 22.81

T 118

Table 3

Experimental result for image 2 by NM–PSO-curve

Image Level Starting point Number of iteration

2 3 (0.33,0.33,94.00,120.00,159.00,1.00,1.00,1.00) 80

Method (P1,P2,l1,l2,l3,r1,r2,r3) H (fitting error) Time (s)

NM–PSO-curve (0.14,0.53,98.91,127.48,170.13,3.87,5.44,10.02) 0.0023 79.51

T1,T2 110, 143

1090 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

velocities Vmax for NM–PSO-curve and NM–PSO–Otsu are 1 and 10, respectively. NM–PSO-

curve is concerned with fitting histogram images

and the value of parameter Pi is smaller than 1,

so we set Vmax to 1. NM–PSO–Otsu deals with

finding the threshold value from gray levels 0–

255, so we set Vmax to 10.

The experiment starts with two standard images

with rectangular objects of uniform gray values,which are bi-level and tri-level images respectively

as shown in Figs. 3 and 4, and aims to verify that

NM–PSO–Otsu and NM–PSO-curve can provide

satisfactory performance. As can be seen from

these two figures, NM–PSO–Otsu (or the Otsu�smethod) and NM–PSO-curve perform equally well

in terms of the quality of image segmentation.

Comparison results for the two standard images

in Table 1 exhibit that both NM–PSO–Otsu andthe Otsu�s method having the identical optimal

threshold values converge in the same amount of

time in bi-level thresholding (image 1), but NM–

PSO–Otsu converges much faster than the Otsu�smethod in multi-level thresholding (image 2). For

images 3–4 (as will be discussed in detail later),

the speed advantage by NM–PSO–Otsu still pre-

vails among the multi-thresholding cases. Notethat the result from NM–PSO–Otsu is a real value

with round-off errors, and the truncated value is

the same as that of Otsu�s result and the number

of iterations for NM–PSO–Otsu is always 10N.

Comparing NM–PSO-curve and the Otsu�smethod execution times, it is clear to see that

NM–PSO-curve needs the way much time to fit

the curves. Nonetheless, the results of Tables 2

Page 10: Optimal multi-thresholding using a hybrid optimization approach

Fig. 5. Bi-level thresholding of screw image (image 3): (a) original image, (b) Otsu or NM–PSO–Otsu, (c) NM–PSO-curve, (d) original

histogram of (a), (e) bi-level threshold of (b), (f) bi-level threshold of (c).

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1091

Page 11: Optimal multi-thresholding using a hybrid optimization approach

Table 4

Experimental result for image 3 by NM–PSO-curve

Image Level Starting point Number of iteration

3 2 (0.50,180.00,210.00,1.00,1.00) 50

Method (P1,l1,l2,r1,r2) H (fitting error) Time (s)

NM–PSO-curve (0.56,180.79,195.13,1.30,13.88) 0.0039 22.78

T 184

1092 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

and 3 reveal that NM–PSO-curve can fit the orig-

inal histogram extremely well, with the mean

square error all smaller than 0.003. This renders

a degree of accuracy in practical applications,

which is unattainable with the Otsu�s method. It

indicates that NM–PSO-curve may serve as a

potential candidate for dealing with the threshold-

ing situations where the Otsu�s method fails toyield acceptable outcomes. The following illus-

trates the difficulties encountered by using the

Otsu�s method.

Fig. 5(b) and (c) show the bi-level thresholding

results of a screw image obtained by utilizing NM–

PSO–Otsu (or the Otsu�s method) and NM–PSO-

curve. From a visualization perspective, the qual-

ity of the segmented image resulting from applyingNM–PSO-curve is superior to that by NM–PSO–

Otsu. Looking at the original image in Fig. 5(a),

there displays a low-intensity contrast between

the screws and background, making the Otsu�smethod unable to expose the screw on the right

and the screw threads clearly. On the contrary, in

Fig. 5(c), the contour of the screws is perfectly

identified and then rehabilitated. The very differentoptimal threshold values of 210 (returned by the

Otsu�s method) and 184 (returned by NM–PSO-

curve) dictate a considerable disparity in segmen-

tation quality between these two thresholding

techniques. Fig. 5(f) and Table 4 illustrate that

NM–PSO-curve nicely fits the original histogram,

with a mean square error of merely 0.0039.

Fig. 6(b) and (c) show tri-level thresholding re-sults of a PCB image using the Otsu�s method and

NM–PSO-curve. Overall, these two methods re-

turn similar results except in the areas of pinholes.

Notice that, due to a tighter threshold value T2 at

125 (as compared to 178 in NM–PSO–Otsu), the

object sizes of the pinholes, electric circuits and

background are completely preserved for a clear

presentation by NM–PSO-curve in a consistent way.

Hence, it can be concluded that the segmentation

difficulty by using the Otsu�s method may arise,

as the size of objects relative to the background

is small. From Fig. 6(f) and Table 5, smoothly fitted

histograms are formed and the threshold values

are properly located at the valleys that separatethe pinholes and electric circuits from the back-

ground, with a mean square error of 0.0031.

Table 1 shows that NM–PSO–Otsu and the

Otsu�s method converge in the same amount of

time for bi-level thresholding in image 3, but for

image 4, NM–PSO–Otsu outperforms the Otsu�smethod in computation time for multi-level thres-

holding. For tri-level case, the computation time ofthe Otsu�s method takes about 18 times as much as

NM–PSO–Otsu. This implies that NM–PSO–Otsu

can likely be applied in real-time for multi-thres-

holding in that it only takes about 0.015s for

three-level thresholding. Yet, in images 3–4 the

efficiency is apparently not our major concern, so

NM–PSO-curve is recommended for maintaining

quality outputs despite its expensive computation.If the speed is of primary importance rather than

the effectiveness for a particular application (as

previously discussed in images 1 and 2), thus

NM–PSO–Otsu should be adopted.

5. Conclusions

Although the Otsu�s method is very efficient in

bi-level thresholding cases, its computation time

becomes aggravated in the case of multi-level

thresholding. To make the Otsu�s method more

practical in on-line object segmentation, we

have proposed a faster searching scheme called

Page 12: Optimal multi-thresholding using a hybrid optimization approach

Fig. 6. Tri-level thresholding of a PCB image (image 4): (a) original image, (b) Otsu or NM–PSO–Otsu, (c) NM–PSO-curve, (d)

original histogram of (a), (e) tri-level threshold of (b), (f) tri-level threshold of (c).

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1093

Page 13: Optimal multi-thresholding using a hybrid optimization approach

Table 5

Experimental result for image 4 by NM–PSO-curve

Image Level Starting point Number of iteration

4 3 (0.33,0.33,60.00,100.00,150.00,1.00,1.00,1.00) 80

Method (P1,P2,l1,l2,l3,r1,r2,r3) H (fitting error) Time (s)

NM–PSO-curve (0.18, 0.75, 58.90, 101.87, 141.10, 8.95, 8.66, 20.30) 0.0031 79.72

T1, T2 78, 125

1094 E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095

NM–PSO–Otsu that solves the Otsu�s method

objective function using a hybrid optimization

method. Experimental results show that NM–

PSO–Otsu can accelerate the Otsu�s method inmulti-level thresholding for real-time applications

and does not degrade the quality of image segmen-

tation. In addition, we have presented an off-line

object segmentation method called NM–PSO-

curve. Comparison results of the Otsu�s method

to NM–PSO-curve demonstrate that NM–PSO-

curve offers higher quality in visualization, object

size and contrast of image segmentation, particu-larly when the image has a complex structure or

the contrast between the object and background

is vague. Not surprisingly, NM–PSO-curve incurs

higher computation time than the Otsu�s method

since curve fitting needs to search for the optimum

values of more parameters. It is clearly seen from

the experimental results that the presented meth-

ods that incorporate hybrid optimization tech-niques successfully avoid local minima and are

extremely easy to implement in practice as they

only minimize the associated objective function.

In closing, NM–PSO–Otsu is a promising and

viable tool for on-line object segmentation in mul-

ti-thresholding due to its computational efficiency,

and NM–PSO-curve proves to be effective for off-

line object segmentation in multi-thresholding dueto its quality performance.

Acknowledgement

The authors would like to thank two anony-

mous referees for their comments that improved

the representation and quality on an earlier draftof this paper. Dr. Fan is partially supported by a

National Science Council grant NSC 93-2213-E-

155-009.

References

Cheng, H.D., Chen, Y.H., Jiang, X.H., 2000. Thresholding

using two-dimensional histogram and fuzzy entropy princi-

ple. IEEE Trans. Image Process 9 (4), 732–735.

Eberhart, R.C., Shi, Y., 2001. Tracking and optimizing

dynamic systems with particle swarms. In: Proc. Congress

on Evolutionary Computation. Seoul, Korea, pp. 94–97.

Fan, S.K., Zahara, E., 2002. A hybrid simplex search and

particle swarm optimization for unconstrained optimiza-

tion. In: Proc. 32nd Internat. Conf. on Computers and

Industrial Engineering, Limerick, Ireland.

Glasbey, C.A., 1993. An analysis of histogram-based thres-

holding algorithms. CVGIP: Graphical Models Image

Process. 55, 532–537.

Gonzalez, R.C., Woods, R.E., 2002. Digital Image Processing.

Prentice Hall, Upper Saddle River, NJ.

Hu, X., Eberhart, R.C., 2001. Tracking dynamic systems with

PSO: Where�s the cheese? In: Proc. Workshop on Particle

Swarm Optimization, Indianapolis, IN, USA.

Kapur, J., Sahoo, P., Wong, A., 1985. A new method for gray-

level picture thresholding using the entropy of the histogram.

Comput. Vision, Graphics, Image Process. 29, 273–285.

Kennedy, J., Eberhart, R.C., 1995. Particle Swarm Optimiza-

tion. In: Proc. IEEE Internat. Conf. on Neural Networks,

Piscataway, NJ, USA, 1942–1948.

Li, C.H., Lee, C.K., 1993. Minimum cross entropy threshold-

ing. Pattern Recognition 26, 617–625.

Nelder, J.A., Mead, R., 1965. A simplex method for function

minimization. Comput. J. 7, 308–313.

Olsson, D.M., Nelson, L.S., 1975. The Nelder–Mead simplex

procedure for function minimization. Technometrics 17, 45–51.

Otsu, N., 1979. A threshold selection method for gray-level

histogram. IEEE Trans. Systems Man Cybernet. 9, 62–66.

Renders, J.M., Flasse, S.P., 1996. Hybrid methods using genetic

algorithms for global optimization. IEEE Trans. Systems

Man Cybernet. Part B: Cybernet. 26, 243–258.

Sahoo, P., Soltani, S., Wong, A., 1988. A survey of thresholding

techniques. Comput. Vision, Graphics, Image Process. 41,

233–260.

Spendley, W., Hext, G.R., Himsworth, F.R., 1962. Sequential

application of simplex designs in optimization and evolu-

tionary operation. Technometrics 4, 441–461.

Synder, W., Bilbro, G., Logenthiran, A., Rajala, S., 1990.

Optimal thresholding—a new approach. Pattern Recogni-

tion Lett. 11, 803–810.

Page 14: Optimal multi-thresholding using a hybrid optimization approach

E. Zahara et al. / Pattern Recognition Letters 26 (2005) 1082–1095 1095

Weszka, J., Rosenfeld, A., 1979. Histogram modifications for

threshold selection. IEEE Trans. Systems Man Cybernet. 9,

38–52.

Yen, J.C., Chang, F.J., Chang, S., 1995. A new criterion for

automatic multilevel thresholding. IEEE Trans. Image

Process 4 (3), 370–378.

Yen, J., Liao, J.C., Lee, B., Randolph, D., 1998. A hybrid

approach to modeling metabolic systems using a genetic

algorithm and simplex method. IEEE Trans. Systems Man

Cybernet.—Part B: Cybernet. 28, 173–191.

Yin, P.Y., 1999. A fast scheme for optimal thresholding using

genetic algorithms. Signal Process. 72, 85–95.