a non-dominance-based online stopping criterion for multi-objective evolutionary algorithms

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERINGInt. J. Numer. Meth. Engng 2010; 84:661–684Published online 4 May 2010 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/nme.2909

A non-dominance-based online stopping criterion formulti-objective evolutionary algorithms

Tushar Goel∗,† and Nielen Stander

Livermore Software Technology Corporation, Livermore, CA, U.S.A.

SUMMARY

A non-dominance criterion-based metric that tracks the growth of an archive of non-dominated solutionsover a few generations is proposed to generate a convergence curve for multi-objective evolutionaryalgorithms (MOEAs). It was observed that, similar to single-objective optimization problems, there weresignificant advances toward the Pareto optimal front in the early phase of evolution while relatively smallerimprovements were obtained as the population matured. This convergence curve was used to terminate theMOEA search to obtain a good trade-off between the computational cost and the quality of the solutions.Two analytical and two crashworthiness optimization problems were used to demonstrate the practicalutility of the proposed metric. Copyright � 2010 John Wiley & Sons, Ltd.

Received 9 October 2009; Revised 4 March 2010; Accepted 12 March 2010

KEY WORDS: convergence; stopping criterion; multi-objective; evolutionary algorithm; crashworthiness;genetic algorithms

1. INTRODUCTION

Most practical engineering problems involve multiple design objectives and constraints. The opti-mization of such systems with more than one objective function is called multi-objective optimiza-tion. These objectives are often in conflict. Contrary to the single-objective optimization problem(SOP), the multi-objective optimization problem (MOP) does not result in a single optimum solu-tion. Instead, it results in a set of optimum solutions that represent different trade-offs among theobjectives. These solutions are known as Pareto optimal solutions or constitute the Pareto optimalsolution set [1]. The function space representation of the Pareto optimal solution set is known asthe Pareto optimal front (POF). The most common strategy to find Pareto optimal solutions is toconvert the MOP to a SOP and then to find a single trade-off solution. There are multiple waysof converting a MOP to a SOP, namely, the weighted sum strategy, inverted utility functions, goal

∗Correspondence to: Tushar Goel, Livermore Software Technology Corporation, Livermore, CA, U.S.A.†E-mail: [email protected]

Copyright � 2010 John Wiley & Sons, Ltd.

662 T. GOEL AND N. STANDER

programming, ε-constraint strategy, desirability index, etc. [1–5]. This biggest drawback of theseconversion strategies is that each optimization simulation results in a single trade-off. Besides,multiple runs may not yield sufficiently diverse trade-off solutions [2].

Genetic algorithms (GAs) have been demonstrated to efficiently solve MOPs because they resultin a diverse set of trade-off solutions in a single simulation run [2]. However, the GA typicallyrequires a large number (tens of thousands) of simulations to converge to the global POF. Thougha recent study highlighted the application of a multi-objective GA with a relatively small numberof simulations [6], the high computational cost remains the biggest potential drawback for solvingengineering problems that may involve impact or other expensive simulations to analyze theproblem.

Clusters of processors have become increasingly common due to a continual reduction in thecost of the hardware. Such systems are particularly useful for running GAs that are inherentlysuited to parallelization. One can reduce the clock time significantly by concurrently running manyindividuals (simulations) in a GA population. The time to analyze each design can be reducedfurther if the analysis code can utilize multiple processors as, for instance, is the case with theMPP version of LS-DYNA� [7] used in this study. While this two-level parallelization has amultiplicative effect on the clock time-savings, the ever-increasing complexity of computationalmodels may easily offset the potential gains of parallelization [8].

Most recent efforts in reducing the computational cost have been focused on developing moreefficient algorithms (e.g. micro GA [9], ParEGO [10]), improving efficiency of existing algorithms[11], using a combination of local search methods with GA [12–14], or using approximationmodels [15–22] for fitness evaluation, but the influence of a stopping criterion on the computationalcost of the GA has largely been overlooked. The maximum number of generations is by far themost popular termination criterion for both single- and multi-objective GA. There have been someefforts in developing the bounds on the number of generations for single-objective GAs [23–25].Some early research work [26–29] also provided conditions for convergence for the multi-objectiveevolutionary algorithms (MOEA).

This paper attempts to reduce the total computation time for solving industrial MOPs byexploiting the convergence properties of the MOEA [30]. It is well known that for single-objectiveoptimization, there is a considerable improvement in the quality of solutions in the initial phasesof the GA simulation and incremental improvements are observed later on. One can terminatethe GA search once a reasonable improvement is obtained thus saving a significant computationalexpense required to converge to the global optimal solution. In the last few years, researchers havestarted exploring efficient stopping criteria for MOEAs [30–37].

Rudenko and Schoenauer [31] proposed the standard deviation of maximum crowding distancecriterion to detect stabilization of the NSGA-II algorithm for two objective optimization problems.Marti et al. [32, 37] estimated the rate of improvement (MGBM criterion) using the measuredrate of improvement based on the ratio of dominated solutions in a population as well as Kalmanfilters. The simulations were stopped when the estimated rate fell below the threshold value. Whilethis measure performed reasonably on low-dimension problems, the authors noted difficulty witha 10-objective problem. This measure required a user-defined parameter to control the estimatedrate of improvement.

Trautmann et al. [33] used the Kolmogorov–Smirnov test to statistically ascertain the conver-gence by comparing the generational distance [38], hyper-volume [39], and spread [40] metricsfor a few generations. In a follow-up paper, Wagner et al. [34] advanced this idea to develop

Copyright � 2010 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Engng 2010; 84:661–684DOI: 10.1002/nme

A NON-DOMINANCE-BASED ONLINE STOPPING CRITERION 663

an online convergence detection (OCD) criterion by carrying out �2-variance test and t-test forlinear behavior on different performance metrics. They stopped simulation when either of thetests indicated convergence. Later Naujoks and Trautmann [35] applied the OCD criterion for thedesign of airfoil surfaces that minimize drag at different flow conditions. This method is complexand requires selecting many parameters. Recently, Trautmann et al. [36] compared the online andoffline convergence criterion.

While researchers have assumed that the MOEAs improve the solutions in the early phase ofevolution, this was never demonstrated. In this paper, (i) a convergence curve for MOEA wasdeveloped using a non-domination criterion-based metric and (ii) a simple termination criterionwas developed to stop the GA simulations when a reasonably good approximation of the truePOF was obtained instead of attempting to locate the global POF. Specifically, the elitist non-dominated sorting genetic algorithm (NSGA-II) [41] augmented with an archiving [18] strategy(NSGA-IIa) was used as a representative MOEA. Two analytical problems and two engineeringproblems were used to illustrate the main concepts. The engineering problems were representedby two crashworthiness examples analyzed using the non-linear dynamic finite element analysisprogram LS-DYNA. It was demonstrated that there were significant improvements in the qualityof the local POF in the early stage of simulations. The evolutionary search was terminated usinga stopping criterion based on the novel non-dominance-based convergence metric such that areasonable trade-off in the quality of solutions and the computational cost was found.

The paper is organized as follows. The analytical and crashworthiness optimization test prob-lems along with the details of the simulation setup and performance metrics are furnishedin Section 2. The proposed metrics and the termination criterion are explained in Section 3.The main results of this study are documented in Section 4, and major findings are summarized inSection 5.

2. TEST PROBLEMS AND PERFORMANCE METRICS

This section lists analytical and industrial test problems used to illustrate the application of proposedmetrics as termination criterion. The numerical setup and performance metrics are also listed infollowing subsections.

2.1. Test problems

The selected test problems were chosen carefully to strike a balance between the analytical bench-marks and industrial problems while considering the important features encountered in practicalmulti-objective optimization e.g. multiple constraints, non-convex Pareto front, discontinuities inthe Pareto front in function and variable spaces, two or more objectives, etc. The detailed descrip-tion of the salient features and challenges of the two analytical benchmark problems [2] and thetwo crashworthiness problems is given as follows.

2.1.1. Analytical problems. TNK problem [42]Minimize:

f1(x) = x1,

f2(x) = x2,



Subject to:C1(x) ≡ x2

1 +x22 −1−0.1cos(16arctan(x1/x2))�0,

C2(x) ≡ 0.5−(x1−0.5)2 −(x2 −0.5)2�0,

0 � x1, x2��.

This problem results in a non-convex and discontinuous global POF that coincides with theboundary of the first constraint. It is important to note that not all the points on the constraintboundary are part of the global POF. This problem poses difficulty in finding diverse solutions onthe Pareto front.

OSY Problem [43]Minimize:

f1(x) = −[25(x1−2)2 +(x2 −2)2 +(x3 −1)2 +(x4 −4)2 +(x5 −1)2]

f2(x) = x21 +x2

2 +x23 +x2

4 +x25 +x2

6 ,

Subject to:

C1(x) ≡ x1 +x2 −2�0,

C2(x) ≡ 6−x1 −x2�0,

C3(x) ≡ 2+x1 −x2�0,

C4(x) ≡ 2−x1 +3x2�0,

C5(x) ≡ 4−(x3−3)2 −x4�0,

C6(x) ≡ (x5 −3)2 +x6 −4�0,

0 � x1, x2, x6�10, 1�x3, x5�5, 0�x4�6.

The POF for this problem has five connected linear sections that coincide with the boundariesof different constraints. This problem poses difficulties in finding all sections of the POF becausesubstantial changes in the variable space are required to move from one section of Pareto frontto another in function space while maintaining the constraint feasibility. The problem also resultsin a non-uniform distribution of the Pareto optimal solutions. It is also noteworthy that while theindividual sections of the Pareto front are linear, the overall shape of the Pareto front is non-convex.

The function space representation of the shape of the global POF for the two analytical problemsis shown while discussing the results.

2.1.2. Crashworthiness problems. The knee-bolster design problem had three objective functionswhereas the multi-disciplinary optimization problem handled crashworthiness and NVH (noise,vibration, and harshness) simulations. The computational requirements to evaluate functions forthese two industrial problems were non-trivial and clusters of computers were used to coordinatethe simulations. The two industrial problems are described next.



Figure 1. Automotive instrument panel with knee bolster system used for knee-impact analysis.(Courtesy: Ford Motor Company.)

Gauge

Radius

Gauge

Width

Width

DepthDepth

Depth

Width

Gauge Radius

Gauge

Radius

Gauge

Width

Width

DepthDepth

Depth

Width

Gauge Radius

Figure 2. Design variables of the knee bolster system.

Knee Bolster Design: The first crashworthiness problem employs a finite element simulationof a typical automotive instrument panel (IP) (shown in Figure 1) impacting the knees [44]. Thespherical objects that represent the knees move in the direction determined from prior physicaltests. The IP comprises a knee bolster that also serves as a steering column cover with a styledsurface, and two energy absorption brackets attached to the cross vehicle IP structure.

A significant portion of the lower torso energy of the occupant is absorbed by appropriatedeformation of these brackets. The wrap-around of the knee around the steering column is delayedby adding a device, known as the yoke, to the knee bolster system. The shape of the brackets andyoke is optimized without interfering with the styled elements. The 11 design variables are shownin Figure 2 and the ranges are given in Table I. To keep the computational expense low, only the



Table I. Design variables used to simulate knee-impact of automotiveinstrument panel structure.

Name Lower Baseline Upper

L-Bracket gauge 0.7 1.1 3.0T-Flange depth 20.0 28.3 50.0F-Flange depth 20.0 27.5 50.0B-Flange depth 15.0 22.3 50.0I-Flange width 5.0 7.0 25.0L-Flange width 20.0 32.0 50.0R-Bracket gauge 0.7 1.1 3.0R-Flange width 20.0 32.0 50.0R-Bracket radius 10.0 15.0 25.0Bolster gauge 1.0 3.5 6.0Yolk radius 2.0 4.0 8.0

Table II. Design constraints for the Knee-impact analysis problem.

Upper bound

Kinetic energy 154 000Yoke displacement 85

driver side IP was modeled using approximately 25 000 elements; and the crash was simulated for40 ms, by which time the knees had been brought to rest.

The design optimization problem accounting for the optimal occupant kinematics was formulatedas follows:

Minimize:

Maximum knee force,Average knee displacement or intrusion, andMass

Subject to the constraints represented in Table II.All responses were scaled. Knee forces were the peak SAE filtered (60 Hz) forces, whereas

all the displacements were represented by the maximum intrusion. LS-DYNA [7] was used tosimulate different designs. Each simulation requires approximately 20 min on a dual-core IntelXeon (2.66 GHz) processor with 4 GB memory.

Multi-disciplinary optimization (MDO) problem: The performance of a National Highway TrafficSafety Administration (NHTSA) vehicle was optimized for crashworthiness. The full frontal impactcrash was simulated using a finite element model containing approximately 30 000 elements, shownin Figure 3(a). A modal analysis was performed on a so-called ‘body-in-white’ model that hadapproximately 18 000 elements. The vibration model is depicted in Figure 3(b) in the first torsionalvibration mode.

The tracking methodology applied to the torsional mode was described in the paper by Craiget al. [45]. The design variables represented gauges of the structural components in the enginecompartment of the vehicle (Figure 4). Seven gauge variables, namely apron, rail-inner, rail-outer,shotgun-inner, shotgun-outer, cradle rail, and cradle cross member were selected to optimize the



Figure 3. Finite element models for the multi-disciplinary optimization problem:(a) crash model and (b) vibration model.

Figure 4. Components affected by thickness design variables (with exploded view).

performance. Twelve parts comprising aprons, rails, shotguns, cradle rails, and the cradle crossmember were affected by selected design variables. LS-DYNA [7] was used for both the crashand modal analysis simulations, in explicit and implicit analysis modes, respectively.

The vehicle performance is characterized by the intrusion, stage pulses, mass, and torsionalfrequency. A multi-disciplinary, MOP is formulated as follows:

Minimize:

Mass, andIntrusion

Subject to the constraints listed in Tables III and IV.The bounds and the baseline design thickness values on the design variables are specified in

Table IV. The three-stage pulses were calculated from the SAE filtered (60 Hz) acceleration x anddisplacement x of a left rear sill node in the following fashion:

Stage j pulse=− k

(d2 −d1)

∫ d2

d1

x dx, k =0.5 for j =1, k =1.0 otherwise



Table III. Design constraints for the MDO problem.

Lower bound Upper bound

Maximum intrusion (xcrash) — 551.27 mmStage 1 pulse (xcrash) 14.512g —Stage 2 pulse (xcrash) 17.586g —Stage 3 pulse (xcrash) 20.745g —Torsional mode frequency (xNVH) 38.27 Hz 39.27 Hz

Table IV. Starting values and bounds on the design variables of the MDO problem.

Name Lower Baseline Upper

Rail inner 1.0 2.0 3.0Rail outer 1.0 1.5 3.0Cradle rails 1.0 1.93 3.0Aprons 1.0 1.3 2.5Shotgun inner 1.0 1.3 2.5Shotgun outer 1.0 1.3 2.5Cradle cross member 1.0 1.93 3.0

with the limits (d1;d2)= (0;184); (184;334); (334;Max(x)) for i =1,2,3, respectively. Alldisplacement units were given in millimeter and the minus sign converted acceleration todeceleration.

In summary, the optimization problem aims to minimize the mass and intrusion without compro-mising the crash and vibration criteria. The objectives and constraints were scaled using the targetvalues to balance the violations of the different constraints. Each crash simulation took about3400–3900 s on one core of the IBM x3455 machine and the modal analysis took about 40 s.

2.2. Numerical setup

All problems were simulated using the LS-OPT� [46] optimization software. For all test prob-lems, the NSGA-IIa algorithm with the binary tournament selection operator, SBX crossover, andpolynomial mutation operators in real-coded space was used as the representative MOEA. The GAsimulation parameters were chosen based on the experience with similar problems and guidelinessuggested in the literature [2]. The numerical values of chosen simulation parameters are docu-mented in Table V. The optimal use of available computational resources was also considered whilechoosing the population size and the number of simulations for the crashworthiness problems.Since the function evaluation expense of the analytical problems was negligible, a single processorwas used for simulations. On the other hand, the computational cost of analyzing crashworthinessproblems was quite high so two different clusters of processors were used.

The analyses for the MDO problem were run on the x3455 cluster of the IBM PoughkeepsieBenchmark Center. Each node of this cluster had two Dual-Core AMD Opteron(TM) 2220 SEprocessors with a clock rate of 2.8 GHz. This cluster had a total of 40 nodes (160 cores). Thejob distribution was handled by the queuing system Tivoli Workload Scheduler LoadLeveler�.Since the chosen population size was 80 in this experiment, LS-OPT submitted 80 crash jobs



Table V. NSGA-II parameters for different problems. Mutation probability (Pmut)∼1/Nv

where Nv is the number of variables.

Pop. Size Max. Gen Pcross Pmut DIcross DImut

TNK 100 200 1.00 0.50 10 100OSY 100 300 1.00 0.17 10 100KNEE 80 120 1.00 0.10 20 20MDO 80 100 0.99 0.15 5 5

and 80 NVH jobs to LoadLeveler at each generation, and the LoadLeveler arranged all jobs tooptimize the use of the computing resource. The smaller of the two crashworthiness problems, theKnee-bolster design problem was simulated on the in-house cluster of 16 nodes. Each node had asingle Dual-Core Intel Xeon processor with a clock rate of 3.6 GHz (32 cores). A shared memoryof 16 GB was available for all processors. The Sun Grid Engine� v 6.1 queuing system was usedfor job scheduling.

2.3. Performance metrics

The multi-objective optimization results in a set of candidate optimal solutions that span thefunction space. Special metrics used to quantify convergence and diversity of the solution sets aredescribed as follows:

(a) Convergence: The sets of non-dominated solutions‡ [2] from various generations arecompared by computing the number of dominated solutions in each set using a weaknon-domination criterion. A small number of dominated solutions indicate better qualityof solutions. There is no dominated solution in the compared sets of solutions when thesimulation has converged to the POF.

(b) Diversity: The diversity of a non-dominated solution set is determined using two criteria: (i)how wide-spread the trade-off solutions are (spread), and (ii) how uniformly the trade-offfront is sampled (uniformity). The two diversity measures are estimated as follows:

(i) Spread [39]: The spread of the front is calculated as the diagonal of the largest hypercubein the function space that encompassed all points. A large spread is desired to find diversetrade-off solutions. The spread measure is derived using the extreme solutions makingit susceptible to the presence of a few isolated points that could artificially improve thespread metric. An equivalent criterion might be the volume of such a hypercube.

(ii) Uniformity measure [41]: This complimentary criterion (to the spread metric) detectsthe presence of poorly distributed solutions by estimating how uniformly the points aredistributed in the Pareto optimal set. The uniformity measure ([2, 41]) is defined as,

�=N∑

i=1

|di − d|N

, d = 1

N

N∑i=1

di (1)

‡A non-dominated solution is not dominated by any other solution in the set. A solution is considered to be dominatedif it is not better than comparing solution in all objectives and is worse in at least one objective.



where di is the crowding distance§ [2] of the solution in the function or variable space.The boundary points are assigned a crowding distance of twice the distance to the nearestneighbor. This measure is similar to the standard deviation of the crowding distance andhence a small value of the uniformity measure is desired to achieve a good distribu-tion of points.

3. PROPOSED METHODOLOGY

The most commonly used termination criteria for multi-objective evolutionary methods are themaximum number of generations or the maximum number of function evaluations. Typically, thesecriteria are chosen conservatively to obtain convergence resulting in a large number of functionevaluations which is rarely feasible for industrial problems that often require large computationtime to evaluate a design. On the other hand, too few simulations are likely to result in a poor localoptimal solution set. Hence, there exists a trade-off between the computational cost (number offunction evaluations) and the quality of final solutions. In this paper, a non-dominance metric-basedstopping criterion is developed to find a good trade-off by exploiting the convergence propertiesof the MOEAs.

It has been demonstrated for the single-objective GAs that significant improvements are achievedin early stages and relatively smaller improvements are obtained in later generations such that aneffective termination criterion could be developed to save computational resources. Intuitively, itcan be said that the similar behavior would be observed for the multi-objective GAs i.e. significantadvances toward the global Pareto optimal solutions would be made in the earlier generations(rapid evolution phase) and relatively smaller improvements would be made in the later generations(slow local search phase).

While the fitness of the objective function is a sufficient criterion to assess the improvementin single-objective optimization, the quantification of improvements in a multi-objective opti-mization is difficult because the optimization results in a set of solutions. The improvementsin the quality of solutions have to be measured using two criteria, convergence to the globalPOF and the diversity of the solutions [2]. A number of metrics are proposed in the liter-ature to quantify the performance of MOEAs, namely, error-ratio, generational distance [38],maximum spread, hypervolume [39], spacing [41, 47], etc. A detailed description of the prop-erties of these metrics can be found in the literature [2, 48]. It is noted that these metrics havebeen developed to analyze the convergence to the global Pareto front and they often requireprior problem information (e.g. reference point for hypervolume computation, global POF forerror ratio and generational distance) that might not be available for industrial problems henceproblem-independent convergence metrics are desired. Consequently, problem-independent non-dominance-based convergence metrics are proposed in this paper that could be used to trackthe performance of MOEA during the evolution process i.e. develop a convergence curve forMOEAs.

§Crowding distance is defined as half the perimeter of the largest hypercube around a point that does not encompassany other solution.



3.1. Non-dominance-based convergence metrics

In the proposed approach, an external archive (Ai ) of non-dominated solutions at the i th generationis maintained. This archive is updated after each generation: (i) to remove the solutions that aredominated by the newly evolved solutions; and (ii) to add new solutions that remain non-dominatedwith respect to the archive. Duplicate copies of the non-dominated solutions are eliminated.

To assess the performance of a MOEA simulation at the i th generation, the external archives ata generation gap of � (Ai and Ai−�) are compared to get the following metrics.

3.1.1. Consolidation ratio (CR). This represents the fraction of archive Ai that has evolved upto the i −�th generation. This is computed as the ratio of the number of members in archiveAi−� that are also present in the archive Ai (non-dominated solutions) to the size of archive Ai .Mathematically,

CR= n(S)

n(Ai ), S ={ai−� :ai−�⊀ai }, ai−� ∈ Ai−�, ai ∈ Ai ,

where n(B) is the size of set B.This metric represents the proportion of potentially converged solutions in the archive. In the

early phase of a MOEA simulation, a large fraction of the non-dominated solutions in the archiveAi−� would be dominated by the solutions in archive Ai due to evolution, thus resulting in a smallfraction of surviving solutions i.e. small value of the CR. However, significantly better solutionsevolve in the later phases such that a large proportion of solutions in the archive Ai−� remainnon-dominated with respect to new solutions leading to a high CR. In the limiting case, the CRapproaches one.

3.1.2. Improvement ratio (IR). This represents the fraction of archive Ai−� dominated by the newsolutions in archive Ai . This is computed as the ratio of the number of members in archive Ai−�that are dominated by the solutions in archive Ai (dominated solutions) to the size of archive Ai .Mathematically,

IR= n(Q)

n(Ai ), Q ={ai−� :ai−� ≺ai }, ai−� ∈ Ai−�, ai ∈ Ai .

The archive Ai includes all non-dominated members of archive Ai−� so no member of thearchive Ai is dominated. The IR quantifies the extent of improvement in the quality of evolvedsolutions. This metric has a high value in the early phase of simulation which gradually convergesto zero when convergence is achieved. It is noted that the IR is similar to the error ratio metric,proposed by van Veldhuizen [38], which calculates the percentage of the non-dominated solutionset that was not part of the global POF. The main difference with respect to the van Veldhuizenmetric is that the error ratio required a priori knowledge of the global POF, whereas the IR doesnot require such information.

Since the evolution is locally random, the parameter � (generation gap) acts as a filter to reducenoise in the CR or the IR metrics. Without any filter (�=1) or with small values of �, the noiseis not filtered adequately which might lead to premature termination of a MOEA simulation whenthe search is stalled for a brief period. On the other hand, a very large value of � is also not desiredbecause it overemphasizes the improvements and would not be able to terminate the search fastenough leading to wasteful simulations.



Figure 5. Consolidation ratio and improvement ratio metrics for the OSY problem:(a) consolidation ratio and (b) improvement ratio.

A sample calculation of the two metrics with different generation gaps for the OSY test problemis shown in Figure 5. Similar results are obtained with the other test problems and hence eliminatedfor brevity. It is obvious from Figure 5 that the archive population stabilizes as the solutionconverges. The IR approaches zero as the sizes of new and old archives become comparablei.e. the CR approaches unity. The IR and the CR do not reach exact zero or unit values, respectively,due to numerical precision limitations. Nevertheless, a small value of the IR or a high value of theCR indicate substantial improvement in the quality of solutions in the archive and hence may beconsidered indicative of convergence. Of the two metrics, the IR is noisier than the CR; and hencethe latter is chosen as a robust metric to detect convergence.

It is also observed from Figure 5 that both metrics are strongly influenced by the locally randombehavior of evolution when the generation gap is small (�=1). The effect of noise is filtered byincreasing the generation gap �. While a small generation gap does not filter the noise adequately,a large generation gap is also not desired because it requires too many simulations to evaluateconvergence. A generation gap of 10 provides a reasonable trade-off between noise filtering andthe convergence rate for all examples hence �=10 is recommended.

3.2. CR as stopping criterion

The CR-vs-generation curve is similar to the convergence curve obtained for a single-objectiveGA with a clear ‘knee’ formation that separates the rapid convergence to the global POF basinand the slow local search regimes. It is proposed that the CR metric be used to terminate theMOEA when the evolution reaches the local search regime. Of course, early termination does notallow the search to find the global POF but results in a good trade-off between the computationalcost and a reasonable approximation of the global POF. The proposed stopping criterion can beimplemented for any MOEA in the following two ways:

3.2.1. Fixed threshold (CRt ). A MOEA simulation can be stopped when the CR reaches a user-defined threshold (CRt ) value e.g. when nearly two-thirds of the current archive Ai is composed ofthe solutions from archive Ai−� or the CR is greater than 0.66. Of course, the choice of the exact



cut-off value may depend on the problem and generation gap. A larger threshold is recommendedfor a smaller �.

3.2.2. Dynamic convergence—utility function-based approach. In this method, the utility of anadditional generation is measured and the search is terminated when the utility falls below thethreshold (Ut ). The average utility of the i th generation Ui is estimated as the change in the CRover a generation interval, given as,

Ui = CRi −CRi−�

�.

There are a few considerations to this approach:

(a) Owing to rapid evolution, the CR may be very small (near to zero) in the very beginning andwould incorrectly result in a small or negative generational utility Ui . This can be avoided byevaluating Ui only when CR>0.5. The value of 0.5 is chosen to allow the search to establisha rapid evolution phase and to approach the knee of the convergence curve.

(b) While �>1 is useful to filter high-frequency noise in evolution, low-frequency noise maystill cause premature termination of the MOEA simulation. Hence the search is terminatedwhen the running average generation utility U∗

i falls below a threshold value (Ut ) where

U∗i = Ui +Ui−�

2.

Although this approach adds computational expense, the probability of premature terminationis substantially reduced.

(c) The choice of utility threshold (Ut ) is very important. A high threshold value would causeearly termination while a low threshold would negate the advantage of using this stoppingcriterion. Fortunately, the convergence curve can be used to estimate Ut . It is proposed thatthe utility threshold be defined as a fraction of the initial utility Uinit, i.e.

Ut =Uinit/F,

where F (>2) is the user-controlled fraction (see note¶ ). The initial utility is estimated basedon the number of generations G taken to achieve a 50% CR (CR�0.5), i.e.

Uinit =CR|�0.5/G.

For example, it took 90 generations to reach CR=0.55 with �=10 (Figure 5). Thus,

Uinit = 0.55

90=0.006.

The rationale behind using CR�0.5 is the same as given earlier, that is it allows the searchto settle initial noise and provides a reasonable estimate of the initial utility.

The choice of threshold CR for the fixed threshold case or the F for the dynamic criterion isnot critical in industrial setups where each analysis takes much longer (h) than the data-processing

¶Mathematical interpretation of parameter F =5 is that the average utility of each generation has fallen below 80%of the initial average utility of each generation.



using GAs (s). Besides, the use of computer clusters is regulated to restrict a single user takingall the processing capability for long periods. So the user can start with a moderate value (F∼5,

CRt =0.80) and if the results obtained are unsatisfactory, the user can iteratively restart the MOEAsimulation by increasing F or CRt until the desired level of convergence is obtained. Based onnumerical experimentation, CRt =0.80 and F =10 were found to be reasonable values for thresholdparameters.

The metrics proposed here are not affected by the number of variables, scaling of objectives,and do not require a priori problem information. These normalized metrics can be used to comparedifferent algorithms for convergence. These metrics do not explicitly account for the improvement indiversity (increasing spread or uniform distribution). However, this issue could be easily addressedby setting a higher threshold for convergence.

3.3. Differences with other stopping criteria

The main differences in the proposed stopping criterion and those available in literature are:

(i) While all other criteria [31–37] have used population data from each generation, the proposedcriterion maintains an external archive‖ of non-dominated solutions at each generation thatavoids the issue of Pareto-drift [18]. The archive also significantly reduces the noise inthe non-dominated solution data and provides robust results. The memory requirement andcomputational cost of maintaining the external archive is much smaller than the memoryand computational cost requirement of the function evaluations for industrial problems.

(ii) The proposed criterion uses a non-dominance-based criterion whereas other researchers haveused either diversity-based criteria [31, 33, 34], or dominance-based criteria [32, 37]. Thenon-domination-based criterion is less noisy compared with the dominance-based criterion.

(iii) Other researchers have computed the so-called ‘online stopping criteria’ at each generationbut the proposed criterion is estimated for every few generations to reduce the possibilityof convergence to a local POF and to reduce the effect of noise.

3.4. Implementation of stopping criterion in MOEA

Figure 6 shows the flowchart of a representative MOEA with the proposed termination criterion.The non-shaded elements of the flowchart represent a MOEA with the conventional (maximumiteration) termination criterion and the shaded elements show the application of the CR-basedstopping criterion with the dynamic approach. The fixed threshold CR simplifies the implementationby eliminating the calculation of utility functions.

4. RESULTS

In this section, the optimization results for the four test problems are described in detail.

‖One can also use the population at any generation instead of the archive, but using the archive is recommended.



Initialize population, Set i=0, j=0

Evaluate and rank population

Create an archive with non-dominated

solutions

Create child population

Evaluate and rank child population

Update the archive

Apply elitism (if any)

i=i+1, j= j+1

i Gen

Stop

j

Comparearchives/population at i and i- generation to

get performance metrics and utility function Ui

Ui* Ut

Reset j=0

Yes

Yes

No

Yes

No

No

Evaluate and rank population

Create an archive with non-dominated

solutions

i

jΔ

≤≥

≥ Δ

Figure 6. Flowchart of multi-objective evolutionary algorithm with the proposed termination criterion.

4.1. Convergence of the MOEA simulation

As discussed earlier, the convergence of a MOEA simulation is tracked by the growth of thearchive in Figure 7. The size of the archive (current-total) typically increases with generationas better solutions are evolved. For the MDO problem, the archive size decreases intermittentlybecause the search is still in the evolution phase where significantly improved solutions evolveand dominate many solutions from the archives at previous generations. This phenomenon is morelikely to appear when the size of the archive is smaller than the population size. The number ofarchive solutions that remains non-dominated (old-non-dominated) also increases for all problemsindicating the progress toward the global POF. The proportion of solutions newly added to thearchive (new-non-dominated) is expectedly high in the initial phase, and it stabilizes with theincrease in generations. As expected, the proportion of old archive member solutions that aredominated by the new individuals (old-dominated) reduces with evolution. These results indicatethe stabilization of the population with generations in terms of convergence.

To further quantify the convergence to the global POF, the CR and the IR metrics proposed hereare plotted in Figure 8. The CR, indicative of surviving old archive members, increases sharplyin the initial phase indicating significant improvements in the convergence characteristics of thearchive. As the evolution reaches a large number of generations, the CR stabilizes at a very highproportion (approx 90%). This ‘knee-shaped’ convergence curve for MOEAs is similar to thatobserved for single-objective GAs. The MOEA simulation can be stopped soon after the formationof the knee in the CR curve to achieve significant cost reduction. A value of 0.8 for the CR isobserved to be a good default to locate the knee.

The IR, that is the ratio of the number of the previous archive (Ai−�) members dominated bythe new members (and hence discarded) and the total number of current archive members (Ai ),is shown in Figure 8(b). As expected, this ratio is significantly high in the earlier generationswhen evolution resulted in better solutions. The ratio decreases with generations and showed signsof stabilization. The results for the MDO problem are noisier than the other problems, but the



Figure 7. Growth of the archive of non-dominated solutions (NDS). Current-total indicates the totalnumber of NDS at this generation (n(Ai )), Old-dominated is the number of solutions that were part ofthe archive (Ai−�) but are removed from the archive (Ai ) because they were dominated by the newindividuals, Old-non-dominated solutions were members of the archive that remained non-dominatedwith respect to the new solutions, and New-non-dominated solutions are newly found NDS: (a) TNK;

(b) OSY; (c) KNEE; and (d) MDO.

trends support the conclusion that the proportion of dominated solutions decreases with evolution.Compared with the CR metric, the IR results in more variation relative to its desired value.

The results shown in this section clearly demonstrate that (a) similar to the single-objectiveevolutionary algorithms, significant improvements toward convergence are obtained in the earlyphase of MOEAs, and (b) while either of the two metrics, the CR or the IR, can be used to decideon the convergence of the MOEA, the CR is preferred. It is noted that early termination of thesimulation may not result in the global POF but the solution set obtained would likely be a goodtrade-off between the computational cost and the convergence to the global POF.

4.2. Spread

The diversity of the solutions is quantified using the spread of the archive at each generation. Theobjectives for the OSY problem are scaled by 250 and 50 units, respectively, and the objectives



Figure 8. The two convergence metrics proposed in this paper. The consolidation ratio isthe proportion of old solutions that have remained non-dominated in the current generationand the improvement ratio is the proportion of old solutions dominated by new solutions:

(a) consolidation ratio and (b) improvement ratio.

Figure 9. Spread of the candidate Pareto optimal front for different problems.

for the MDO problem are scaled by 0.1 units each to provide equivalence among all problems.The scaling only affects the scale of the curve. It is observed in Figure 9 that the spread of thecandidate POF generally increases in the early stage of evolution and then stabilizes indicatingthe shift of the focus of MOEA to local refinement from global search. This demonstrates thepotential advantages of stopping the search early. The spread should be used in conjunction withother criteria to terminate the search due to its sensitivity to scaling.

Apparently opposite trends for the OSY problem are largely due to the presence of a widelyspread local Pareto front (discussed later). The solutions in this non-dominated set are dominatedby better solutions in the subsequent generations such that the spread decreases. It is also noted



Figure 10. Uniformity measures in function and variable space for allproblems: (a) function space and (b) variable space.

that the optimization search could not find the complete Pareto front for the OSY problem in300 generations. A gradual increase in the spread for this problem is observed because moresolutions are being continuously evolved to traverse the POF. This behavior may not be typicalfor many problems and indicates the potential lack of convergence attributed to early termination.However, the result obviously supports the hypothesis of significant savings in the computationalcost by accepting a small compromise in the quality of solutions for this problem also.

4.3. Uniformity measures

As shown in Figure 10, the uniformity measures in both function and variable spaces decreaseswith generations representing improvement in the distribution of solutions. The low value of theuniformity measure also confirms a good distribution. The noisy behavior of the uniformity measurein the function space for the OSY problem is mainly due to the fact that the extreme solutionsin the different regions of the POF are identified before the entire region is populated with theoptimal solutions. Overall, the trends for the OSY function also agree with the other problems.

4.4. Early termination using CR metric

The results discussed above show that the MOEA simulations can be stopped earlier to achievea substantial computational cost reduction. The threshold value of the CR, CRt =0.8, is usedto evaluate the fixed threshold criterion. To assess the convergence dynamically, the averagegenerational utility function (U∗

i ) is estimated for each problem as shown in Figure 11.Two threshold values for the utility functions (Ut ) are obtained by considering F =5 andF =10 that manifest 80 and 90% reduction in the initial utility (Uinit), respectively, and aredepicted on each graph by dashed and dash-dotted lines. The thick vertical line on eachgraph represents the number of generations (G) taken to satisfy CR�0.5. This has beenused to estimate Uinit. The curve to the right of this vertical line should be used to detectconvergence.

As expected, the average utility of additional generations decreases with generations. This trendis the most prominent for the Knee-bolster design problem. While the global pattern of the utility



Figure 11. Generational utility function for different examples. The vertical line indicates the number ofgenerations to exceed a consolidation ratio of 0.5 such that the curve on the right-hand side of the vertical

line is relevant to track convergence.

function curve suggests reduction in the utility of additional generations, the occasional localincrease in utility is due to finding solutions in a previously unsampled section of the POF for theOSY problem. For the MDO problem, the initial evolution phase is not complete (G =100, upperbound on the number of generations) hence the utility function curve could not be used to deriveany conclusions about the convergence.

The final generation numbers for different test problems using fixed threshold or utility function-based metrics are tabulated in Table VI. The initial convergence generation (G) is problemdependent. All the methods indicate an earlier termination of the search for the analytical andknee examples than the generation limit resulting in a significant computational cost reduction.The dynamic criterion with F =5 terminates the search the earliest while, as expected, F =10requires more generations to terminate the search. The fixed threshold criterion terminates thesearch intermediately.



Table VI. Comparing the final generation numbers obtained fromdifferent stopping conditions proposed here.

CR�0.5(G) CRt =0.8 F =5 F =10 Genmax

TNK 40 110 90 90 200OSY 90 150 140 180 300KNEE 30 100 60 110 120MDO 100 — — — 100

The number of generations taken to achieve CR�0.5 is used to estimate initial utility in the utility function-basedapproach. Genmax is the maximum number of generations.

4.5. Validating the stopping criterion

A comparison of the quality of the solutions in the archive at the predicted stopping generation andcomplete simulation results for all problems is shown in Figure 12. As indicated by the differentmetrics, there are significant improvements in the quality of trade-off solutions in the initial phasefor all the problems.

As predicted, there are significant improvements in the first 40 generations for the TNK problemand the entire POF is traversed after 90 generations (dynamic criterion) for the TNK problem(Figure 12(a)). The number of points on the POF increases by running the simulations longer.Minor improvements in the convergence for some solutions are also noted. Similar to the TNKproblem, the evolution generated large improvements initially (up to 90 generations) for the OSYproblem (Figure 12(b)). The archive obtained after 140 generations (F =5) has five sections butonly two sections ( f1 ranging from −260 to −130 units) are well identified; the third section ( f1ranging from −110 to −40 units) is sparsely approximated and the remaining two sections ( f2ranging from 4 to 28 units, and 50 to 80 units) are far from the global POF. The distribution andconvergence improved when F =10 is used to terminate the search at 180 generations but twosections are still sparsely populated and the one section is far from the POF. Nevertheless, earlytermination using the proposed criterion manifests a reasonable trade-off between the computationalcost and the approximation of POF for the analytical problems.

The pictorial representation of archive solutions at different generations for the knee-bolsterdesign problem is depicted in Figure 12(c). While it is difficult to assess the convergence pictori-ally for this three objectives problem, initial improvements are obvious. A noticeable overlap inthe solutions from the 100th and 120th generations (predicted termination generation) confirms theconvergence to a local POF. The diversity of the solutions is good and the same is noticed in themetrics discussed earlier. The improvements in the convergence of the solutions are also noted inthe first 80 generations for the MDO problem in Figure 12(d). The improvements between the trade-off solutions obtained between the 90th and 100th generations are relatively small∗∗ indicating theprogress toward convergence.

All the results presented here confirm that (i) significant improvements are obtained in the initialphase of evolution, and (ii) the CR-based termination criterion results in a good trade-off betweenthe quality of the solutions and the computational cost.

∗∗The limitations on the computational budget restricted us from running more than 100 generations for the MDOproblem.



Figure 12. Non-dominated solution archives at different generations. Gen-i correspondsto ith generation: (a) TNK; (b) OSY; (c) KNEE; and (d) MDO.

5. SUMMARY AND FUTURE WORK

The issue of computational cost is very important for the application of evolutionary algorithms toindustrial problems, particularly while considering multiple objectives. This issue is addressed inthis paper by developing a convergence curve for multi-objective evolutionary algorithms (MOEAs)using a novel non-dominance-based metric. Later this convergence curve is used to develop aneffective termination criterion.

First, the convergence during a simulation is assessed using a few metrics which tracks thequality of solutions in an archive of non-dominated solutions set obtained at each generation.The CR metric tracks the proportion of the potentially converged POF solutions in an archive,whereas the IR is defined as the proportion of non-dominated solutions from the previous genera-tions that are dominated by the newly evolved solutions. While both metrics are problem indepen-dent, the CR is found to be more robust. The convergence of a MOEA simulation is determinedby considering two termination criteria, (i) when the CR reaches a user-defined threshold and



(ii) when a utility function that estimates the potential improvement due to additional generationsdrops below a threshold value.

The two termination criteria were studied using two analytical problems and two crashworthinessproblems. It was demonstrated that, similar to the single-objective GA, significant improvementsin the convergence and the diversity of the solutions were obtained in the initial phase of evolutionwhile relatively smaller improvements were observed later. A ‘knee’ in the CR-vs-generationscurve suggested trade-off between the computational cost and the non-dominated solution set.Both the fixed threshold and the utility function-based criteria effectively terminated the searchafter significant improvements were obtained. The dynamic termination criterion with F =10performed the best and resulted in approximately 40% reduction in computational cost. As expected,the increasing threshold values for both fixed and dynamic termination criteria required moresimulations. It was argued that this behavior could be exploited in the industrial setups such that theusers could start with a moderate threshold value e.g. F =5 or CRt =0.8, and add more simulationsif the results were unsatisfactory.

While the proposed criterion is demonstrated with the elitist NSGA-II, it is anticipated to beapplicable to any MOEA. Similarly, other convergence metrics like dominated hypervolume canalso be used with the proposed framework to terminate the MOEA search. There are two potentialdrawbacks of the proposed criterion: (a) a dominance-based metric may improperly emphasizeimprovements by ignoring geometric changes, (b) the method may be difficult to apply with thehigh-dimensional problems because the weak non-domination criterion results in too many non-dominated solutions. These problems are anticipated to be overcome by using convergence metricsthat account for both convergence and diversity measures e.g. dominated hypervolume and is thesubject of our ongoing research.

ACKNOWLEDGEMENTS

The authors thank the IBM Poughkeepsie Benchmark Center for providing computing equipment and fortheir strong technical support. Dr Guangye Li from the IBM Deep Blue Computing Group at Houston,TX provided a lot of help with the simulations for the MDO problem. We also thank the anonymousreviewers for their constructive criticism that helped to improve the paper.

REFERENCES

1. Miettenen K. Non-Linear Multi-Objective Optimization. Kluwer: Dordrecht, 1999.2. Deb K. Multi-Objective Optimization Using Evolutionary Algorithms. Wiley: New York, 2001.3. Rao SS. Engineering Optimization: Theory and Practice. Wiley: New York, 1996.4. Harrington J. The desirability function. Industrial Quality Control 1965; 21(10):494–498.5. Trautmann H, Weihs C. On the distribution of the desirability index using Harrington’s desirability function.

Metrika 2006; 63(2):207–213.6. Goel T, Stander N, Lin Y-Y. Efficient resource allocation for genetic algorithm based multi-objective optimization

with 1000 simulations. Structural and Multi-Disciplinary Optimization 2009; DOI: 10.1007/s00158-009-0426-9.7. Hallquist JO. LS-DYNA� User’s Manual, Version 971, May 2007.8. Venkataraman S, Haftka RT. Structural optimization: what has Moore’s law done for US? Structural and

Multi-Disciplinary Optimization 2004; 28(6):375–387.9. Coello CAC, Pulido GT. Multiobjective optimization using a micro genetic algorithm. Proceedings of the Genetic

and Evolutionary Computation Conference (GECCO-2001), San Francisco, CA, 2001.10. Knowles J. ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multi-objective

problems. IEEE Transactions on Evolutionary Computation 2006; 10(1):50–66.



11. Jensen MT. Reducing the run-time complexity of multiobjective EAs: the NSGA-II and other algorithms. IEEETransactions on Evolutionary Computation 2003; 7(5):503–515.

12. Isibuchi M, Murata T. A multi-objective genetic local search algorithm and its application to flowshop scheduling.IEEE Transactions on Systems, Man and Cybernetics—Part C: Applications and Reviews 1998; 28(3):392–403.

13. Deb K, Goel T. A hybrid multi-objective evolutionary approach to engineering shape design. Proceedings ofthe First International Conference on Evolutionary Multi-Criterion Optimization (EMO’01), Zurich, Switzerland,March 2001; 385–399.

14. Goel T, Deb K. Hybrid methods for multi-objective evolutionary algorithms. Proceedings of the 4th Asia–PacificConference on Simulated Evolution And Learning (SEAL-01), Singapore, November 2001; 188–192.

15. El-Beltagy MA, Nair PB, Keane AJ. Metamodeling techniques for evolutionary optimization of computationallyexpensive problems: promises and limitations. Proceedings of the Genetic and Evolutionary ComputationConference (GECCO-1999), Orlando, FL, 1999.

16. Wilson B, Cappalleri D, Simpson TW, Frecker M. Efficient pareto frontier approximation using surrogateapproximations. Optimization and Engineering 2001; 2(1):31–50.

17. Giannakoglou KC. Design of optimal aerodynamic shapes using stochastic optimization methods and computationalintelligence. Progress in Aerospace Sciences 2001; 38(1):43–76.

18. Goel T, Vaidyanathan R, Haftka RT, Shyy W. Response surface approximation of pareto optimal frontin multiobjective optimization. Computer Methods in Applied Mechanics and Engineering 2007; 196(4–6):879–893. Also presented at the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference(MAO, Sep-2004).

19. Voutchkov I, Keane AJ. Multi-objective optimization using surrogates. Proceedings of the 7th InternationalConference on Adaptive Computing in Design and Manufacture, Bristol, U.K., April 2006.

20. Ray T, Smith W. A surrogate assisted parallel multi-objective evolutionary algorithm for robust engineeringdesign. Engineering Optimization 2006; 38(8):997–1011.

21. Emmerich MTM, Giannakoglou KC, Naujoks B. Single- and multi-objective evolutionary optimization assistedby gaussian random field metamodels. IEEE Transactions on Evolutionary Computation 2006; 10(4):421–439.

22. Goel T, Dorney DJ, Haftka RT, Shyy W. Improving the hydrodynamic performance of diffuser vanes via shapeoptimization. Computers and Fluids 2008; 37(6):705–723.

23. Aytug H, Koehler GJ. Stopping criteria for finite length genetic algorithms. ORSA Journal of Computing 1996;8(2):181–193.

24. Aytug H, Koehler GJ. New stopping criteria for genetic algorithms. European Journal of Operations Research1996; 126:662–674.

25. Pendharkar P, Koehler GJ. A general steady state distribution based stopping criteria for finite length geneticalgorithms. European Journal of Operations Research 2007; 176:1436–1451.

26. van Veldhuizen DA, Lamont GB. Evolutionary computation and convergence to a pareto front. Proceedings ofthe 3rd Annual Conference on Genetic Programming, San Francisco, CA, August 1998; 221–228.

27. Hanne T. On the convergence of multiobjective evolutionary algorithms. European Journal of OperationalResearch 1999; 117(3):553–564.

28. Rudolph G, Agapie A. Convergence properties of some multi-objective evolutionary algorithms. Proceedings ofthe IEEE Congress on Evolutionary Computation (CEC-2000), San Diego, CA, 2000; 1010–1016.

29. Rudolph G. Self-adaptive mutations lead to premature convergence. IEEE Transactions on EvolutionaryComputation 2001; 5(4):410–414.

30. Li G, Goel T, Stander N. Assessing the convergence properties of NSGA-II for direct crashworthiness optimization,Tenth International LS-DYNA� User’s Conference, Detroit, MI, 8–10 June 2008.

31. Rudenko O, Schoenauer M. A steady performance stopping criterion for pareto-based evolutionary algorithm.Proceedings of the 6th International Conference on Multi-Objective Programming and Goal Programming,Hammamet, Tunisia, 2004.

32. Marti L, Garcia J, Berlanga A, Molina JM. A cumulative evidential stopping criterion for multi-objectiveoptimization evolutionary algorithms. Proceedings of the 9th Genetic And Evolutionary Computation Conference(GECCO-07), London, U.K., July 2007; 2835–2842.

33. Trautmann H, Ligges U, Mehnen J, Preuss M. A convergence criterion for multi-objective evolutionary algorithmbased on systematic statistical testing. Proceedings of the 10th Parallel Problem Solving From Nature (PPSN-X),Dortmund, Germany, September 2008; 825–836.

34. Wagner T, Trautmann H, Naujoks B. OCD: online convergence detection for evolutionary multi-objectivealgorithms based on statistical testing. Proceedings of the 5th International Conference on Multi-CriterionOptimization (EMO-09), Nantes, France, April 2009; 198–215.



35. Naujoks B, Trautmann H. Online convergence detection for multiobjective aerodynamics applications. Proceedingsof the IEEE Congress on Evolutionary Computation (CEC-2009), Trondheim, Norway, May 2009.

36. Trautmann H, Wagner T, Naujoks B, Preuss M, Mehnen J. Statistical methods for convergence detection ofmultiobjective evolutionary algorithms. Evolutionary Computation Journal 2009; 17(4):493–509.

37. Martí L, García J, Berlanga A, Molina JM. An approach to stopping criteria for multi–objective optimizationevolutionary algorithms: the MGBM criterion. 2009 IEEE Conference on Evolutionary Computation (CEC 2009),Trondheim, Norway, May 2009.

38. Van Veldhuizen DA. Multiobjective evolutionary algorithms: classifications, analyses, and new innovations. Ph.D.Thesis, Department of Electrical and Computer Engineering, Air Force Institute of Technology, Wright PattersonAir Force Base, OH, 1999.

39. Zitzler E, Thiele L. Multiobjective evolutionary algorithms: a comparative case study and the strength paretoapproach. IEEE Transactions on Evolutionary Computation 1999; 3(4):257–271.

40. Deb K, Jain S. Running performance metrics for evolutionary multi-objective optimization. Proceedings of theFourth Asia-Pacific Conference on Simulated Evolution and Learning (SEAL-02), Singapore, 2002; 13–20.

41. Deb K, Pratap A, Agarwal S, Meyarivan T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEETransactions on Evolutionary Computation 2002; 6(2):182–197.

42. Tanaka M, Watanabe H, Furukawa Y, Tanino T. GA-based decision support system for multi-criteria optimization.Proceedings of the International Conference on Systems, Man, and Cybernetics, Vancouver, BC, Canada, vol. 2,1995; 1556–1561.

43. Osyczka A, Kundu S. A new method to solve generalized multicriteria optimization using the simple geneticalgorithm. Structural Optimization 1995; 10(2):94–99.

44. Akkerman A, Thyagarajan R, Stander N, Burger M, Kuhm R, Rajic H. Shape optimization for crashworthinessdesign using response surfaces. Proceedings of the International Workshop on Multidisciplinary DesignOptimization, Pretoria, South Africa, 2000; 270–279.

45. Craig KJ, Stander N, Dooge DA, Varadappa S. Automotive crashworthiness design using response surface-basedvariable screening and optimization. Engineering Computations 2005; 22:38–61.

46. Stander N, Roux WJ, Goel T, Eggleston T, Craig KJ. LS-OPT� Version 4.0 User’s Manual, Livermore SoftwareTechnology Corporation, April 2009.

47. Schott JR. Fault tolerance design using single and multi-criteria genetic algorithms. Master’s Thesis, Departmentof Aeronautics and Astronautics, MIT, Boston, MA, 1995.

48. Hansen MP, Jaskiewicz A. Evaluating the quality of approximation to the non-dominated set. Technical ReportIMM-REP-1998-7, Institute of Mathematical Modeling, Technical University of Denmark, Lygnby, 1998.


a non-dominance-based online stopping criterion for multi-objective evolutionary algorithms

Documents