reliability growth—a new graphical model

QUALITY AND RELIABILITY ENGINEERING INTERNATIONAL

Qual. Reliab. Engng. Int.15: 167–174 (1999)

RELIABILITY GROWTH—A NEW GRAPHICAL MODEL

JOHN DONOVAN1∗AND EAMONN MURPHY2

1Institute of Technology, Sligo, Ireland2University of Limerick, Limerick, Ireland

SUMMARYA new reliability growth model is presented which is simpler to plot and fits the data more closely than the Duanemodel over the range of Duane slopes normally observed during a reliability growth programme. The model isderived from variance stabilization transformation theory. The problem, inherent in Duane’s model, of providingtoo much influence to earlier failures is overcome. A method is also presented of comparing different modelswhich have differenty-axes. Extensive simulations were conducted and the results indicate that for Duane slopesless than 0.5 the new model is more effective. The leverage and influence aspects of both models are evaluatedby means of Cook’s distance. Finally, two published datasets are analysed and the findings confirm the simulationresults. Copyright 1999 John Wiley & Sons, Ltd.

KEY WORDS: reliability growth models; Duane; leverage; Cook’s distance

1. INTRODUCTION

A reliability growth model is an important toolfor monitoring and tracking reliability improvementduring product development. Many models have beenproposed and are in use, although Duane’s model [1]has remained one of the primary graphical models.Crow [2] observed that this model is essentially anNHPP model with a Weibull intensity function. Itis however somewhat surprising that other graphicalmodels have not been developed with the expresspurpose of displaying growth to the developmentteams. Apart from the works of Xie and Zhao [3,4],few have looked at other graphical alternatives toDuane’s model.

The Duane model represents a relationship betweenthe cumulative MTBF (θ ) and the cumulative testhours (T ) such thatθ = α1T β1. Data following thisrelationship, when plotted on log–log paper, fall on astraight line

ln(θ) = ln(α1) + β1 ln(T )

This model has a number of inherent difficulties,one being that early failures have a high influence onboth the slope and the graphical display. Therefore,should one attempt to utilize it as a means of observinggrowth during testing, the resulting graph is overlyaffected by those failures occurring early in time.

∗Correspondence to: J. Donovan, Institute of Technology,Ballinode, Sligo, Ireland

Secondly, if a number of failures occur towards thelatter part of the test, these points tend to be clusteredtogether owing to the nature of ln(cum. time).

To overcome the difficulties cited above, the authorshave developed a new graphical model which solvesthe problems associated with the Duane model whileat the same time making the resulting graph easier toplot, interpret and visualize. This new model is foundto be better than Duane’s model over the range ofslopes normally observed during reliability growth.

2. DATA TRANSFORMATIONS

While reliability growth is not specifically a regressionanalysis situation, regression analysis methods areused to develop the slope and intercepts for the Duanemodel. Data transformations are a common techniqueused in regression analysis to plot what are in effectnon-linear data and transform them so that they canbe plotted on a linear scale. One of the common datatransformation techniques is variance stabilization,discussed by Montgomery and Peck [5]. While thistechnique is ideally suited for the situation wherethe linear regression assumption of constant varianceis violated, it also has application to the reliabilitygrowth situation.

The authors have continued the precedent ofplotting cumulative MTBF on the y-axis andcumulative time on thex-axis. As each failure occurs

CCC 0748–8017/99/030167–08$17.50 Received 10 January 1998Copyright 1999 John Wiley & Sons, Ltd. Revised 24 August 1998

168 J. DONOVAN AND E. MURPHY

in accordance with a Poisson process, the cumulativetime can be viewed as a ‘count’ of the numberof hours during which a number of failures haveoccurred, or in other words, a count of the time inwhich a specific cumulative MTBF has been reached.Variance stabilization transformation theory suggeststhat for a Poisson process the transformation equalsthe square root of the count. Therefore in the caseof cumulative time the transformation of thex-axisbecomes

√cum. time.

An advantage of this model is that there is no needto transform they-axis, so that cumulative MTBFis plotted directly without transformation. The modeltherefore becomes

θ = α2 + β2√

T

where θ represents the cumulative MTBF,Trepresents the cumulative time,α2 is the intercept andβ2 is the slope.

One further advantage of the variance stabilizationtransformation technique is that it avoids the clusteringeffect associated with failures occurring late in time.

3. LEVERAGE AND INFLUENCE

One of the difficulties of the Duane model is the highleverage associated with failures occurring early intime. Leverage points are points which have relativelyhigh or very low values on thex-axis. These pointscan dramatically affect the slope of the model. A high-leverage point is considered a high-influence point if italso has a relatively high or low value on they-axis.

Within regression analysis the slope of any model isrelated as

slope= Sxy/Sx x

As Sxy = ∑(xi−x)yi , one can observe that the further

away xi is from x , the greater the affect it has onSxy and thereby on the slope. Now, as Duane’s modelemploys the natural log transformation of both thex-and they-axis, the effect is that the values close tothe origin having low natural log values have undueinfluence on the slope. Such points are known as high-influence points and distort the model. As a means ofovercoming this problem, some have suggested thatthe failures within an initial test period be unplottedbut accounted for within the cumulative MTBF figure.An initial test period of 100 h has been suggested [6],varying up to half of the predicted MTBF [7].

There is a statistic known as Cook’s distance(Di ) which provides a measure of the leverageand influence of each point or failurei within the

regression model. This can be calculated as

Di = r2i

p

(hi

1 − hi

)

where p is the number of variables in the reliabilitygrowth model and is always two.hi represents theleverage of each failurei and can be calculated from

hi = 1

n+ (xi − x)2∑n

i=1(xi − x)2

wheren is the number of failures. The minimum valueof leverage (hi ) is 1/n, while as the distancexi fromx increases, so does the leverage of that point.ri is thestandardized residual and can be calculated from

ri = ei√MSE(1 − hi )

ei is the residual of each failure and represents theobserved y− predicted y(i.e. the difference betweenthe plottedy value and the value ofy predicted by themodel). In the case of the Duane model these valuesof y and y represent the transformed values.

MSE(mean square error) =∑n

i (y − y)2

n − 2

The value of Cook’s distanceDi is comprised oftwo components, one which measures how well themodel fits the observed valueyi and another whichreflects the remoteness ofxi from the remainingn − 1failures. In this way, Cook’s distance measures boththe influence and leverage of each point or failure. Alarge Cook’s distance may arise from either (or both)of these components. Typically, a value ofDi greaterthan one is considered to be influential.

It is worth noting that it may not be inferred thatthere is something wrong with a data point that has ahigh influence or leverage associated with it.

4. TYPICAL DUANE SLOPES OBSERVED INRELIABILITY GROWTH PROGRAMMES

O’Connor [8] identifies typical Duane slopes associ-ated with a reliability growth or a test, analyse andfix (TAAF) programme. In this, he has also providedsome guidelines as to the product/process reliabilityprogramme effectiveness based on this Duane slope.Values of slope ranging between 0.4 and 0.6 areconsidered very good growth rates. Between 0.3 and0.4 the growth is quite good, while between 0.2 and 0.3it is considered reasonable. Duane originally observedtypical slopes ranging between 0.2 and 0.4. It is overthese total ranges of values that the two models will becompared and contrasted.

Copyright 1999 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int.15: 167–174 (1999)

RELIABILITY GROWTH—A NEW GRAPHICAL MODEL 169

Table 1. Overview of simulation results at varying values of Weibull slope

BetterR∗2

Weibull Slope Slope Mean Std dev. Median (% ofshape Model mean Std dev. R∗2(%) R∗2(%) R∗2(%) datasets)

0.5 Duane 0.672 0.071 98 1.9 99 97New 94 2.6 94 3

0.7 Duane 0.542 0.093 94 6.0 96 68New 93 4.4 94 32

0.75 Duane 0.518 0.099 93 6.7 95 58New 93 4.6 94 42

0.8 Duane 0.493 0.098 92 7.4 94 47New 93 5.1 94 53

0.98 Duane 0.437 0.098 89 9.1 91 28New 91 6.5 92 72

5. MODEL COMPARISONS ON DIFFERENTSCALES

In determining how effective the new model is, it isnecessary to see how it compares with the Duanemodel in terms of modelling the reliability growth.

In terms of ease of plotting, the new model is clearlymore effective, as fewer transformations are requiredand predicted cumulative MTBF figures are obtaineddirectly from the plot.

The simplest measure of model adequacy is thecoefficient of determination (R2), which is a measureof how well the model fits the data. It representsthe percentage variability iny explained by x .It is important to remember, however, that theR2 of respective models provides an inappropriatecomparison if the modely-axes differ.

As the new model has completely different axesfrom the Duane model, the method chosen to comparethe two models is from Reference [9]. This requiresfinding the coefficient of determination betweeny and y. In the case of a transformedy thisimplies first applying the inverse transformation toobtain y values. In the Duane model this meansgetting exp[ln(cum. M ˆTBF)] for the various failures.One then evaluates the coefficient of determinationbetween these predicted values and they values (i.e.the observed cumulative MTBF). It has been decidedto call thisR∗2 to distinguish it from theR2 describedearlier. In this respect,R∗2 is a measure of thecloseness of the predicted to the actual. If they-axisis not transformed, the coefficient of determinationbetweeny and y (R∗2) is identical toR2. The bettermodel is considered to be the model with the higherR∗2.

6. SIMULATION RESULTS

Simulated data were used to compare the performanceof the Duane and the new model. The Weibulldistribution was used to generate random time-to-failure data. Various simulations were conducted withthe shape of the Weibull distribution changing and thescale remaining at 20,000 h. The number of failuresper dataset was chosen as 20. The objective was tocreate a number of datasets that, when plotted usingDuane’s model, would create datasets with varyingslopes and coefficients of determination. These samedatasets would then be used with the new model andthe results of the Duane and the new model compared.

In total, over 6400 datasets were simulated, butof these, less than 3% were discarded, because aminimum cut-off value of maximumR∗2 equal to 70%was chosen as acceptable. This still left over 6200dataset for analysis. The results of this analysis arepresented in tabular form in Table1 and graphically inthe following figures. The percentage of the simulationdatasets for which the various models had the betterR∗2 is shown in the last column of Table1.

While Table1 provides an overview of the datasetsgenerated at varying values of Weibull slope and thebehaviour of the two models, subsequent figures anddiagrams relate to the 6200 simulated datasets groupedtogether.

Figure1 presents, in histogram format, the Duaneslopes of the entire 6200 datasets. From Figures2and 3 it can be seen that the Duane model is betterwhen this Duane slope is>0.5, while the new modelis better when the Duane slope is<0.5. At a Duaneslope of 0.5 the two models appear to be equivalent.From the point of view of reliability growth, if we



Figure 1. Histogram of Duane slopes for simulated datasets

Figure 2. Duane slopes for which Duane model had largerR∗2

are interested in only one model, then the new modelis better as it provides the best fit to the data overthe range most frequently observed during a TAAFprogramme.

Figure 4 represents a scatter diagram containingeach of the 6200 simulated datasets. The largerR∗2

of each dataset is presented versus its Duane slope.One can observe that the value ofR∗2 is taperedtowards the maximum possible value of 100% as theDuane slope increases. This can be explained whenone understands that a large Duane slope is actuallya measure of the slope of the ‘best fit line’ through thefailure data.

If the number of failures is reasonably large, then ahigh slope (>0.8) is only arising because all failureslie on a straight line, thereby providing a highR∗2.

Therefore, when a high Duane slope (>0.8) arises,it is reasonable to expect that theR∗2 should also behigh. The reverse is not true of course: a highR∗2 doesnot mean a high slope, nor does a low slope mean a lowR∗2.

When one takes into account that the Duane modelis better when the Duane slope is>0.5, Figure 4

Figure 3. Duane slopes for which new model had largerR∗2

Figure 4. LargerR∗2 of each dataset versus its Duane slope

explains the situation presented in Figures5 and 6,whereby datasets for which the Duane model wasbetter tend to have higher values ofR∗2. Clearly theselie within the range of slopes for which the Duanemodel is better but which are unlikely to be observedduring a reliability growth programme.

Figure 7 is particularly interesting as it presents,as a scatter diagram, the difference inR∗2 of thetwo models for each of the simulated datasets. Apositive value indicates that the Duane model is better,while a negative value indicates that the new model isbetter. At a Duane slope of 0.5, the two models areidentical. This equality is proven mathematically inReference [10]. The further the Duane slope deviatesfrom 0.5, then the more likely will be the differencein the respectiveR∗2 of the two models. The Duanemodel is better at higher Duane slopes, while the newmodel is better for the vast range of slopes normallyobserved during a reliability growth programme.

Reference lines are included in Figure7 at values ofDuane slope equal to 0.5 for the aid of the reader.

At this stage it is worthwhile to take a look at the



Figure 5.R∗2 of datasets for which Duane model was better

Figure 6.R∗2 of datasets for which new model was better

Figure 7. Difference inR∗2 of models related to Duane slope

Figure 8. Histogram of dataset failure numbers with Cook’s distance>1 when using Duane model

Figure 9. Histogram of dataset failure numbers with Cook’s distance>1 when using new model

leverage and influence aspect associated with each ofthe models.

As explained earlier, this can be achieved byevaluating Cook’s distance. A model containing afailure with a Cook’s distance>1 was deemed tobe an influential failure. Both models were generallyequally affected by influential failures that exertedinfluence on their respective model slopes. Of themodels containing the highestR∗2, approximately63% had a Cook’s distance>1. This was tied quiteclosely to the relative performance of each model atvarious values of Weibull slope, implying that bothmodels were influenced equally.

What is interesting, however, is not the percentageof datasets containing influential points, but ratherthe specific failures within the datasets whichwere influential. Histograms of those failures whichthe models considered influential are presented inFigures8 and9. In summary, what this implies is thatwhile the Duane model is influenced by early failures,the new model is influenced by late failures.

From a predictive point of view it could be



Table 2. Data from Reference [11]

Failure Cumulativenumber time (h)

1 1032 3153 8014 11835 13456 29577 39098 57029 7261

10 8245

Figure 10. Duane model plot of data from Table2

considered more appropriate that the more recentfailures should have greater influence than the earliestfailure.

While simulation has shown that the new model isbetter at values of Duane slope< 0.5, the next step isto check whether this holds in real life.

7. VALIDATION OF MODELS

Two examples are provided on the assessment of themodels using published data. The first example relatesto data from a reliability growth programme and istaken from a text by Leitch [11].

7.1. Example 1. Data taken from Reference [11]

The growth plots of the data from Table2 arepresented in Figures10 and11 for the Duane and thenew model respectively.

Analysis of the models revealed that the Duaneslope was 0.492. TheR∗2 of the Duane model was99.07%, while theR∗2 for the new model was 99.11%.In terms of fitting the data, the new model is slightlybetter.

Figure 11. New model plot of data from Table2

Table 3. Data from Reference [12]

Failure Cumulativenumber time (h)

1 852 1703 6494 16145 20836 24687 32648 35419 4468

10 480811 521312 536313 564014 614315 663916 725917 1130918 1437319 1489520 1669221 1902122 2281923 2559424 2798725 2889626 2989027 5810228 7305829 7502930 80337



Figure 12. Duane model plot of data from Table3

This result concurs with the simulation analysis,whereby the new model is better when the Duane slopeis less than 0.5. However, the closer the slope is to 0.5,then the less is the difference between theR∗2 of thetwo models.

In relation to influential points of the models, failurenumber 1 in the Duane model has a Cook’s distance of1.3, making it an influential point in determining theslope.

There were no failures with a Cook’s distance>1 inthe new model. As described earlier, this is consistentwith the Duane model where in general early failureshave significant influence on the slope.

7.2. Example 2. Data taken from Reference [12]

The second example is taken from a paper byCurrit et al. [12] where the data are from a softwaredevelopment project.

The data from Table3 have a Duane slope of 0.496.TheR∗2 of the Duane model was 94.5%, while theR∗2

for the new model was 94.6%. Once again the resultsare consistent with simulation expectations. Therespective growth plots are presented in Figures12and13.

No failure on either model had a Cook’s distance>1.

8. CONCLUSIONS

Reliability growth is normally observed over the rangeof Duane slopes from 0.3 to 0.6. The new graphicalreliability growth model performs quite well over thisrange, outperforming the Duane model up to a slope of0.5. At a Duane slope of 0.5 the two models becomeequal, while above 0.5 the Duane model is moreeffective. Evaluation of the new model has also shownthat, unlike with the Duane model, early failures have

Figure 13. New model plot of data from Table3

little or no influence on the final slope. The new model,on the other hand, is influenced by the latest failures,which should realistically be more important than thevery earliest failures. The question one could ask iswhy, after accumulating over 100,000 h of testing,should the very first failure be the most influential?Surely from an estimation and extrapolation point ofview it does not seem realistic. To overcome thisdeficiency, the very earliest failures are often omittedfrom the Duane plot. This becomes unnecessary withthe new model, which has the added advantage ofplotting all the failure data. Finally, the new model issimpler, avoiding the need to transform they-axis, sothat the cumulative MTBF can be read directly fromthe plot.

REFERENCES

1. J. T. Duane, ‘Learning curve approach to reliabilitymonitoring’, IEEE Trans. Aerospace, AS-2, 553–566 (1964).

2. L. H. Crow, ‘Reliability analysis for complex, repairablesystems’, in F. Proschan and R. J. Serfling (eds),Reliabilityand Biometry: Statistical Analysis of Lifetimes, SIAM,Philadelphia, PA, 1974, pp. 379–410.

3. M. Xie and M. Zhao, ‘On some reliability growth models withsimple graphical interpretations’,Microelectron. Reliab., 33,149–167 (1993).

4. M. Xie and M. Zhao, ‘Reliability growth plot—an underuti-lized tool in reliability’, Microelectron. Reliab., 36, 797–805(1996).

5. D. C. Montgomery and E. A. Peck,Introduction to LinearRegression Analysis, 2nd edn, Wiley, New York, 1992, p. 98.

6. P. H. Mead, ‘Reliability growth of electronic equipment’,Microelectron. Reliab., 14, 439–443 (1975).

7. M. J. Mondro, ‘Practical guidelines for conducting a reliabilitygrowth program’,Proc. Institute of Environmental Sciences,1993, pp. 110–116.

8. P. D. T. O’Connor,Practical Reliability Engineering, 3rd ednrevised, Wiley, Chichester, 1995, pp. 308–310.

9. L. C. Hamilton,Regression with Graphics: A Second Coursein Applied Statistics, Brooks/Cole, Pacific Grove, CA, 1992,p. 181.



10. J. J. Donovan, ‘Reliability of Electronic Systems: NewModels for Growth and Test Termination’,PhD Dissertation,University of Limerick, 1998.

11. R. D. Leitch, Reliability Analysis for Engineers, OxfordUniversity Press, New York, 1995, p. 98.

12. P. A. Currit, M. Dyer and H. D. Mills, ‘Certifying thereliability of software’,IEEE Trans. Softw. Engng., SE-12, 3–11 (1986).

Authors’ biographies:

John Donovangraduated with a B.Sc. (Electronic Systems)in 1982 from the University of Limerick. He has alsostudied at University College Galway and received a PhD inReliability Engineering from the University of Limerick in

1998. From 1983 to 1993 he worked with Digital EquipmentCorporation where he managed a Design Assuranceorganisation. He has lectured in Quality Engineering at theInstitute of Technology, Sligo, since 1993. His researchinterests include Reliability Engineering, Statistical ProcessControl and Experimental Design.

Eamonn Murphy lectures in statistics at the Universityof Limerick in Ireland. He holds a primary and Master’sdegrees in Science and a Doctorate which he completedin statistics. He has consulted extensively in qualitymanagement and quality systems throughout Europe. Heis currently leading a programme in ‘Managing Change’within industry in Ireland and is the Research Director ofthe National Centre for Quality Management (NCQM) inIreland.


reliability growth—a new graphical model

Documents