Journal of Computer Science 5 (5): 392-397, 2009 ISSN 1549-3636 2009 Science Publications 392 Measuring the Software Product Quality during the Software Development Life-Cycle: An International Organization for Standardization Standards Perspective Rafa E. Al-Qutaish Department of Software Engineering, Alzaytoonah University of Jordan, Airport Street, Amman 11733, Jordan Abstract:Problemstatement: TheInternationalOrganizationforStandardization(ISO)publisheda setofinternationalstandardsrelatedtothesoftwareengineering,suchasISO12207andISO9126. However, there is a set of cross-references between these two standards. Approach: The ISO 9126 on softwareproductqualityandISO12207onsoftwarelifecycleprocesseshadbeenanalysedto investigatetherelationshipsbetweenthemandtomakeamappingfromtheISO9126quality characteristicstotheISO12207activitiesandversversa.Results:Thisstudypresentedasetof commentsandsuggestionstoimprovetheISO9126.Conclusion:Theweaknessesofthecross-referencesbetweenthetwoISOstandardshadbeenhighlighted.Inaddition,thisstudyprovideda numberofcommentsandsuggestionstobetakenintoaccountonthenextversionoftheISO9126 international standard. Key words:ISO 9126, ISO 12207, ISO software engineering standards, software life cycle processes, software product quality evaluation, software product measurement INTRODUCTION Thelastdecadeof20thcenturyhasseenarapid increase in the use of the software products in a variety ofapplicationareasandtheircorrectoperationisoften criticalforbusinesssuccessandhumansafety.This trend is expected to continue in light of the progress and utilizationofthemeasurementevaluationofsoftware productquality,particularlyfromtheinternational standardization perspective. Intheareaofsoftwareengineering,theconceptof softwaremeasurement(orwhatiscommonlycalled software metrics) is not new. Since 1972, a number of so-calledsoftwaremetrics,ormeasures,havebeen developed.In order to standardize the software product quality measurementprocess,in1991,theISOpublishedits firstinternationalconsensusontheterminologyforthe qualitycharacteristicsforsoftwareproductevaluation: Thisstandardwascalledsoftwareproductevaluation-qualitycharacteristicsandguidelinesfortheiruse (ISO9126:1991)[5].From2001-2004,theISO published an expanded version, containing both the ISO qualitymodelsandinventoriesofproposedmeasures for these models (ISO 9126 parts 1, 2, 3 and 4)[9,11,12]. Since1991,ISOproducedanumberof InternationalStandards(IS)andTechnicalReports (TR)relatedtothesoftwareengineering.However,in thisstudywehavehighlightedtheISO9126metrics andtheirrelationshipstotheISO12207softwarelife cycle processes and activities. In other words, this study explainswhichsoftwaremetricsyoucancompute during the software life cycle.In 2002, ISO produced the ISO 15939 international standard[8] which contains the definitions of the terms to be used in the software measurement process, including thetermmeasureinsteadofthetermmetrics. However, in this study we will use the term metrics in order to be alignedwiththefour parts of the ISO 9126 international standard. ThecurrentversionoftheISO9126international standardconsistsofinventoriesofproposedmetricsto measurethequalityoftheinternal,externalandin-use softwareproduct.However,foreachofthesemetrics thereisacross-referenceonwheretheycouldbe measuredduringtheISO12207SoftwareLifeCycle Processesandactivities(SLCP).Thisstudyprovidesa mappingbetweenthesetwostandardstohighlightsthe weaknessesofthesecross-referencesandproposesa way to address them and it provides an approach to use theISO9126metricstomeasurethesoftwarequality during the software life cycle. TheInternationalOrganizationforStandardization (ISO)publishedasetofinternationalstandardsrelated tothesoftwareengineering.However,someofthese internationalstandardshavemanyrelationships betweenthem.Thisstudyinvestigatestherelationships between ISO 9126 on software product quality and ISO 12207 on software life cycle processes.J. Computer Sci., 5 (5): 392-397, 2009 393 However,therelationshipsbetweenthemare many-to-manyrelationshipssinceeachofISO9126 characteristic/subcharacteristiccouldbemeasuredin differentISO12207softwarelifecycleprocessesand activitiesandineachISO12207softwarelifecycle process or activity we can measure different ISO 9126 characteristics/subcharacteristics.Furthermore,this studyhighlightstheweaknessesandthecross-referencesbetweenthetwoISOstandards.Finally,a numberofsuggestionsandcommentshavebeen addressed. Recently,theISOhasrecognizedaneedfor furtherenhancementofISO9126International Standard,primarilyasaresultofadvancesinthe fieldsofinformationtechnologiesandchangesin environment[3].Therefore,theISOisnowworkingon thenextgenerationofsoftwareproductquality standards[12],whichwillbereferredtoasSoftware ProductQualityRequirementsandEvaluation (SQuaRE-ISO25000series).Thisseriesofstandards willreplacethecurrentISO9126International Standard.However,manyresearcheshavefocusedon someweaknessesonthecurrentISO9126andeven on the draft versions of the upcomingnew ISO 25000 series of standards (SQuaRE)[1,2]. MATERIALS AND METHODS ISO9126onsoftwarequalitymeasurement:The ISO9126seriesofstandardsnowconsistsofone internationalstandard[7]andthreetechnical reports[11,12]: ISO 9126-1: Quality model[7]

ISO TR 9126-2: External metrics[9]

ISO TR 9126-3: Internal metrics[10]

ISO TR 9126-4: Quality in-use metrics[11]

ThefirstdocumentoftheISO9126series-Quality model-containstwo-partqualitymodelforsoftware product quality[7]: Internal and external quality model Quality in-use model Thefirstpartofthetwo-partqualitymodel determinessixcharacteristicsinwhichtheyare subdividedintotwenty-sevensubcharacteristicsfor internalandexternalquality,asinFig.1[7].These subcharacteristicsarearesultofinternalsoftware attributesandarenoticeableexternallywhenthe softwareisusedasapartofacomputersystem.The second part of the two-part model indicates four quality in-use characteristics. All the quality characteristics and theircorrespondingsub-characteristicsaredefinedin ISO 9126-1[7]. The2nd,3rdand4thdocumentsoftheISO9126 series provide the following information[11,12]: Setsofmetricsforeachexternalqualitysub-characteristic,internalqualitysub-characteristic and quality in-use characteristic Explanations of how to apply and use these sets of metrics Examples of how to apply these metrics during the software product lifecycle ISO 12207 on software life cycle processes: It consists ofprocesses,activitiesforeachprocessandtasksfor each activity[5,7]. Figure 2 shows the software life cycle processes,thenumberofactivitiesineachprocessand the number of tasks in each process. The full list of the process, activities and tasks can be seen in ISO 12207[6] andIEEE/EIA12207[4](theIEEE/EIA12207isthe IEEE version of the ISO 12207). Fig. 1: ISO 9126 quality model for external and internal quality (characteristics and sub-characteristics)[7] J. Computer Sci., 5 (5): 392-397, 2009 394 Fig. 2: ISO 12207 software life cycle processes, activities and tasks TheISO12207softwarelifecycleprocessesare groupedintothreebroadclasses:Primary;supporting andorganizational.Primaryprocessesaretheprime moversinthelifecycle;theyareacquisition,supply, development,operationandmaintenance.Supporting processesaredocumentation,configuration management,qualityassurance,jointreview,audit, verification,validationandproblemresolution.A supportingprocesssupportsanotherprocessin performingaspecializedfunction.Organizational processes are management, infrastructure, improvement andtraining.Anorganizationmayemployan organizational process to establish, control and improve a life cycle process. RESULTS For each metric of the internal, external and in-use metrics,theISO9126parts2-4providesthefollowing information:metricname,purposeofthemetric, methodofapplication,measurementformula, interpretationofmeasuredvalue,metricscaletype, measuretype,inputtomeasurement,ISO12207SLCP Reference and target audience[11,12]. Hereinafterwewilldiscussthedetailedmappings betweentheISO9126qualitymetricsoftheInternal, external and in-use software product and the ISO 12207 softwarelifecycleprocessesandactivitieswillbe provided.Furthermore,thismappingwillfocusonan investigationoftheISO12207SoftwareLifeCycle Processes(SLCP)ReferencesprovidedbyISO9126 for each of its metrics. Internalqualitymetrics:WithintheISO9126-3on softwareproductinternalqualitymetrics,thereare70 metrics.Thesemetricscanbemeasuredduringthe softwarelifecycle.InternalqualitydefinedinISO 9126-1 as the totality of characteristics of the software productfromaninternalview.Internalqualityis measuredandevaluatedagainsttheinternalquality requirements.Detailsofsoftwareproductqualitycan beimprovedduringcodeimplementation,reviewing and testing, but the fundamental nature of the software product quality represented by internal quality remains unchanged unless redesigned[10]. Table1showsthenumberofinternalquality metricswhich can bemeasured during each of the ISO 12207softwarelifecycleprocesses.Forexample, withintheverificationprocess(ofthesupporting processes) 59 metrics can be measured.J. Computer Sci., 5 (5): 392-397, 2009 395 Table 1:Mapping between the ISO 9126 internal quality metrics and the ISO 12207 processes ISO 12207 processesNo. of related ISO 9126 metrics 5.0 Primary life cycle processes: 5.1 Acquisition0 5.2 Supply0 5.3 Development1 5.4 Operation0 5.5 Maintenance0 6.0 Supporting life cycle processes: 6.1 Documentation0 6.2 Configuration management0 6.3 Quality assurance2 6.4 Verification:59 6.5 Validation13 6.6 Joint review59 6.7 Audit0 6.8 Problem resolution4 Fig. 3:ExampleontheISO9126-3internalquality metrics and where they could be measured during theISO12207softwarelife-cycleprocessesand activities As an example, Fig. 3 shows a detailed structure of thesoftwareproductinternalqualitymetricsnames and where they can be measured during the verification process(whichisapartoftheISO12207supporting process)alongwiththecorrespondingcharacteristic andsubcharacteristicreferencenumberforeachof thosemetrics(forexample1.2meanstheaccuracy subcharacteristicofthefunctionalitycharacteristic).In Fig.3onlythesoftwarelifeprocesses/activitieswhich have internal quality metrics are mentioned. However, from Table 1 we can note that there is no anymetricwhichcouldbemeasuredduring4outof5 primary life cycle processes. This means that there is no anymetricfromISO9126externalqualitymetrics couldbeusefulduringtheacquisition,supply, operation and maintenance primary life cycle processes. Table 2:Mapping between the ISO 9126 external quality metrics and the ISO 12207 processes ISO 12207 ProcessesNo. of related ISO 9126 Metrics 5.0 Primary life cycle processes: 5.1 Acquisition0 5.2 Supply0 5.3 Development14 5.4 Operation93 5.5 Maintenance84 6.0 Supporting life cycle processes: 6.1 Documentation0 6.2 Configuration management0 6.3 Quality assurance14 6.4 Verification0 6.5 Validation47 6.6 Joint review0 6.7 Audit0 6.8 Problem Resolution1 Moreover,thereisnoanymetricwhichcouldbe measuredduring3outof8 ofthesupportinglifecycle processes;thatis,documentation,configuration management and audit processes. Externalqualitymetrics:WithintheISO9126-2on software product external quality metrics, there are 110 metrics.Thesemetricscanbemeasuredduringthe softwarelifecycle.ExternalqualitydefinedinISO 9126-1asthetotalityofcharacteristicsofthesoftware productfromanexternalview.Itisthequalitywhen thesoftwareisexecuted,whichistypicallymeasured and evaluatedwhile testing in a simulated environment withsimulateddatausingexternalmetrics.During testing,mostfaultsshouldbediscoveredand eliminated. However, some faults may still remain after testing.Asitisdifficulttocorrectthesoftware architectureorotherfundamentaldesignaspectsofthe software,thefundamentaldesignusuallyremains unchanged throughout testing[9]. Table2showsthenumberofexternalquality metricswhich can bemeasured during each of the ISO 12207softwarelifecycleprocesses.Forexample, withintheoperationprocessoftheprimary processes, 93 metrics can be measured. Qualityin-usemetrics:WithintheISO9126-2on softwareproductqualityin-usemetrics,thereare15 metrics.Thesethe15metricscanbemeasuredduring thesoftwarelifecycle.Qualityin-usedefinedinISO 9126-1 as the users view of the quality of the software product when it is used in a specific environment and a specific context ofuse. Itmeasures the extent towhich userscanachievetheirgoalsinaparticular environment,ratherthanmeasuringthepropertiesof the software itself. The term user refers to any type of intendedusers,includingbothoperatorsand maintainers and their requirements can be different[11]. J. Computer Sci., 5 (5): 392-397, 2009 396 Table 3:MappingbetweentheISO9126qualityin-usemetricsand the ISO 12207 processes ISO 12207 ProcessesNo. of related ISO 9126 metrics 5.0 Primary life cycle processes: 5.1 Acquisition0 5.2 Supply0 5.3 Development12 5.4 Operation15 5.5 Maintenance0 6.0 Supporting life cycle processes: 6.1 Documentation0 6.2 Configuration management0 6.3 Quality assurance0 6.4 Verification0 6.5 Validation11 6.6 Joint review0 6.7 Audit0 6.8 Problem resolution0 Table 3 shows the number of quality in-use metrics whichcanbemeasuredduringeachoftheISO12207 softwarelifecycleprocesses.Forexample,duringthe softwarequalificationtestingactivityofthe developmentprocessoftheprimaryprocesses,12 metrics can be measured. DISCUSSION InISO9126-3,therearesomeexternalquality metrics-asinTable4whichhavebeenreferredtobe measuredduringtheintegrationactivityofthe developmentprocessoftheprimaryprocesses.But, withinthedevelopmentprocess,therearetwo activitiesrelatedtotheintegration,thatis,system integrationandsoftwareintegration.However,this document(ISO9126-3)didnotspecifyduringwhich integration activity those metrics can be measured. ItisclearlynotedthatthroughtheISO12207 organizationalprocessesnoneofthe195quality metrics-fromtheISO9126seriesofinternational standards-canbemeasured.Therefore,theISO9126is not usable for the ISO 12207 organizational processes. AsmentionedinISO9126-1,thequalityin-use metrics should be measured during the execution of the softwareproductinanactualworkingenvironment. However,fromTable4wecanseethatthereare12 metrics which could be measured through the software qualificationtestingactivity.ButsinceISO12207 mentionedthatthesoftwarequalificationtesting activity is a part of the development process Thus, its strange and make no sense to measure that 12 metrics. Thejointreviewprocessofthesupporting processesconsistsofthreeactivities:Oneofthese activitiesisthetechnicalreviewsactivity.This activitycontainsonetask,thatis,Technicalreviews shallbeheldtoevaluatethesoftwareproductsor servicesunderconsiderationandprovideevidencethat theyarecomplete[6].Now,ifwego back to Table 1, Table 4:Externalonqualitymetricswhichisnotclearwheretobe measured 1Estimated latent 9Fault removal fault density10Restartability 2Incorrect operation avoidance11Mean time between 3 Failure density 12User support against test casesfailures (MTBF) 4Availabilityfunctional consistency 5Failure resolution13Breakdown 6Mean down time14Restore effectiveness 7 Fault density15Failure avoidance 8Mean recovery time16Restorability wewillfindthatthereare59internalqualitymetrics thatcouldbemeasuredduringthejointreview process.Whereas,fromTable2and3,itisseenthat there is no any external quality or quality in-use metrics that can be measured during joint review process. CONCLUSION ThecurrenteditionoftheISO9126consistsof inventoriesofproposedmetricstomeasurethequality oftheinternal,externalandin-usesoftwareproduct. However,foreachofthesemetricsthereisacross-reference on where it could be measured during the ISO 12207 software life cycle processes and activities. This study provided a mapping between those two standards to investigate the cross-references between them. Based onthismapping,thefollowingcommentsand suggestionsfortheupcomingnewISO25000seriesof standards (SQuaRE) can be concluded: There is no any metric can be measured during the organizational processes Anumberofexternalqualitymetricswhere mentionedinISO9126-2tobemeasuredduring theintegrationactivity.However,withintheISO 12207therearetwoactivitieslabeledsystem integrationandsoftwareintegration,Table4for thenamesofthesemetrics.However,thiswill make the user of the ISO 9126 confused ManyoftheISO9126qualitymetricsreferredto processes.However,asknown,eachprocessin ISO 12207 contains a number of different activities. Thus,itismoreusablefortheISO9126usersto refer to the activities of the ISO 12207. This can be doneusingcross-referencenumbersfromISO 12207.Forexample,thecross-referencenumber 5.3.9isreferringtoprimaryprocesses, developmentprocessandsoftwarequalification testing activity, respectively, Fig. 3 for an example onwheretheISO9126internalqualitymetrics couldbemeasuredwithintheISO12207 verification activities J. Computer Sci., 5 (5): 392-397, 2009 397 In addition to the mapping in this study, as a future work,itisagoodideatoinvestigatewheretocollect the data for each of the ISO 9126 quality metrics in the ISO12207softwarelifecycleprocessesandactivities. Thiswillsavetimeandassurethatthedatahavebeen completelycollectedbeforethemeasurementofthe metrics is performed. REFERENCES 1.Abran,A.,R.E.Al-QutaishandJ.M.Desharnais, 2005.Harmonizationissuesintheupdatingofthe ISOstandardsonsoftwareproductquality.Metr. News J., 10: 35-44. http://www.gelog.etsmtl.ca/publications/pdf/975.pdf2.Abran,A.,R.E.Al-Qutaish,J.M.Desharnaisand N.Habra,2005.Aninformationmodelfor softwarequalitymeasurementwithISOstandards. ProceedingsoftheInternationalConferenceon SoftwareDevelopment,(SWDC-REK'05), Reykjavik,Iceland,pp:104-116. http://www.lrgl.uqam.ca /publications/pdf/904.pdf 3.Azuma, M., 2001. SQuaRE: The next generation of ISO/IEC9126and14598internationalstandards seriesonsoftwareproductquality.Proceedingsof theEuropeanSoftwareControlandMetrics Conference,(ESCOM01),London,UK.,pp:337-346. http://miro.sztaki.hu/husei/events/SQUAD/reload /SQUAD-SQuaRE-Budapest-vers1.ppt4.IEEE,1996.IEEE/EIA-12207:Information technology-softwarelifecycleprocesses.The InstituteofElectricalandElectronicsEngineers, NewYork,USA. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=707581&isnumber=15306 5.ISO.,1991.ISO/IECIS9126-softwareproduct evaluation-qualitycharacteristicsandguidelines fortheiruse.InternationalOrganizationfor Standardization,Geneva,Switzerland.http://www.tbs-sct.gc.ca/its-nit/standards/tbits26/crit26-eng.asp 6.ISO.,1995.ISO/IEC12207-information technology-softwarelifecycleprocesses. internationalorganizationforstandardization. Geneva, Switzerland. http://www.iso.org/iso/isocatalogue/cataloguetc/cataloguedetail.htm?csnumber=21208 7.ISO.,2001.ISO/IEC9126-1:Software engineering-productquality-part1:Qualitymodel. InternationalOrganizationforStandardization, Geneva, Switzerland. http://www.iso.org/iso/isocatalogue/cataloguetc/cataloguedetail.htm?csnumber=22749 8.ISO.,2007.ISO/IECIS15939:Software engineering-softwaremeasurementprocess. InternationalOrganizationforStandardization, Geneva, Switzerland. http://www.iso.org/iso/isocatalogue/cataloguetc/cataloguedetail.htm?csnumber=44344 9.ISO.,2003.ISO/IECTR9126-2:Software engineering-productquality-part2:External metrics.InternationalOrganizationfor Standardization,Geneva,Switzerland. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=22750 10.ISO.,2003.ISO/IECTR9126-3:Software engineering-productquality-part3:Internal metrics.InternationalOrganizationfor Standardization,Geneva,Switzerland. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=22891 11.ISO.,2004.ISO/IECTR9126-4:Software engineering-productquality-part4:Qualityinuse metrics.InternationalOrganizationfor Standardization,Geneva,Switzerland. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=39752 12.Suryn,W.,A.AbranandA.April,2003.ISO/IEC SQuaRE:Thesecondgenerationofstandardsfor softwareproductquality.Proceedingsofthe7th IASTEDInternationalConferenceonSoftware EngineeringandApplications,(ICSEA03), California, USA., pp: 1-9. http://paginaspersonales. deusto.es/cortazar/doctorado/articulos/2003/square799.pdf