Statistical Software Quality Assurance

• Implies– Information about defects is collected and

categorized– An attempt is made to trace each defect to

underlying cause– Use of Pareto Principle to identify vital

causes(80% of defects can be traced to 20% of

causes/mistakes)– Move to correct the problems that have

caused the defects

Software Engineering II

Lecture 30

Fakhar Lodhi

Recap

Example• Information about defects is collected for one year and

categorized as follows:

1. Incomplete or erroneous specifications (IES)2. Misinterpretation of customer communication (MCC)3. Intentional deviation from specifications (IDS)4. Violation of programming standards (VPS)5. Error in data representation (EDR)6. Inconsistent component interface (ICI)7. Error in digital logic (EDL)8. Incomplete or erroneous testing (IET)9. Inaccurate or incomplete documentation (IID)10. Error in programming language translation or design (PLT)11. Ambiguous or inconsistent human computer interface

(HCI)12. Miscellaneous (MIS)

Error Category Serious Moderate Minor Sub Total

IES 34 68 103 205

MCC 12 68 76 156

IDS 1 24 23 48

VPS 0 15 10 25

EDR 26 68 36 130

ICI 9 18 31 58

EDL 14 12 19 45

IET 12 35 48 95

IID 2 20 14 36

PLT 15 19 26 60

HCI 3 17 8 28

MIS 0 15 41 56

   

Total 128 379 435 942

Error Category Sub Total% of Total

errors

IES 205 22

MCC 156 17

IDS 48 5

VPS 25 3

EDR 130 14

ICI 58 6

EDL 45 5

IET 95 10

IID 36 4

PLT 60 6

HCI 28 3

MIS 56 6

   

Total 942  

IES, MCC and EDR are vital errors - cause

53% of all errors

Error Category Serious

% of Serious errors

IES 34 27

MCC 12 9

IDS 1 1

VPS 0 0

EDR 26 20

ICI 9 7

EDL 14 11

IET 12 9

IID 2 2

PLT 15 12

HCI 3 2

MIS 0 0

 

Total 128  

IES, EDR, PLT and EDL constitute about 80% of serious errors

Excel example

• Now start corrective action focused on vital few

• For example for EDR – Review the data representation techniques

to identify the possible improvement areas– Adopt a use case tool for data modeling and

perform stringent data design reviews

Error Index (EI) Used to develop an overall indication of

improvement in software quality

• Ei – the total number of errors uncovered during the ith step in the SE process

• Si – number of serious errors• Mi – number of moderate errors• Ti – number of minor errors• PSi – product size at the ith step• ws, wm, wt – weighting factors for serious,

moderate, and minor errors• Recommended values – 10, 3, 1 respectively

Error Index (EI)• At each step of the software process a Phase

Index is computed PIi = ws(Si/Ei) + wm(Mi/Ei) + wt(Ti/Ei)

• EI – cumulative effect on each PIi = ∑(i x PIi)/PSi

– Weighting errors encountered in the SE processes more heavily than those encountered earlier

– Used to develop an overall indication of improvement in software quality

Software Reliability

• Defined as:– Probability of failure free operation of a

computer program in a specified environment for a specified time

– E.g. program X is estimated to have a reliability of 0.96 over 8 elapsed hours.

• What is meant by the term failure?– Failure is non-conformance to software

requirements– Grading

• From annoying to catastrophic• Time to fix from minutes to months• Ripples from fixing

Software reliability

• Hardware versus software reliability– Hardware reliability is predicted on failure

due to wear, rather than failure due to design

– Software – no wear and tear• Mean time between failure – MTBF

– MTBF = MTTF + MTTRwhere MTTF is mean time to failure andMTTR is mean time to repair

Software reliability

• Arguably MTBF is far better than defects/kloc– Each error does not have the same failure

rate– User is concerned with failure and not with

total error count