seng521 (fall 2002)far@enel.ucalgary.ca1 seng 521 software reliability & testing defining...

SENG521 (Fall 2002) far@enel.ucalgary.ca 1

SENG 521SENG 521Software Reliability & Software Reliability & TestingTesting

Defining Necessary Reliability

(Part 3b)Department of Electrical & Computer Engineering, University of Calgary

B.H. Far （ far@enel.ucalgary.ca ）http://www.enel.ucalgary.ca/~far/Lectures/SENG521/03b/

SENG521 (Fall 2002) far@enel.ucalgary.ca 2

Necessary Reliability: How toNecessary Reliability: How to1) Define failure with “failure severity classes (FSC)”

for the product.2) Choose a common measure for all associated

systems (natural or time unit).3) Set a “failure intensity objective (FIO)” for each

system to be tested.4) Find the developed software failure intensity

objective.5) Engineer strategies to meet the software failure

intensity objective.

SENG521 (Fall 2002) far@enel.ucalgary.ca 3

How to Define FSCHow to Define FSC Mainly experience based. List all factors that may be considered as

failure severity for the project Narrow the list down to the most critical

and/or measurable ones Some factors may be hard to measure, such

as impact on company reputation, etc.

SENG521 (Fall 2002) far@enel.ucalgary.ca 4

How to Set FIO /1How to Set FIO /1 Setting FIO in terms of system reliability (R):

λ is failure intensity

R is reliability

t is natural unit (time, etc.) If reliability (R) is around 0.992 for 8 hours,

λ=0.001 or 1 failure for 1000 hours

95.0

1ln

Rfor

t

Ror

t

R

SENG521 (Fall 2002) far@enel.ucalgary.ca 5

How to Set FIO /2How to Set FIO /2 Setting FIO in terms of system availability

(A):

λ is failure intensity is downtime per failure

If a product must be available 99% of time and downtime is 6 min, then FIO is about 0.1 per hour.

mm tA

Aor

tA

1

1

1

mt

SENG521 (Fall 2002) far@enel.ucalgary.ca 6

FIO for Developed ProductFIO for Developed Product Find the developed software failure intensity

objective: Find expected failure intensity for acquired

components. Compute software failure intensity for developed

components.

SENG521 (Fall 2002) far@enel.ucalgary.ca 7

Computing Developed FIOComputing Developed FIO Example:Example: System failure intensity objective

= 100 failure/1,000,000 transactions Failure intensity for hardware

= 0.1 failure/hour OS failure for a load of 100,000 transactions

= 0.4 failure/hour Therefore, developed software FIO

= 95 failure/1,000,000 transactions

SENG521 (Fall 2002) far@enel.ucalgary.ca 8

Strategies to Meet FIOStrategies to Meet FIO Engineer strategies to meet the software

failure intensity objective for the developed software.

4 main strategies: Fault prevention Fault removal Fault tolerance Fault/failure forecasting

SENG521 (Fall 2002) far@enel.ucalgary.ca 9

Fault PreventionFault Prevention To avoid fault occurrences by construction. Activities:

Requirement review Design review Clear code Establishing standards (ISO 9000-3, etc.) Using CASE tools with built-in check mechanisms

Effectiveness factor: Proportion of the faults remaining after prevention

activities.

SENG521 (Fall 2002) far@enel.ucalgary.ca 10

Fault RemovalFault Removal To detect, by verification and validation, the

existence of faults and eliminate them. Activities:

Code review test

Effectiveness factor: Reduction of failure intensity due to code review. Ratio of failure intensity after test and before test.

SENG521 (Fall 2002) far@enel.ucalgary.ca 11

Fault ToleranceFault Tolerance To provide, by redundancy, service

complying with the specification in spite of faults occurrences.

Activities: Designing and implementing redundancy

Effectiveness factor: Reduction of failure intensity as a result of

redundant design.

SENG521 (Fall 2002) far@enel.ucalgary.ca 12

Fault/Failure ForecastingFault/Failure Forecasting To estimate, by evaluation, the presence of

faults and the occurrences of failures. Activities:

Establishing reliability model Collecting failure data Analysis and interpretation of results

Effectiveness factor: Reduction of failure intensity as a result of

applying reliability engineering.