daimihenrik bærbak christensen1 what is software quality?

DAIMI Henrik Bærbak Christensen 1

What is Software Quality?


Reflection

What is software quality to you ?

Name a program/application/system of high quality?

Low quality?

What aspects of it make it ‘high quality’ or ‘low quality’?


Definition of quality

Criteria for a good definition of quality:

allow us to measure quality in a meaningful way

Why?– avoid dogmas

– measure if techniques really improve quality

– what investment gives most ‘return on investment’?


Quality views Kitchenham & Pleeger

Transcendental view– quality can be recognized but not defined

User view– quality is fitness for purpose

Manufacturing view– quality is conformance to specification

Product view– quality is inherent characteristics of the product

Value-based view– quality is the amount costumer will pay for product


View clashing

Stakeholders have different view on same product

Morale: Trade-offs between cost and quality

The product

Researchers and TeachersProduct view

CustomersUser view

DevelopersManufacturing view

ManagementValue-based view


Reflection

What views did we use in our discussion of good/bad systems before?

dPaSS / dSoftArk course preaches:

– Low coupling / high cohesion

– Favor object composition over class inheritance

What is the view here ???


Quality frameworks

Quality has many faces...


Quality Attributes

... measure quality in a meaningful way.

This requires

– that we define the aspects/qualities we measure– that we agree on some kind of scale: a metric


Measuring quality

Performance

Maintainability

Usability

Quality Attribute

Metric

Measurement

Choose alternatives

Quality Framework


Quality Frameworks

Of course – a lot of different quality frameworks have been proposed.

– Bass, Clements and Kazman: • Architecture Quality Attributes

– Sommerville: Software Engineering books

– IEEE Standard


Quality framework (Bass et al.)

System quality attributes– Availability– Modifiability– Performance– Security– Testability– Usability

Business qualities– Time to market– Cost– Projected lifetime– Targeted market– Roll-out schedule– Integration with legacy sys.

Architectural qualities– Conceptual integrity– Correctness and

completeness– Buildability


Bass’ System Qualities

Availability– probability that the system will be operational when needed

Modifiability– ease with which the system supports change

Performance– response time

Security– ability to withstand attacks/threats

Usability– how easy it is for the user to accomplish a desired task

Testability– ease with which the software can be made to demonstrate its

faults


Discussion

Which of these qualities can be judged by a testing process?

Which can be measured?– quantitatively (hard numbers: “4.3”)– qualitatively (judgment: “low/high”)


Dependability Sommerville

Name systems where one is very important? What is the relation to ‘usefulness’?

equates to its trustworthiness.


IEEE (Burnstein)

IEEE Standard Glossary of SE terminology: 1. Quality relates to the degree to which a

system, system component, or process meets specified requirements.

2. Quality relates to the degree to which a system, system component, or process meets customer or user needs, or expectations.

Exercise: Relate to Kitchenham’s quality views...


War Story

Major Danish company, highly active in services on the WWW.

Requirement specification:

Make something that is smarter than the competitors’ product...


Reliability and failure

Reliability: Probability that a software system will not cause the failure of the system for a specified time under specified conditions.

Failure is then defined by either– the specification (IEEE definition 1)

– the user/customer (IEEE definition 2)


Reliability is in the eyes of the beholder

Note that conditions are part of the definition.

But conditions may change !

– Fault causing crash in Word’s fax service ?

– Developer may be able to use his own prototype tool, but others may make it crash every time…

– Our car’s engine would cut out during heavy rain


Reliability and Failures

Thus fixing any defect and removing a failure will not give same increase in reliability...

Example: two defects– Word cannot save documents due to defect– Word cannot reconfigure button panel due to defect

Which defect removal will increase reliability most?


Quality View

Cem Kaner: The measure of a product’s or service’s

quality is the satisfaction of their customers, not the match to a specification.

A program’s quality is:– the features that make the customer want to use the

program– the flaws that make the customer wish he’d bought

something else


Yet another view

Beizer There can never be an absolute definition for

bugs, nor an absolute determination of the existence. The extent to which a program has bugs is measured by the extent to which it fails to be useful. This is a fundamentally human measure.

Interesting… It makes many programs into defects instead


Ensuring reliability

Reliability is compromised by failures. Failures happen when the system is in an error state, which may be caused by a defect. So – reliability can be enhanced by several measures.– Fault avoidance: simply avoid introducing defects!

– Fault detection and removal: Find and remove the defects before they cause failures.

– Fault tolerance: Ensure that faults does not lead to failures.


Fault avoidance

Use methods, tools, languages, techniques that are known to minimize the probability of introducing defects.– Corollary: Avoid using m,t,l,t that are notoriously

known for introducing defects

That is: Never, ever, use C or C++ Use safe languages (Java, C#) Use proven design techniques

– encapsulation, no goto’s , type-checking, etc.


Fault detection and removal

The primary focus of this course:

Hunt down the defects by what-ever means (execution, review) and remove them.


Fault tolerance

If we know that cannot remove all defects then at least ensure that they do not lead to failures– Exception handling:

• handle IO errors gracefully• null pointer exceptions?

– N-version programming• let n teams develop the same software unit• vote on the n results• Space shuttle maintains five parallel systems

– one running base flight software

– four running mission and flight software


Run-time cycle

Faults cause failures when faulty code is executed with inputs that expose the fault.– I_e: input that will lead the system into error state

Program execution

state

I_e

error states

Input space


Economic consequences

Not all input have the same probability– Which function to you use most in Word?

• save document• reconfigure button panel

– If a defect is present which unit would you rather have it in?

• save document algorithm• reconfigure button panel algorithm

Thus with a given testing budget not all test cases are born equal!


Summary

Quality is many things... There are several frameworks defining quality All, however, must deal with reliability

– Reliability: Probability that a software system will not cause the failure of the system for a specified time under specified conditions

Reliability techniques– fault avoidance; fault removal; fault tolerance

Not all faults are equally important for reliability Testing budget should be used wisely

daimihenrik bærbak christensen1 what is software quality?

Documents

software quality

measure quality

low quality

user view quality

good definition of quality

product slide

metric slide

lowhigh slide