software engineering1 verification: the software should conform to its specification validation:...

Software Engineering 1

Verification: The software should conform to its

specification

Validation: The software should do what the user really

requires

Assuring that a software conforms with the specification and meets user's needs

Verification and Validation


Validation and Verification

• Happens during the whole software life cycle– Requirement validation– Design verification– Code testing– Static code verification


• Dynamic verification Concerned with exercising implementations and observing product operational behaviour (testing)

• Static verification inspection analysis of the static system representation (documents or hard copy of programs) to discover problems

Dynamic and static verification


• Examine source representation of the system (e.g. documents, models, code and test cases)

• Check documents against specification.

• Static analysers are software tools for source text processing

• They parse the program text and try to discover potentially erroneous conditions and bring these to the attention of the V & V team

Static Verification


Static verification

• Program inspection procedure:– Build an inspection team.– Prepare a checklist of the defects you want to check.– Individual preparation– Inspection meeting (go through the code line-by-line)

– You need the specification document and organization standard

– Find errors, but don’t suggest corrections

• More effective and less expensive than dynamic

Inspection checks

Fault class Inspection checkData faults Are all program variables initialised before their values

are used?Have all constants been named?Should the lower bound of arrays be 0, 1, or somethingelse? Should the upper bound of arrays be equal to the size ofthe array or Size -1?If character strings are used, is a delimiter explicitlyassigned?

Control faults For each conditional statement, is the condition correct?Is each loop certain to terminate?Are compound statements correctly bracketed?In case statements, are all possible cases accounted for?

Input/output faults Are all input variables used?Are all output variables assigned a value before they areoutput?

Interface faults Do all function and procedure calls have the correctnumber of parameters?Do formal and actual parameter types match? Are the parameters in the right order? If components access shared memory, do they have thesame model of the shared memory structure?

Storage managementfaults

If a linked structure is modified, have all links beencorrectly reassigned?If dynamic storage is used, has space been allocatedcorrectly?Is space explicitly de-allocated after it is no longerrequired?

Exceptionmanagement faults

Have all possible error conditions been taken intoaccount?


• Defect testing– Tests designed to discover system defects– A successful defect test is one which reveals the presence of

defects in a system– Debugging is concerned with locating and repairing these

errors– Regression testing ?

• Statistical testing– tests designed to reflect the frequency of user inputs. Used

for reliability estimation

Types of testing


The debugging process

Locateerror

Designerror repair

Repairerror

Re-testprogram

Testresults Specification Test

cases


• Unit testing– testing of individual components

– done by programmers during implementation.

• Module testing– testing of collections of dependent components

– Specified during design.

• Sub-system testing– testing collections of modules integrated into sub-systems

– Interface testing

Testing stages


Testing stages

• Integration testing– testing the complete system prior to delivery– planned during the design phase.

• Acceptance testing– testing by users to check that the system satisfies

requirements. – Specified during specification phase.– Alpha testing: used for customized products.– Beta testing: used for generic products.


Defect testing

• The objective of defect testing is to discover defects in programs

• A successful defect test is a test which causes a program to behave in an abnormal way

• Tests show the presence not the absence of defects

• Only exhaustive testing can show a program is free from defects. However, exhaustive testing is impossible


Black-box testing (Functional)

• Approach to testing where the program is considered as a ‘black-box’

• Test cases are based on the system specification • Equivalence partitioning: classification of input & output

into categories which are similar according to system behaviour.– If input is a 5-digit integer between 10,000 and 99,999,

equivalence partitions are: • < 10,000• 10,000 - 99, 999 • > 10, 000

• Choose test cases at the boundary of these sets


• Sometime called white-box testing

• Derivation of test cases according to program structure. Knowledge of the program is used to identify additional test cases

• Objective is to exercise all program statements at least once (not all path combinations)

Structural testing

CodeDerive

Test data

Apply

Test output


Path testing

• Structural test• The objective is to execute every independent

path in the program at least once• Used for unit and module testing• The starting point for path testing is a program

flow graph that shows nodes representing program decisions and arcs representing the flow of control

• Ignore sequential statements in the flow graph


• Describes the program control flow

• Used as a basis for computing the cyclomatic complexity

• Complexity = Number of edges - Number of nodes +2

• The number of tests to test all control statements equals the cyclomatic complexity

Program flow graphs


Example Procedure sort

do while records remainread recordif record field1 =0

then process recordstore in buffer

else if record field2 =0then reset counterelse process record

end ifend if

end doend

1:

2:3:

4:5:6:7:

8:9:


• 1, 9

• 1, 2,3, 8, 1, 9

• 1, 2, 4, 5, 7, 8, 1, 9

• 1, 2, 4, 6, 7, 8, 1, 9

• Test cases should be derived so that all of these paths are executed

Example flow graph1

2

43

6 5

8

9

7


Defect testing approaches

Interfacetesting

Functionaltesting

Structuraltesting

Sub-systemSystemUnit andmodule

Testingteam

Developmentteam


• Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light

• Stressing the system tests failure behavior.• Systems should not fail catastrophically. Stress

testing checks for unacceptable loss of service or data

• Particularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded

Stress testing


• The components to be tested are object classes that are instantiated as objects

• Testing levels:– Test operations associated with objects– Test object classes– Test clusters of cooperating objects– Test the complete OO system

Object-oriented testing


Object class testing

• Complete test coverage of a class involves– Testing all operations associated with an object– Setting and reading all object attributes– Exercising the object in all possible states (Use a state

model to identify state transitions)

• Inheritance makes it more difficult to design object class tests as the information to be tested is not localised


Object integration

• Levels of integration are less distinct in object-oriented systems

• Cluster testing is concerned with integrating and testing clusters of cooperating objects– Use-case scenario or sequence diagram testing

• Testing is based on a user interactions with the system

• Advantage is that it tests system features as experienced by users


Test Plan

• Concerned with defining standards for the test process.

• Manager’s responsibility

• Continuously changing document

• Test cases:– Objective of the test– Input of test– Expected output


The V-model of development

Requirementsspecification

Systemspecification

Systemdesign

Detaileddesign

Module andunit codeand tess

Sub-systemintegrationtest plan

Systemintegrationtest plan

Acceptancetest plan

ServiceAcceptance

testSystem

integration testSub-system

integration test


The Test Plan• Section 1: Introduction

– Objective of testing– Tested items (documents tested)

– The testing process (phases and strategies)

– Hardware and software requirements– Constraints (anticipated problems)

• Section 2: Testing schedule (when, who and how long)

• Section 3: Test recording procedures• Section 4: Test procedure

– Module tested– Objective and type of testing– Test data– Expected outcome

software engineering1 verification: the software should conform to its specification validation:...

Documents

beta testing

alpha testing

errorsregression testing

system behaviour

complete system

program text

specification phase

program structure