Software Reviews & testingSoftware Reviews & testing
An OverviewAn Overview
ObjectiveObjective
To provide an overview of procedures for To provide an overview of procedures for software reviews & testing activitiessoftware reviews & testing activities
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 33
Intended audienceIntended audience
Project team who are involved in software Project team who are involved in software development, reviews & testingdevelopment, reviews & testing
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 44
SDLCSDLC
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 55
UR PhaseUR Phase
A preliminary phase to the software development life cycle called the ‘User Requirements Definition Phase' (UR phase).
The UR phase is the ‘problem definition phase’ of a software project. The review of the URD is done by the users. The approved URD is the input to the SR phase.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 66
SR PhaseSR Phase
The SR phase is the ‘analysis’ phase of a software project.
A vital part of the analysis activity is the construction of a ‘model’ describing ‘what’ the software has to do, and not ‘how’ to do it.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 77
SR PhaseSR Phase
The SRD must be reviewed formally by the users, by the computer hardware and software engineers, and by the managers concerned, during the Software Requirements Review (SR/R).
The approved SRD is the input to the AD phase.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 88
AD PhaseAD Phase
The purpose of the AD phase is to define the “structure of the software”.
This model is transformed into the architectural design by allocating functions to software components and defining the control and data flow between them.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 99
DD PhaseDD PhaseThe purpose of the DD phase is to detail the design of the software, and to code, document and test it.
During this phase, unit, integration and system testing activities are performed according to verification plans established in the SR and AD Phases. As well as these tests, there should be checks on software quality.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1010
UR Reviews
User requirements which are rejected in the review process do not have to be removed from the URD, especially if it is anticipated that resources may be available at some later date to implement them.
Nonapplicable user requirements shall be clearly flagged in the URD.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1111
SR Reviews
The outputs of the SR phase shall be formally reviewed during the Software Requirements Review (SR/R). This should be a technical review Participants should include the users, the operations personnel, the developers and the managers concerned
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1212
ADD Reviews
The architectural design should be reviewed and agreed layer by layer as it is developed during the AD phase.
Walkthroughs should be used to ensure that the architectural design is understood by all those concerned. Inspections of the design, by qualified software engineers, may be used to eliminate design defects.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1313
DDD Reviews
The project leader should participate in these reviews, together with the team leader and team members concerned.
After modules have been coded and successfully compiled, walkthroughs or inspections should be held to verify that theimplementation conforms to the design..
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1414
End of ReviewsBy the end of the UR review, the SR phase section of the SPMP shall be produced (SPMP/SR).SCMP shall be produced (SCMP/SR).SVVP shall be produced (SVVP/SR).SQAP shall be produced (SQAP/SR)Acceptance Test Plan.
During the SR phase, the AD phase section of the SPMP shall be produced (SPMP/AD).SCMP shall be produced (SCMP/AD).SVVP shall be produced (SVVP/AD).SQAP shall be produced (SQAP/AD).System Test Plan.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1515
End of ReviewsDuring the AD phase, DD phase section of the SPMP shall be produced (SPMP/DD).SCMP shall be produced (SCMP/DD).SVVP shall be produced (SVVP/DD).SQAP shall be produced (SQAP/DD).Integration Test Plan.
During the DD phase, the TR phase section of the SPMP shall be produced (SPMP/TR).SCMP shall be produced (SCMP/TD).SVVP shall be produced (SVVP/TD).SQAP shall be produced (SQAP/TD).Unit Test Plan.
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 1616
Reviews improve schedule performance
Reviews reduce rework.Rework accounts for 44% of development. Cost!
Requirements (1%)Design (12%)
Coding (12%) Testing (19%)
Reviews are pro-active tests.- Find errors not possible through testing.
Reviews are training.- Domain, corporate standards, group.
Testing activitiesTesting activities
1. Unit Testing
2. Module/ Integration Testing
3. System Testing
4. Acceptance Testing
5. User Documentation Review
Unit Testing Unit Testing
This is the testing performed on the most basic This is the testing performed on the most basic item of the software product i.e. the individual item of the software product i.e. the individual source code component to uncover the errors source code component to uncover the errors and to ensure that this is meeting all the and to ensure that this is meeting all the specifications as described in SDDspecifications as described in SDD . .
Testing activitiesTesting activities
Module/Integration TestingModule/Integration Testing
This is the testing performed at sub-system level This is the testing performed at sub-system level to uncover errors related to interfacingto uncover errors related to interfacing ..
Testing activitiesTesting activities
System TestingSystem Testing
System testing is a validation activity used to System testing is a validation activity used to demonstrate that the entire software product is demonstrate that the entire software product is conforming to the user requirement expressed conforming to the user requirement expressed and agreed upon earlier. This test is carried out and agreed upon earlier. This test is carried out after integrating all the components of the after integrating all the components of the product in a common environmentproduct in a common environment . .
Testing activitiesTesting activities
Acceptance Testing Acceptance Testing
Acceptance testing is a validation activity before Acceptance testing is a validation activity before the system is accepted by the user for the system is accepted by the user for operational useoperational use..
Testing activitiesTesting activities
User Documentation ReviewUser Documentation Review
This is to verify that the user documentation has This is to verify that the user documentation has explained all the features of the software explained all the features of the software delivered correctly and clearly along with the delivered correctly and clearly along with the operational procedure .operational procedure .
Testing activitiesTesting activities
Testing: A process of executing a Testing: A process of executing a program with the intent of finding an program with the intent of finding an errorerror
A good test has a high probability of A good test has a high probability of finding an undiscovered errorfinding an undiscovered error
A successful test uncovers an A successful test uncovers an undiscovered errorundiscovered error
Testing ObjectivesTesting Objectives
Testing strategy is a road map describingTesting strategy is a road map describing
– Steps to be conducted as part of testingSteps to be conducted as part of testing
– When these steps should be planned and When these steps should be planned and undertakenundertaken
– How much effort, time, and resources How much effort, time, and resources will be requiredwill be required..
Testing strategyTesting strategy
Testing strategy must incorporateTesting strategy must incorporate– Test planningTest planning– Test case designTest case design– Test executionTest execution– Data collection and evaluationData collection and evaluation
Testing strategyTesting strategy
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 2626
Test known internal workings to assure that internal operation Test known internal workings to assure that internal operation performs according to specs. Tests logical paths and procedural performs according to specs. Tests logical paths and procedural detaildetail
Using white box testing, the software engineer can derive test Using white box testing, the software engineer can derive test cases thatcases that
– Guarantee that all independent paths within a module have Guarantee that all independent paths within a module have been exercised at least oncebeen exercised at least once
– Exercise all logical decisions on the their true and false Exercise all logical decisions on the their true and false sidessides
– Execute all loops at their boundaries and within their Execute all loops at their boundaries and within their operational boundsoperational bounds
– Exercise internal data structures to assure their validityExercise internal data structures to assure their validity
White Box TestingWhite Box Testing
Focuses on functional requirements of softwareFocuses on functional requirements of software– Incorrect or missing functionsIncorrect or missing functions– Interface errorsInterface errors– Error in data structure & external data base accessError in data structure & external data base access– Performance errorsPerformance errors– Initialization & termination errorsInitialization & termination errors
Tends to be applied during later stages of testingTends to be applied during later stages of testing
Black Box TestingBlack Box Testing
Number of possible logical paths, even in small Number of possible logical paths, even in small programs, can be extremely largeprograms, can be extremely large
Therefore exhaustive white box testing is impracticalTherefore exhaustive white box testing is impractical
Often the best test strategy is to perform both white Often the best test strategy is to perform both white box and black box testingbox and black box testing
Test Case DesignTest Case Design
Flow Graph NotationFlow Graph Notation
One or more non-branching PDL or source codestatementsSequence
If
While
Until
Case
Flow GraphFlow Graph1
2,3
10
11
6
7 8
9
4,5
Edge
Node
Region
R1
R4
R3
R2
Nodes, Edges, And RegionsNodes, Edges, And Regions
•Each circle is a node and represents one or more procedural statements (a sequence of process boxes and decision diamonds can map into a single node; each node that contains a condition is called a predicate node•Arrows are called edges and must terminate at a node•Areas bounded by edges and nodes are called regions, including the area outside the graph
Cyclomatic ComplexityCyclomatic Complexity
•Cyclomatic complexity provides a quantitative measure of the logical complexity of a program
–It is the upper bound for the number of tests that must be conducted to assure that all statements have been executed at least once
•Cyclomatic complexity, V(G) is given by the graph
V(G) = E - N + 2, where E is the number of flow graph edgesand N is the number of flow graph nodes
V(G) = P + 1, where P is the number of predicate nodescontained in the flow graph
V(G) = number of regions
Unit TestingUnit Testing
•Interface
•Local data structure
•Boundary conditions
•Independent paths
•Error-handling paths
Boundary TestingBoundary Testing
•Last and probably most important unit test step–Errors often occur when nth element of n-dimensional array is processed, when ith repetition of loop with i passes is invoked, when max or min allowable value is encountered–Exercise data structure, control flow, and data values just below, at, and just above maxima and minima
Integration TestingIntegration Testing
•Construct program structure while testing to uncover errors associated with interfacing
–Data can be lost across interface–One module can have an inadvertent adverse effect on another module–Subfunctions, when combined, may not provide the desired total function–Individually acceptable imprecision may be magnified to unacceptable levels
System TestingSystem Testing
•Recovery testing is a system test that forces software to fail in a variety of ways and verifies that recovery is performed•Security testing attempts to verify that system protection systems work•Stress testing executes a system in an abnormal manner - frequency of interrupts, input data rates, use of memory•Performance testing - resource utilization, execution intervals
NOV-26-2003NOV-26-2003 AG/MKT/OverviewAG/MKT/Overview 3737
Process - Review & testing activitiesProcess - Review & testing activities
For detailed descriptions & tasks of review &
testing activities, refer to process manual -
Software Verification, Validation & Testing (SVVT) Manual
Thank YouThank You