software testing foundations part 2 - testing in software lifecycle

Software Testing Foundations #2Testing in the Software Lifecycle

Nikita Knyshnknysh@gmail.comhttp://www.facebook.com/groups/istqb/

Agenda• Levels of Test: The V-model•Testing New Product Versions•Types of Testing•Testing Related to Changes

Development Activities•Req specification: needs & reqs of the customer,

desired characteristics and features of the system.•Func system design: reqs are mapped onto system

functions & dialogs.•Technical system design: implementation of system is

designed (architecture, interfaces, etc.).•Component specification: task, behavior, inner

structure & interfaces are defined for each subsystem.•Programming.

Levels of Test: The V-modelRequirements

Definition

Functional System Design

Technical System Design

Component Specification

Programming

Component Test

Integration Test

System Test

Acceptance Test

Implementation &Integration

Validation

Verification

V-model: Test Activities #1• Component test: test functionality, robustness, efficiency

and maintainability of components. Techniques: white box, black box, test-driven development.

• Integration test: test of collaboration between components after assembling to expose faults in interfaces and interaction between components. Test drivers, stubs and monitors are used. Integration in the large – integration of several software systems.▫ Integration strategy has to be designed by test manager

taking into account system architecture, project plan and test plan constraints. Top-down, bottom-up, ad-hoc, backbone.

V-model: Test Activities #2• System test: check if the integrated product meets the

specified requirements. System is examined from the point of view of customer and future user. Environment should be as close to the operation environment as possible, but this should NOT be the customer’s operational environment.

• Acceptance test: “Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.” [After IEEE 610]. Customer must be involved.

V-model: Acceptance Test on Lower Levels

•A commercial off-the-shelf software product (COTS) may be acceptance tested when it is installed or integrated.

•Acceptance testing of new functionality may come before system testing (using a prototype).

V-model: Forms of Acceptance Test•According to contract•User acceptance testing•Operational testing•Field testing▫alpha – at the producer’s site▫beta – at the customer’s site

Extent of acceptance test depends on application risk.

Testing New Product Versions•Software maintenance:▫testing after maintenance (anything new or changed is

tested),▫testing after change of environment,▫testing for retirement (data archiving or data migration

into future system).•Release: full regression test (if possible) to find side

effects of changes to the software.

Testing New Product Versions #2•Testing in incremental development: several V-

models after each other, where every next "V" reuses existing test material and adds the tests for everything new, or for higher reliability require ments.

Types of Testing: Overview•Functional•Non-functional•Testing of Software Structure•Testing related to Changes

Types of Testing: FunctionalTest what the system does. Tests are based on functional specs, check system’s input-output behavior and use black-box methods.•Requirements-based testing focuses on single

system functions.•Business-process-based testing focuses on whole

process consisting of many steps.

Types of Testing: Non-functional #1Test how well the (partial) system carries out its functions.

[ISO 9126]: reliability, usability, efficiency, maintainability, portability.

Types of Testing: Non-functional #2• Load test: measuring system behavior when working with the

number of users, number of transaction at a time.• Performance test: measuring system processing speed and

response time (depending on increasing load).• Volume test: observation of system behavior on large amount of

data (e.g. processing very large files).• Stress test: observation of system behavior when it is

overloaded.• Security test: against unauthorized access, denial of service

attacks, etc.• Stability (reliability) test during permanent operation, e.g. time

between failures or failure rate within a given user profile.

Types of Testing: Non-functional #3• Robustness test for response to operating errors, wrong programming

hardware failure, etc., as well as examination of exception handling and recovery.

• Testing of compatibility (to other systems) and data conversion (import / export).

• Testing of different configurations of the system, e.g. OS version, UI language, hardware platform, etc.

• Usability test: ease of learning the system, ease and efficiency of operation, etc. with respect to a specific group of users.

• Checking of documentation for compliance with the system behavior (e.g. user manual and GUI).

• Checking of maintainability: is documentation understandable and up-to-date? Does the system have modular structure, etc.

Types of Testing: Testing Software Structure

Using white box techniques to test object’s internal code or architecture.

•Goal is to cover all structure items.•Most used in component and integration testing.

Testing Related to Changes•Re-test of changed (modified or repaired) parts. •Regression test of whole program (part) after

modification to ensure that no faults are introduced as side effects of changes.

Volume of Regression Test•Defect retest (confirmation test): rerunning of tests

that detected faults that have been then fixed.•Testing of altered functionality: testing of all

program parts that were changed or corrected.•Testing of new functionality: testing of all newly

integrated program parts.•Complete regression test: testing of the whole

system.

Shrinking Regression Test EffortDeciding which test cases we can omit:•Only high-priority test cases (according to test plan)

are repeated.• In the functional test, certain variations (test cases)

are omitted.•Restriction of the test to certain configurations only

(on one OS only, in one language only, etc.)•Restriction of the test to certain subsystems or test

levels.

Thank you!

http://www.facebook.com/groups/istqb/