testing in the lifecycle

Tester Training Course

Hoang V. Nguyen

[email protected]

startnewday85.blogspot.com

Dept. of Computer Science – FIT - HUA

Tester Training Course

TESTING IN THE LIFECYCLE

Tester Training Course 2011


implementation

Detailed Design

Global Design

System requirements

User requirements

Unit Testing

Integration Testing

System Testing

Acceptance Testing


Also know as component, module, program testing

Each “part” is tested in isolation

lowest level and most thorough look at detail

Goals:

• to ensure that the code written for the unit meets its specification, prior to its integration with other units

• both functional and non-functional characteristics

Test bases:

• component requirements

• detailed design

• the code


Who do it?

• usually done by programmer

Reports

• Defects are typically fixed as soon as they are found, with out formally recording the incidents found

Tools:

• Programmer often writes pieces of code for unit test

• Mock frameworks

• Dependency Injection and IoC Containers

• Unit test frameworks


Programmers:

• have to create pieces of code are separately testable

Test Driven Development

• An approach to software development using Unit Test

• Test –first approach

• is an iterative approach to unit test

• is good in practice

• Stubs and DriversA

B

A

Stubs

Driver

B


more than two components/systems

Goals:

• discover defects in interface and interaction between components/systems

• what the set can perform that is not possible individually


Test bases:

• software and system design

• software architecture

• workflows and use-cases


There may be more than one level of integration testing

• component integration testing

• system integration testing

Who do it?

• Developer

• Designer

• Independent tester


Integration strategy

• big-bang

• incremental


In theory

• if we have already tested components why not just combine them all at once? Wouldn’t this save time?

• based on false assumption of no faults

In practice: difficult to isolate any errors found

• takes longer to locate and fix faults

• re-testing after fixes more extensive

==> don’t save time


In theory

• start with a tested component/system

• increment: add a tested component/system and test it

• baselines

Advantages

• easier fault location and fix

• easier recovery from disaster/problems

Some types

• top-down

• bottom-up

• functional


Baselines

• baseline 0: a tested component a

• baseline 1: a+b

• baseline 2: a+b+c

• baseline 3: a+b+c+d

• …

Need to call to lower level components not yet integrated

Stubs: simulate missing components

a

b c

d e f g

h i j k l m

n o

a

cb

d

Stubs


Advantages:

• critical control structure tested first and most often

• can demonstrate system early (show working menus)

Disadvantages:

• needs stubs

• detail left until last

• may be difficult to "see" detailed output (but should have been tested in component test)

• may look more finished than it is


Baselines

• baseline 0: component n

• baseline 1: n+i

• baseline 2: n+i+o

• baseline 3: n+i+o+d

• …

Needs driver to call the base line configuration

Also needs stubs for some baselines

a

b c

d e f g

h i j k l m

n on o

i

d

Drivers

Stubs Drivers: test harness/scaffolding


Advantages:

• lowest levels tested first and most thoroughly (but should have been tested in unit testing)

• good for testing interfaces to external environment (hardware, network)

• visibility of detail

Disadvantages

• no working system until last baseline

• needs both drivers and stubs

• major control problems found last


Baselines

• baseline 0: component a

• baseline 1: a+b

• baseline 2: a+b+d

• baseline 3: a+b+d+h

• …

Need Stubs

Shouldn’t need drivers(if top down)

a

b c

d e f g

h i j k l m

n o

a

b

d

h


Advantages:• control level tested first and most often

• visibility of detail

• real working partial system earliest

Disadvantages• needs stubs


Thread Integration

order of processing some eventdetermines integration order

interrupt, user transaction

minimum capability in time

advantages:

• critical processing first

• early warning ofperformance problems

disadvantages:

• may need complex drivers and stubs

a

b c

d e f g

h i j k l m

n o


Integration Guidelines

minimise support software needed

integrate each component only once

each baseline should produce an easily verifiable result

integrate small numbers of components at once

• one at a time for critical or fault-prone components

• combine simple related components


integration should be planned in the

architectural design phase

the integration order then determines the

build order

• components completed in time for their

baseline

• component development and integration

testing can be done in parallel - saves time


Component testing should be performed by developer

Component testing is alse know as isolation or module testing

Component testing should have completion criteria planned

Component testing does not involve regression testing

Which of the following statements about component testing is not true?

Question


i and ii

i, ii and iii

iii

Which of the following combinations correctly describes avalid aproach to component testingi. Functional testing of the component in isolationii. Structure-based testing of the code with out recording

incidentsiii. Automated tests that rare run until the component

passesiv. Functional testing of the interfaces between modules

Question

ii and iv


System Testing

Component Testing

Integration Testing

Unit Testing

Testing activity which is performed to exposedefects in the interfaces and in the interactionbetween integrated components is:

Question


Statement and branch testing

Usability testing

Security testing

A common test technique during component test is

Question

Performance testing


Top-down integration

Module integration

Bottom-up integration

None of the above

The process starting with the terminal modulesis called

Question


Because incremental integration has better early defects screening and isolation ability

Because big bang integration is suitable only for real time applications

Incremental integration is preferred over big bang integration only for bottom-up development model

Why is incremental integration preferred over big bang integration?

Question

Because incremental integration can compensate for weak and inadequate component testing


Last integration step

end-to-end perspective

Goals:

• discover failure in behaviors of whole system in environment


Test bases

• System and software requirement specifications

• Use cases

• system, user and operation manuals

Who do it

• often and should be independent test team


• Usability

• configuration / installation

• Security

• reliability / qualities

• Documentation

• back-up / recovery

• Storage

• performance, load, stress

• volume


Performance Tests

Timing Tests • response and service times• database back-up times

Capacity & Volume Tests• maximum amount or processing rate• number of records on the system• graceful degradation

Endurance Tests (24-hr operation?)• robustness of the system• memory allocation


Multi-User Tests

Concurrency Tests• small numbers, large benefits• detect record locking problems

Load Tests• the measurement of system behaviour under

realistic multi-user load

Stress Tests• go beyond limits for the system - know what will

happen• particular relevance for e-commerce


Who should design / perform these tests?

Usability Tests

messages tailored and meaningful to (real) users?

coherent and consistent interface?

sufficient redundancy of critical information?

within the "human envelope"? (7±2 choices)

feedback (wait messages)?

clear mappings (how to escape)?


Security Tests

passwords

encryption

hardware permission devices

levels of access to information

authorisation

covert channels

physical security


Configuration and Installation

Configuration Tests• different hardware or software environment

• configuration of the system itself

• upgrade paths - may conflict

Installation Tests• distribution (CD, network, etc.) and timings

• physical aspects: electromagnetic fields, heat, humidity, motion, chemicals, power supplies

• uninstall (removing installation)


Reliability / Qualities

Reliability

• "system will be reliable" - how to test this?

• "2 failures per year over ten years"

• Mean Time Between Failures (MTBF)

• reliability growth models

Other Qualities

• maintainability, portability, adaptability, etc.


Back-up and Recovery

Back-ups• computer functions

• manual procedures (where are tapes stored)

Recovery• real test of back-up

• manual procedures unfamiliar

• should be regularly rehearsed

• documentation should be detailed, clear and thorough


Documentation Testing

Documentation review

• check for accuracy against other documents

• gain consensus about content

• documentation exists, in right format

Documentation tests

• is it usable? does it work?

• user manual

• maintenance documentation


Final stage of validation

Goals

• To provide the end users with confidence that the system will function according to their expectations

Test bases

• Requirement s specification

Who do it

• Customers/users

• Also can include testers


Done by users

To check that the system meets their business needs

Can include

• Factory acceptance testing

• Site acceptance testing


Also called operational readiness testing

To check that the processes and procedures are in place to allow the system to be used and maintained

Include checking:

• Back-up facilities

• Procedures for disaster recovery

• Training for end users

• Maintenance procedures

• Data load and migration tasks

• Security procedures


The criteria for accepting a system can documented in a contact

To check that these criteria have been met

• Not what the user wish they had asked for!

• This system, not wish system


Check that governmental, legal or safety standard have been met


Similarities:

• When software is stable

• To collect comments fault found

how the product meets their expectations

improvement suggestions

Differences :

• Alpha testing is done on developer site

• Beta testing is done on customer site


Integration testing

System testing

Unit testing

Component integration testing

System Integration testing should be done after

Question


Testing performed by potential customers at the developers location.

Testing performed by portential customer at their own locations.

Testing performed by product developers at the customer ’s location

When is beta testing?

Question

Testing performed by product developers at their own locations.


Stress testing

Load testing

Performance testing

Volume testing

Which testing is used to verify that thesystem can perform properly when internalprogram or system limitations have beenexceeed

Question


Additional features being required

The system being migrated to a newoperating platform

The system being retired

Planned upgrade to COTS-based system

New faults being found requiring fixing(these can be ‘hot fixes’)


Maintenance testing is important!


To ensure that software system stillworks correctly (both new and oldcharacteristics)

We have to:• Breath test to establish overall confidence

• Depth test to investigate changes and critical areas

Predominantly regression testing


What to test in maintenance testing

Test any new or changed code

Impact analysis• what could this change have an impact on?

• how important is a fault in the impacted area?

• test what has been affected, but how much?most important affected areas?

areas most likely to be affected?

whole system?

The answer: “It depends”


Impact Analysis assesses the effect on the system of a defect found in regression testing

Impact Analysis assesses the effect of a new person joining the regression test team

Impact Analysis assesses whether or not a defect found in regression testing has been fixed correctly

Impact Analysis assesses the effect of a change to the system to determine how much regression to do

Which of the following is correct?

Question


Unit testing

Load testing

Usability testing

Testing where in we subject the target of thetest, to varying workloads to measure andevaluate the performance behaviors andability of the target and of the test tocontinue to function properly under thesedifferent workloads.

Question

Intergration testing


Breadth Test and Depth Test

Re-Testing

Confirmation Testing

Sanity testing

Methodologies adopted while performing maintenance testing:

Question


Unit, integration, system, validation

System, integration, unit, validation

Unit, integration, validation, system

What is the normal order of activities in which software testing is orgranized?

Question

None of the above


True

False

Beta testing is performed at developingorganization’s site where as Alpha testing isperformed by people at their own locations.

Question


Performed by customers at their own site

Performed by customers at their software developer’s site

Performed by an independent test team

Beta testing is

Question

Useful for test bespoke software


Exit Criteria

How much regression testing should be done

How many more test cases need to written

Different tools to perform regression testing

Impact Analysis helps to decide

Question

testing in the lifecycle

Technology

timetester training

istester training course

lasttester training

codetester training

component test

b b c baseline

unit test test

o baseline