gp software testing(2)

8/8/2019 GP Software Testing(2)

http://slidepdf.com/reader/full/gp-software-testing2 1/44

SOFTWARE TESTING 0791914408

0801914408

INTRODUCTION

Software testing is any activity aimed at evaluating an attribute or capability of a program or system

and determining that it meets its required result. Testing is very important but an expensive activity. It

is more than just a running program with the intention of finding faults. This is a unique and crucial

area with altogether different problems. Although crucial to software quality and widely deployed by

programmers and testers , software testing still remains an art , due to the limited understanding of the

principles of software . The difficulty in software testing stem from the complexity of software .The purpose pf testing can be quality assurance, verification and validation or reliability estimation.

Testing can be used as a generic metric as well. Software testing is a trade –off between budget, time

and quality.

Researchers , practitioners and developers are working hard to establish ways and means for effective

testing in order to deliver good quality maintainable software.

giBS 1




0801914408

WHAT IS SOFTWARE TESTING ?

The concept of software testing has evolved from simple program” check-out” to a broad set

of activities that cover the entire software life-cycle.

There are five distinct levels of testing :-

Debug: It is defined as the successful correction of s failure.

Demonstrate- The process of showing that major features work with typical input.

Verify: The process of finding as many faults in the application under test as possible.

Validate: The process of finding as many faults in requirements, design and AUT.

Prevent: To avoid errors in development of requirements, design , implementation by self-checking techniques , including “test before design”

giBS 2




0801914408

Some terminologies

Error, Mistake, Bug, Fault and Failure

People make errors. A good synonym is mistake. This may be syntax error or

misunderstanding of specifications. Sometimes, there are logical errors. When developers

make mistakes while coding, we call these mistakes “bugs”. A requirement error may be

magnified during design, and amplified still more during coding. If it could not be detected

prior to release, it may have serious implications in the field.

An error may lead to one or more faults. It is more precise to say that a fault is the

representation of an error, where representation is the mode of expression, such as narrative,

data flow diagrams, ER diagrams, source code etc. Defect is a good synonym for fault. If

fault is in source code, we call it a bug.

A failure occurs when a fault executes. It is the departure of the output of program from the

expected output. Hence failure is dynamic. The program has to execute for a failure to occur.

Test, Test Case and Test Suite

Test and Test case terms are used interchangeably. In practice, both are same and are treatedas synonyms. Test case describes an input description and an expected description. Inputs are

of two types: preconditions (circumstances that hold prior to test case execution) and the

actual inputs that are identified by some testing methods. Expected outputs are also of two

types: post conditions and actual outputs. Every test case will have identification.

During testing, we set necessary preconditions, give required inputs to program, and compare

the observed output with expected output to know the outcome of a test case. If expected and

observed outputs are different, then, there is a failure and it must be recorded properly in

order to identify the cause of failure. If both are same, then, there is n failure and program

behaved in the expected manner.

Test cases are valuable and useful- at least as valuable as source code. They need to be

developed, reviewed, used, managed, and saved.

The set of test cases is called a test suite. We may have a test suite of all possible test case.

We may have a test suite of effective/good test cases. Hence any combination of test cases

may generate a test suite.

giBS 3




0801914408

Verification and validation

Verification and validation are often used interchangeably but have different meanings.

Verification is the process of confirming that software meets its specification.

However, Validation is the process of confirming that software meets the customer’s

requirements. Both the definitions may seem to be similar. One is related to specification and

other is related to customer’s requirements. We should not assume that our specifications are

always correct. We should verify the specifications and validate the final product. In general,

we may say:

Verification: Checking the software with respect to specifications.

Validation : Checking the software with respect to customer’s expectations.

Alpha, Beta and Acceptance testing

Acceptance testing is used when the software is developed for a specific customer. A series

of tests are conducted to enable the customer to validate all requirements. These tests are

conducted by the end user/ customer and may range from adhoc tests to well planned

systematic series of tests. Acceptance testing may be conducted for few weeks or months.

The discovered errors will be fixed and better quality software will be delivered to the

customer.

The terms alpha and beta tests are used when the software is developed as a product for

anonymous customers. Hence formal acceptance testing is not possible in such cases The

alpha tests are conducted at the developer’s site by a customer. These tests are conducted in a

controlled environment. Alpha testing may be started when formal testing process is near

completion.

The beta tests are conducted by the customers/end users at their site. Beta testing isconducted in a real environment that cannot be controlled by the developer. Customers are

expected to report failures, if any, to the company. After receiving such failure reports,

developers modify the code and fix the bug and prepare the product for final release.

Quality Assurance and Quality Control

giBS 4




0801914408

White Box Testing

Every software product is realized by means of a program code. White box testing is a way

of testing the external functionality of the code by examining and testing the program code

that realizes the external functionality. It is also known as Clear box, glass box or open box

testing or structural testing. It takes into account the program code , code structure and

internal design flow.

A number of events come about because of incurring translation of requirements and design

into program code. Some other defects are created by programming errors and programming

language. The different methods of white box testing can help reduce the delay between the

injection of defect in the program code and its detection. Furthermore, since the program

code represents what the product actually does, testing by looking at a program code makes

us get closer to what the product actually doing.

Classification of White Box Testing

1. Static Testing

2. Structural Testing

3. Path Testing

Static Testing

Static testing is the type of testing which requires only the source code of the product, not the

binaries or executables. Static testing does not involve executing the programs on computer but involves select people go and through the code to find out whether

• The code works according to the functional requirement;

• The code has been written in accordance with the design developed earlier in the project

life cycle;

• The code for any functionality has been missed out;

• The code handles error properly.

There are multiple methods to achieve static testing by humans. These are:

1. Desk checking of the code2. code walkthrough

3. Code review

4. Code inspection

giBS 6




0801914408

Desk Checking

Normally done manually by the author of the code, desk checking is a method to verify the

portions of the code for correctness. Such verification is done by comparing the code with the

design or specifications to make sure that the code does wht it supposed to do and

effectively. This is the desk checking that most programmers do before compiling andexecuting the code. Whenever errors are found, the author applies the correction for errors on

the spot.

The main advantage offered by this method is that the programmer who knows the code and

the programming language very well is well equipped to read and understand his/her code.

Some of the disadvantages of this method of testing are:

1. A developer is not the best person to detect problems in his/her own code.

2. Developers generally prefer to write new code rather than do any form of testing.

3. This method is essentially person-dependent and informal and thus may not work consistently across all developers.

Code Walkthrough

This method and formal inspection are group-oriented methods. Walkthroughs are less

formal than inspections. The line drawn in formalism between walkthroughs and inspections

is very thin and varies from organization to organization. The advantage that walkthrough

has over desk checking is that it brings multiple perspectives. In walkthroughs, a set of

people look at the program and raise questions for the author. The author explains the logic

of the code, and answers the questions. If the author is unable to answer some questions, he

or she then takes those questions and finds their answers. Completeness is limited to the area

where questions are raised by the team.

Formal inspection

Formal inspection also known as code inspection is a method with a high degree of

formalism. The focus of this method is to detect all faults , violations and other side effects.

This method increases the number of defects detected by

1. demanding thorough preparation before an inspection/review.2. enlisting multiple diverse views;

3. assigning specific roles to the multiple participants; and

4. going sequentially through the code in the structured manner.

giBS 7




0801914408

Code Coverage Testing is made up of the following types of coverage.

1. Statement coverage

2. path coverage3. Function coverage

Statement Coverage

Statement coverage refers to writing the test cases that execute each of the program

statement. One can start with the assumption that more the code covered, the better is the

testing of the functionality, as a code realizes functionality. Based on this assumption, the

code coverage can be achieved by providing coverage to each of the statements.

• Sequential control flow

• Two-Way decision statements like if then else

• Multi-way decision statements like switch

• Loops like while-do, repeat-until and for

Path Coverage

In Path coverage, we split a program into number of distinct paths. A program can start from

the beginning and take any of the paths to its completion.

giBS 9




0801914408

Path Testing

Path testing is the name given to a group of test techniques based on judiciously selecting a

set of test paths through the program. If the set of paths is properly chosen, then it means thatwe have achieved some measure of test thoroughness. It requires complete knowledge of the

program’s structure and used by developers to unit test their own code. The effectiveness of path testing rapidly deteriorates.

This type of testing involves:

• Generating a set of paths that will cover every branch in the program.

• Finding a set of test cases that will execute every path in this set of program paths.

The two steps are not necessarily executed in sequence. Path generation can be performed

through the static analysis of the program control flow and be automated.

Flow Graph

The control flow f a program can be analyzed using a graphical representation known as flow

graph. The flow graph is a directed graph in which nodes are either entire statements or fragments of a statement, and edges represents flow of control.

A flow graph can easily be generated from the code of any problem. The basic constructs of flow graph are given as:

DD Path Graph

As we know, flow graph is the first step of path testing. The second step is to draw a DD path

graph from the flow graph. The DD path is known as decision to decision path graph. Here,

giBS 10




0801914408

we concentrate only on decision nodes. The nodes of flow graph, which are in a sequence are

combined into a single node.

Hence, DD path is a directed graph in which nodes are sequence of statements and edges

represent control flow between nodes.

Independent Path

The DD path is used to find independent paths. We are interested to execute all independent paths at least once during path testing.

An independent path is any path through the DD path graph that introduces at least one new

set of processing statements or new conditions. Therefore, an independent path must movealong at least one edge that has not been traversed before the path is defined.

It is quite interesting to use independent paths in order to ensure that

• Every statement in the program has been executed at least once.

• Every branch has been exercise for true and false conditions.

Example:

Consider the program for the determination of the nature of roots of a quadratic equation. Its

input is triple of positive integers and values may be from interval [0, 100].

The output may have one of the following words:

Not a quadratic equation, Real roots, Imaginary roots, Equal roots

Draw the flow graph and DD path graph. Also find independent paths from the DD pathgraph

#include<stdio.h>

#include<conio.h>#include<math.h>

Int main()

{

int a,b,b,validInput=0,d; printf(“Enter the ‘a’ value:”);

scanf(“%d”,&a);

printf(“Enter the ‘b’ value:”);scanf(“%d”,&b);

printf(“Enter the ‘c’ value:”);

scanf(“%d”,&c);if((a>=0) && (a<=100) && (b>=0) && (b<=100) && (c>=0) && (c<=100))

{

validInput=1;if(a==0)

giBS 11




0801914408

{

validInput=-1;

}}

if (validInput=1)

{d=b*b-4*a*c;

if (d==0)

{ printf(“The roots are equal”);

}

else if(d>0)

{ printf(“The roots are real”);

}

else

{ printf(“Roots are imaginary”);

}}

else if (validInput==-1)

{

printf(“Not a quadratic equation”);}

else

{ printf(“The inputs belong to invalid range”);

}

getche();return 1;

}

giBS 12




0801914408

giBS 13

51 2 3 4 6 7

1

8

8

2

9

1

6

1

3

3

1

2

3

6

1

5

1

4

1

7

9

3

1

1

0

3

2

3

52

2

3

7

2

1

1

1

2

5

3

9

2

6

33

2

8

3

8

3

0

2

3

3

4

2

7

1

9

2

4

2

0




0801914408

giBS 14

A

R

B

N

C

D

L

F

E

I

H

M

O

G

K J

P

S

Q




0801914408

The mapping table for DD path graph is:

Flow Graph

Nodes

DD Path graph

corresponding nodes

Remarks

1 to 10 A Sequential nodes11 B Decision node

12 C Intermediate node

13 D Decision node

14,15 E Sequential nodes

16 F Two edges are combined here

17 G Two edges are combined and decision

node

18 H Intermediate node

19 I Decision node

20,21 J Sequential nodes

22 K Decision node23,24,25 L Sequential nodes

26 to 29 M Sequential nodes

30 N Three edges are combined here

31 O Decision node

32,33 P Sequential nodes

34 to 36 Q Sequential nodes

37 R Three edges are combined

38,39 S Sequential nodes with exit node

Independent paths are:

1) ABGOQRS

2) ABGOPRS3) ABCDFGOQRS

4) ABCDEFGOPRS

5) ABGHIJNRS

6) ABGHIKLNRS7) ABGHIKMNRS

giBS 15




0801914408

Cyclomatic Complexity

The cyclomatic complexity is also known as structural complexity because it gives internalview of the code. This approach is used to find the number of independent paths through a

program. This provides us the upper bound for the number of tests that must be conducted to

ensure that all statements have been executed at least once and every condition has beenexecuted on true and false side. If a program has backward branch then it may have infinite

number of paths. Although it is possible to define a set of algebraic expressions that gives the

total number of possible paths through a program, however, using total number of paths has been found to be impractical. Because of this, the complexity measures is defined in terms of

independent paths- that when taken in combinations will generate every possible path. An

independent path is any path through the program that introduces at least one new set of

processing statements or a new condition.Cyclomatic complexity of a graph G with n vertices, e edges, and P connected components

is:

V(G) = e-n+2P

Properties of Cyclomatic Complexity

1) V(G) >=1

2) V(G) is the maximum number of independent paths in graph G.

3) Inserting & deleting functional statements to G does not affect V(G).4) G has only one path if and only if V(G)=1.

5) Inserting a new row in G increases V(G) by unity.

6) V(G) depends only on the decision structure of G.

Cyclomatic complexity V(G) of a flow graph G is equal to the number of predicate nodes plus one.

Cyclomatic complexity is equal to the number of regions of flow graph.

Example:- Calculate its cyclomatic complexity by all three methods

1:- V(G) = e -n+2P=13-10+2=5

2:- V(G) = 1 1 + 1

= 4+1=5

3:- V(G) =number of regions

= 5

giBS 16




0801914408

giBS 17




0801914408

Advantages of White Box Testing are

i) As the knowledge of internal coding structure is prerequisite, it becomes very easy to find

out which type of input/data can help in testing the application effectively.

ii) The other advantage of white box testing is that it helps in optimizing the code

iii) It helps in removing the extra lines of code, which can bring in hidden defects.

Disadvantages of White box testing are:

i) As knowledge of code and internal structure is a prerequisite, a skilled tester is needed to

carry out this type of testing, which increases the cost.

ii) And it is nearly impossible to look into every bit of code to find out hidden errors, which

may create problems, resulting in failure of the application.

giBS 18




0801914408

Black Box Testing

Black box testing involves looking at the specifications and does not require examining the

code of a program. Black box testing is done from the customer’s viewpoint. The testengineer engaged in black box testing only knows the set of inputs and expected outputs and

is unaware of how those inputs are transformed into outputs by the software.

Black box tests are convenient to administer because they use the complete finished product

and do not require any knowledge of its construction. Independent test laboratories can

administer black box tests to ensure functionality and compatibility.

Black box testing thus requires a functional knowledge of the product to be tested. If it does

not mandate the knowledge of the internal logic of the system nor does it mandate the

knowledge of the programming language used to build the product.

giBS 19

Input

test

data

Output

test

data

Syste

m

under

test




0801914408

Need of Black Box Testing

Black box testing helps in the overall functionality verification of the system under test.

• Black box testing is done based n requirements:- It helps in identifying any incomplete,

inconsistent requirement as well as any issues involved when the system is tested as a

complete entity.

• Black box testing addresses the stated requirements as well as the implied requirements: -

Not all the requirements are stated explicitly, but are deemed implicitly.

• Black box testing encompasses the end user perspectives:- Since we want to test the

behavior of a product from an external perspective, and-user perspectives are an integral partof black box testing.

• Black box testing handles valid and invalid inputs:- It is natural for users t make errors

while using a product. Hence, it is not sufficient for black box testing to simply handle valid

inputs. Testing from the end user perspective includes testing for these error or invalid

conditions. This ensures that the product behaves as expected in a valid situation and does

not hang or crash when provided with an invalid input.

When to do Black Box Testing?

Black box testing activities require involvement of the testing team from the beginning of the

software project life cycle, regardless of the software development life cycle model chosen

for the project.

Testers can get involved right from the requirement gathering and analysis phase for the

system under test. Test scenarios and test data are prepared during the test construction phase

of the test life cycle, when the software is in the design phase.

Once the code is ready and delivered for testing, test execution can be done. All the test

scenarios developed during the construction phase are executed. Usually, a subse of these test

scenarios is selected for regression testing.

giBS 20




0801914408

Methods of Black Box Testing

• Requirements based testing

• Positive and Negative testing

• Boundary value analysis

• Decision tables

• Equivalence partitioning

• State based testing

Requirements Based Testing

Requirements testing deals with validating the requirements given in the software

requirement specification of the software system.

All requirements are not explicitly stated, some of the requirements are implied or implicit.

Explicit requirements are stated and documented as part of the requirement specification.

Implied or implicit requirements are those that are not documented but assumed to be

incorporated in the system.

The precondition for requirements testing is the detailed review of the requirements

specification. Requirement review ensures that they are consistent, correct, complete and

testable. This process ensures that some implied requirements are converted and documented

as explicit requirements, thereby bringing better clarity to requirements and making

requirements based testing more effective.

All explicit requirements and implied requirements are collected and documented as “testrequirement specification (TRS)”. Requirement based testing can also be conducted based on

such TRS, as it captures the testers perspective as well.

Requirements are tracked by a requirement traceability matrix (RTM). An RTM traces all the

requirements from there genesis through designed, development and testing. This matrix

evolves through the life cycle of the project. To start with these requirements are given a

giBS 21




0801914408

unique id along with brief description. The requirement identifier and description can be

taken from the requirements specification or any other available document that lists the

requirement to be tested fro the product.

Positive and Negative Testing

Positive testing tries to prove that a given product does what it is supposed to do. When a test

case verifies the requirement of the product with a set of expected output, it is called positive

test case. The purpose of positive testing is to prove that the product works as per

specification and expectations. A product delivering an error when it is expected to give an

error, is also a part of positive testing. Positive testing can thus be said to check the product’s

behavior for positive and negative conditions as stated in the requirement.

Negative testing is done to show that the product does not fail when an unexpected input is

given. The purpose of negative testing is to try and break the system. Negative covers

scenarios for which the product is not designed and coded. In other words, the input values

may not have been represented in the specification of the product.

The difference between positive testing and negative is in their coverage. For positive testing

if all documented requirements and test conditions are covered, then coverage can be

considered to be 100 percent. if the specifications are very clear , then coverage can be

achieved . In contrast there is no end to negative testing, and 100 percent coverage in

negative testing is impractical. Negative testing requires a high degree of creativity among

the testers to cover as many” unknowns” as possible to avoid failure at a customer site.

Boundary Value Analysis

Experience shows that test cases that are close to boundary conditions have a higher chanceof detecting an error. Here boundary condition means, an input value may be on the

boundary, just below the boundary, or just above the boundary. Suppose we have an input

variable ranges from 1 to 100. The boundary values are 1, 2, 99, 100.

giBS 22




0801914408

The basic idea of boundary value analysis is to use input variable values at their minimum,

just above minimum, a nominal value, just below their maximum, and at their maximum.

Here, we have an assumption of reliability theory known as “single fault” assumption. Thissays that failures are rarely the result of the simultaneous occurrence of two faults. Thus,

boundary value analysis test cases are obtained by holding the values of all but one variable

at their nominal values and letting that variable assume its extreme value. The boundary

value analysis yields 4n+1 test cases for a program of n variables.

Example

Consider a program for the determination of the nature of roots of a quadratic equation. Its

input is a triple of positive integers (say a, b, c) and values may be from interval [0, 100]. The

program output may have one of the following words:

[Not a quadratic equation, Real roots, Imaginary roots, Equal roots]. Design the boundary

value test cases.

Solution:

Quadratic equation will be of type:

ax2+bx+c=0

Roots are real if (b2-4ac)>0

Roots are imaginary if (b2-4ac) < 0

Roots are equal if (b2-4ac) = 0

Equation is not quadratic if: a = 0

The boundary value test cases are:

Test

Case

a b c Expected Output

giBS 23




0801914408

1 0 50 50 Not Quadratic

2 1 50 50 Real Roots

3 50 50 50 Imaginary Roots





8 50 99 50 Imaginary roots

9 50 100 50 Equal Roots





giBS 24




0801914408

Robustness Testing

It is nothing but the extension of boundary value analysis. Here, we would like to see, what

happens when the extreme values are exceeded with a value slightly greater than themaximum and a value slightly less than minimum. It means, we want to g outside the

legitimate boundary of input domain. This type of testing is quite common in electric and

electric circuits.

There are four additional test cases which are outside the legitimate input domain. Hence

total test cases in robustness testing are 6n+1, where n is the number of input variables.

Example:

Consider the program for the determination of nature of roots of quadratic equation as

explained in boundary value analysis.

Solution:

As we know, robust test cases are 6n+1. Hence, in 3 variable input cases total number of test

cases is 19 as given below:

Test

Case

x y z Expected Output

1 -1 50 50 Invalid Input

2 0 50 50 Not Quadratic Equation





7 101 50 50 Invalid Input

giBS 25




0801914408

8 50 -1 50 Invalid Input






14 50 50 -1 Invalid Input






Worst- Case Testing

If we reject “singe fault” assumption theory of reliability and may like to see what happens

when more than one variable has an extreme value. In electronic circuits analysis this is

called “worst case analysis”. It is more enough in the sense that boundary value test cases are

a proper subset of worst case test cases. It requires more effort. Worst case testing for a

function of n variables generates 5n test cases.

giBS 26




0801914408

Equivalence Class Testing

In this method, input domain of a program is partitioned into a finite number of equivalence

classes such that one can reasonably assume, but not be absolutely sure, that the test case of arepresentative value of each class is equivalent to a test of any other value. That is, if one test

case in a class detects an error, all other test cases in the class would be expected to find same

error. Conversely, if a test case did not detect an error, we would expect that no other test

cases in the class would find an error. Two steps are required in implementing this method:

1. The equivalence classes are identified by taking each input condition and partitioning into

valid and invalid classes. For example, if an input condition specifies a range of values from

1 to 999, we identify one valid equivalence class [1<item<999]; and two invalid equivalence

classes [item < 1] and [ item > 999 ].

2. Generate the test cases using the equivalence classes identified in the previous step. This

is performed by writing test cases covering all the valid equivalence classes. Then a test case

is written for each invalid equivalence class so that no test case contains more than one

invalid class. This is to ensure that no two invalid classes mask each other.

Input domain Output Domain

giBS 27

Invalid Input

Valid

Inputs

Syste

m

under

Test

Outputs




0801914408

Equivalence Partitioning

giBS 28




0801914408

Example:

Consider the program for the determination of nature of roots of quadratic equation as

explained in boundary value analysis.

Solution:

Output domain equivalence class test cases can be identified as follows:

O1 = { <a,b,c>: Not a quadratic equation if 1=0}

O2 = {<a,b,c>: Real roots if (b2-4ac) > 0 }

O3 = { <a,b,c>: Imaginary roots if (b2

-4ac) < 0 }

O4 = {<a,b,c>: Equal roots if (b2-4ac) = 0 }

The number of test cases can be derived from above relations and shown below:

Test

Case

a b c Expected Output

1 0 50 50 Not a quadratic equation




We may have another set of test cases based on input domain.

I1 = {a: a= 0}

I2 = {a: a < 0}

I3 = {a: 1 <=a <= 100 }

giBS 29




0801914408

Here test cases 5 and 8 are redundant test cases. If we choose any value other than nominal,

we may not have redundant test cases. Hence total test cases are 10+4=14 for this problem.

giBS 31




0801914408

Decision Table Based Testing

Decision tables are useful for describing situations in which a number of combinations of

actions are taken under varying sets of conditions. Decision tables have been used torepresent and analyze complex logical relationships since early 1960s. Some of the basic

terms are shown in the following table:

Condition

stub

c1

c2

c3

Entry

True False

True False True False

True False True False True False -

Actiona1

Stub

a2

a

3

a4

X X X

X X X

X X

X X X

There are four portions of a decision table namely, conditions stub, Action stb, Condition

entries & Action entries. When conditions c1, c2, c3, are all true, actions a1, and a2 occur.

When conditions c1 and c2 are true and c3 is false, actions a1 and a3 occur. The decision tables

in which entries are binary are called limited entry decision tables. If conditions are allowed

to have several values, the resulting tables are called extended Entry Decision Tables.

Test Case design

giBS 32




0801914408

To identify test cases with decision tables, we interpret conditions as inputs, and actions as

outputs. Sometimes, conditions end up referring to equivalence classes of inputs, and actions

refer to major functional processing portions of the item being tested. The rules are then

interpreted as test cases. Because the decision table can mechanically be forced to completed,

we have a comprehensive set of test cases.

Example:

Consider a simple program to classify a triangle. Its input is a triple of positive integers and

the data type for input parameters ensures that these will be integers greater than 0 and less

than or equal to 100. The program output may be one of the following words:

[Scalene, Isosceles, Equilateral, Not a triangle]

Design Decision table for it.

Solution:

c1: x, y, z are sides of triangle?

c2: x = y ?

c3 : x = z ?c4 : y = z ?

N Y

- Y N

- Y N Y N

- Y N Y N Y N Y N

a1 : Not a trianglea2 : Scalene

a3: Isoscelesa4 : Equilateral

a5 : Impossible

X

X

X X XX

X X X

giBS 33




0801914408

Advantages of Black Box Testing

• Efficient when used on Larger systems

• As the tester and developer are independent of each other, test is balanced and

unprejudiced

• Tester can be non-technical.• There is no need of having detailed functional knowledge of system to the tester.

• Tests will be done from a end user's point of view. Because end user should accept

the system. (This is reason, sometimes this testing technique is also called as

Acceptance testing)

• Testing helps to identify the vagueness and contradiction in functional specifications.

• Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing

• Test cases are tough and challenging to design, without having clear functional

specifications

• It is difficult to identify tricky inputs, if the test cases are not developed based on

specifications.

• It is difficult to identify all possible inputs in limited testing time. So writing test

cases is slow and difficult

• Chances of having unidentified paths during this testing

• Chances of having repetition of tests that are already done by programmer .

giBS 34




0801914408

Performance Testing

In this internet era, when more and more of business is transacted online, there is a big and

understandable expectation that all applications run as fast as possible. When applications

run fast, a system can fulfill the business requirements quickly and put it in a position toexpand its business and handle future needs as well. A system or a product that is not able to

service business transactions due to its slow performance is a big loss for the product

organization, its customers, and its customers’ customers. Hence, performance is a basicrequirement for any product and is fast becoming a subject of great interest in the testing

community.

Purposedemonstrate that the systems meets performance criteria.compare two systems to find which performs better.measure what parts of the system or workload cause the system to perform

badly.

Factors Governing Performance Testing

A product is expected, to handle multiple transactions in a given period. The capability of the

system or the product in handling multiple transactions is determined by a factor calledthroughput. Throughput represents the number of requests/business transactions processed by

the product in specified time duration. It is important to understand that the throughput varies

according to the load the product is subjected to. The load to the product can be increased by

increased by increasing the number of users or by increasing the number of concurrent

operations of the product.

Throughput represents how many business transactions can be serviced in a given durationfor a given load. It is equally important to find out how much time each of the transactions

took to complete. Hence measuring “response time” becomes an important activity of

performance testing. Response time can be defined as the delay between the point of requestand the first response from the product. In a typical client-server environment, throughput

represents the number of transactions that can be handled by the server and response time

represents the delay between the request and response.

In reality, not all the delay that happens between the request and the response is caused by

the product. In the networking scenario, the network or other products which are sharing thenetwork resources can cause the delays. Hence, it is important to know what delay the product causes and what delay the environment causes. This brings up yet another factor for

performance- latency. Latency is a delay caused by the application, operating system, and by

the environment that are calculated separately.

The next factor that governs the performance testing is tuning. Tuning is a procedure by

which the product performance is enhanced by setting different values to the parameters of

giBS 35




0801914408

the product, operating system, and other components. Tuning improves the product

performance without having to touch the source code of the product. Each product may have

certain parameters or variables that can be set a run time to gain optimum performance. Thedefault values that are assumed by such product parameters may not always give optimum

performance for a particular deployment. Doing performance testing, tuning of parameters is

an important activity that needs to be done before collecting actual numbers.

Yet another factor that needs to be considered for performance testing is performance of competitive products. A very well-improved performance of a product makes no business

sense if that performance does not match up to the competitive products. Hence it is very

important to compare the throughput and response time of the product with those of the

competitive products. This type of performance testing wherein competitive products arecompared is called benchmarking.

One of the most important factors that affect performance testing is the availability of

resources. A right kind of configuration is needed to derive the best results from performancetesting and for deployments. The exercise to find out what resources and configurations are

needed is called capacity planning. The purpose of a capacity planning exercise is to helpcustomers plan for the set of hardware and software resources prior to installation or upgrade

of the product.

Performance Testing Sub-Genres

load

stress

endurance

spike

scalability

Load Testing

This is the simplest form of performance testing. A load test is usually conducted to understand the

behavior of the application under a specific expected load. This load can be the expected concurrent

number of users on the application performing a specific number of transaction within the set duration.

This test will give out the response times of all the important business critical transactions. If the

database, application server, etc are also monitored, then this simple test can itself point towards the

bottleneck in the application software

Stress Testing

This testing is normally used to break the application. Double the number of users are added to the

application and the test is run again until the application breaks down. This kind of test is done to

determine the application's robustness in times of extreme load and helps application administrators to

determine if the application will perform sufficiently if the current load goes well above the expected

load.

giBS 36




0801914408

Endurance Testing (Soak Testing)

This test is usually done to determine if the application can sustain the continuous expected load. During

endurance tests, memory utilization is monitored to detect potential leaks. Also important, but often

overlooked is performance degradation. That is, to ensure that the throughput and/or response times after

some long period of sustained activity are as good or better than at the beginning of the test.

Spike Testing

Spike testing, as the name suggests is done by spiking the number of users and understanding the

behavior of the application whether it will go down or will it be able to handle dramatic changes in load.

Scalability Testing

Scalability testing is an extension of performance testing - it's part of the battery of non-functional tests, is

the testing of a software application for measuring its capability to scale up or scale out in terms of any of

its non-functional capability - be it the user load supported, the number of transactions, the data volume

etc. The purpose of scalability testing is to identify major workloads and mitigate bottlenecks that can

impede the scalability of the application.

Methodology for Performance Testing

Performance testing is complex and expensive due to large resource requirements and the time ittakes. Hence, it requires careful planning and a robust methodology. Performance testing is

ambiguous because of the different people who are performing the various roles having differentexpectations. Additionally, a good number of defects that get uncovered during performance

testing may require design and architecture change. Finally, a fix performance defect may evencause some functionality to stop working, thereby requiring more effort during regression. A

methodology for performance testing involves the following steps:

1. Collecting requirements

2. Writing test cases

3. Automating performance test cases4. Executing performance test cases

5. Analyzing performance test results

6. Performance tuning

7. Performance benchmarking8. Recommending right configuration for the customers

Automation Testing

giBS 37




0801914408

Test automation is the use of software to control the execution of tests, the comparison of

actual outcomes to predicted outcomes, the setting up of test preconditions, and other test

control and test reporting functions. Commonly, test automation involves automating a

manual process already in place that uses a formalized testing process.

Although manual tests may find many defects in a software application, it is a laborious andtime consuming process. In addition it may not be effective in finding certain classes of

defects. Test automation is a process of writing a computer program to do testing that would

otherwise need to be done manually. Once tests have been automated, they can be run

quickly. This is often the most cost effective method for software products that have a long

maintenance life, because even minor patches over the lifetime of the application can cause

features to break which were working at an earlier point in time.

There are two general approaches to test automation:

•

Code-driven testing.• Graphical user interface

Code-driven testing

A growing trend in software development is the use of testing frameworks such as the xUnit

frameworks that allow the execution of unit tests to determine whether various sections of the

code are acting as expected under various circumstances. Test cases describe tests that needto be run on the program to verify that the program runs as expected.

Code driven test automation is a key feature of Agile software development, where it is

known as Test-driven development (TDD). Unit tests are written to define the functionalitybefore the code is written. Only when all tests pass is the code considered complete.

Proponents argue that it produces software that is both more reliable and less costly than

code that is tested by manual exploration. It is considered more reliable because the codecoverage is better, and because it is run constantly during development rather than once at the

end of a waterfall development cycle. Because the developer discovers defects immediately

upon making a change, when it is least expensive to fix. Also, since the only code that is

written is what is required to make the tests pass, the tendency to write too much code isremoved. Finally, rework is safer. When code is made faster or is cleaned-up, all of the tests

that passed must continue to pass or the reworked code is not working as it should.

Graphical User Interface (GUI) testing

Many test automation tools provide record and playback features that allow users to

interactively record user actions and replay it back any number of times, comparing actual

results to those expected. The advantage of this approach is that it requires little or no

giBS 38

http://en.wikipedia.org/wiki/Software

http://en.wikipedia.org/wiki/Software_testing

http://en.wikipedia.org/wiki/Graphical_user_interface

http://en.wikipedia.org/wiki/XUnit

http://en.wikipedia.org/wiki/Unit_test

http://en.wikipedia.org/wiki/Code

http://en.wikipedia.org/wiki/Test_case

http://en.wikipedia.org/wiki/Agile_software_development

http://en.wikipedia.org/wiki/Test-driven_development

http://en.wikipedia.org/wiki/Waterfall_model

http://en.wikipedia.org/wiki/Rework

http://en.wikipedia.org/wiki/Software

http://en.wikipedia.org/wiki/Software_testing

http://en.wikipedia.org/wiki/Graphical_user_interface

http://en.wikipedia.org/wiki/XUnit

http://en.wikipedia.org/wiki/Unit_test

http://en.wikipedia.org/wiki/Code

http://en.wikipedia.org/wiki/Test_case

http://en.wikipedia.org/wiki/Agile_software_development

http://en.wikipedia.org/wiki/Test-driven_development

http://en.wikipedia.org/wiki/Waterfall_model

http://en.wikipedia.org/wiki/Rework




0801914408

software development. This approach can be applied to any application that has a graphical

user interface. However, reliance on these features poses major reliability and maintainability

problems. Relabelling a button or moving it to another part of the window may require the

test to be re-recorded. Record and playback also often adds irrelevant activities or incorrectly

records some activities.

A variation on this type of tool is for testing of web sites. Here, the "interface" is the web

page. This type of tool also requires little or no software development. However, such a

framework utilizes entirely different techniques because it is reading html instead of

observing window events.

Another variation is scriptless test automation that does not use record and playback, but

instead builds a model of the application under test and then enables the tester to create test

cases by simply editing in test parameters and conditions. This requires no scripting skills,

but has all the power and flexibility of a scripted approach. Test-case maintenance is easy, as

there is no code to maintain and as the application under test changes the software objectscan simply be re-learned or added. It can be applied to any GUI-based software application.

What to test

Testing tools can help automate tasks such as product installation, test data creation, GUI

interaction, problem detection (consider parsing or polling agents equipped with oracles),

defect logging, etc., without necessarily automating tests in an end-to-end fashion.

One must keep following points when thinking of test automation:

• Platform and OS independence• Data driven capability (Input Data, Output Data, Meta Data)

• Customizable Reporting (DB Access, crystal reports)

• Email Notifications (Automated notification on failure or threshold levels)

• Easy debugging and logging

• Version control friendly – minimum or zero binary files

• Extensible & Customizable (Open APIs to be able to integrate with other tools)

• Common Driver (Ant or Maven)

• Headless execution for unattended runs (For integration with build process or batch runs)

• Support distributed execution environment (distributed test bed)

• Distributed application support (distributed SUT)

Framework approach in automation

A framework is an integrated system that sets the rules of Automation of a specific product.

This system integrates the function libraries, test data sources, object details and various

giBS 39

http://en.wikipedia.org/wiki/Computing_platform

http://en.wikipedia.org/wiki/Operating_system


http://en.wikipedia.org/wiki/Crystal_reports


http://en.wikipedia.org/wiki/Version_control

http://en.wikipedia.org/wiki/Apache_Ant

http://en.wikipedia.org/wiki/Apache_Maven


http://en.wikipedia.org/wiki/Testbed

http://en.wikipedia.org/wiki/System_Under_Test

http://en.wikipedia.org/wiki/Computing_platform



http://en.wikipedia.org/wiki/Version_control

http://en.wikipedia.org/wiki/Apache_Ant


http://en.wikipedia.org/wiki/Testbed

http://en.wikipedia.org/wiki/System_Under_Test




0801914408

Robust: Test produces same result now and forever. Tests are not affected by changes in the

external environment.

Sufficient: Tests verify all the requirements of the software being tested.

Necessary: Everything in each test contributes to the specification of desired behavior.

Clear: Every statement is easy to understand.

Efficient: Tests run in a reasonable amount of time.

Specific: Each test failure points to a specific piece of broken functionality; unit test failures

provide "defect triangulation".

Independent: Each test can be run by itself or in a suite with an arbitrary set of other tests in

any order.

Maintainable: Tests should be easy to understand and modify and extend.

Traceable: To and from the code it tests and to and from the requirements

When Not to Automate

It is better to run the test cases manually when the life of the solution is short, when very few

releases are planned ,or when the test cases are not repetitive in nature. This is because testautomation come at a cost- tool cost , cost of skilled resources, and, most important, the time

required to automate. If the costs cannot be recovered in the which automated script are run,it is best not to automate

Benefits of Automated Testing

Reliable: Tests perform precisely the same operations each time they are run, thereby

eliminating human error

Repeatable: You can test how the software reacts under repeated execution of the sameoperations.

Programmable: You can program sophisticated tests that bring out hidden information from

the application.

Comprehensive: You can build a suite of tests that covers every feature in your application.

giBS 41




0801914408

CONCLUSION

We discuss various testing in our report and each testing has been classified further. These

testing has their own domain and worked according to that . Some testing has their areas in

which they perform. On the basis of their functionality each have their advantages anddisadvantages

giBS 43




0801914408

BIBLIOGRAPHY

”Software testing” ,by K. k. Aggarwal & Yogesh Singh

”Software testing”, by Gopalaswamy

url:-

www.buzzle.com/editorials/4-10-2005-68350.asp

www.codeproject.com/KB/.../Black _ Box _ Testing.aspx

en.wikipedia.org/wiki/Test _ automation

www.infosys.com/IT.../ test-automation-post-deployment.pdf

gp software testing(2)

Documents