Lauri Väätäinen

ICSM 2013, 26.9.2013

Tanja Toroi, Anu Raninen and Lauri Väätäinen :

Identifying Process Improvement Targets in

Test Processes: A Case Study

26.9.2013 2

Agenda

1. Problems in the test processes

2. Motivation to light-weight process improvement

3. Case study and A Short Introduction to LAPPI technique

4. Results of the case study with conclusions

26.9.2013 3

Problems in the Test Processes (earlier results)

•Test planning, test cases and unit testing are often vague and neglected

•Lack of testing know-how and insufficient testing tools often hinder testing development

•Lack of testing expertise prevents organizations from using software testing methods

•Verification and validation are first processes to neglect when time runs out

26.9.2013 4

Motivation to Light-weight Process Improvement

•Software testing is considered important factor in quality assurance

•Continuous efforts on improving a testing process is important

•E.g. CMMI and ISO/IEC 15504 standards are often considered too heavy-weigh in small software companies

Light-weight process modeling is one possible solution

26.9.2013 5

Case study

•How to identify software test process problems in a cost-efficient way?

•What are the major problem areas in software testing?

•LAPPI technique was applied to model the test processes and identify process problems

Research article available:

A. Raninen, J. J. Ahonen, H.-M. Sihvonen, P.Savolainen and

S. Beechman: LAPPI: A light-weight technique to practical process modeling and improvement target identification,

J. Softw. Evol. And Proc. 2012, DOI: 10. 1002/smr.1571

26.9.2013 6

Backbone of the LAPPI technique (1/3)

Workshop 1: Roles and information flow modeling

PROGRAMMER

TEST DESIGNER

TESTER

Complete features of the next release

Test cases to implement into a test automation

Test report

Don’t understand the test case

A problem in the information flow

Role Information flow

26.9.2013 7

Backbone of the LAPPI technique (2/3)

Workshop 2:

Process modeling

26.9.2013 8

Backbone of the LAPPI technique (3/3)

Workshop 2:

Process modeling

26.9.2013 9

Features of case organizations

A B C

Market Farming Metal Industry Telecom

Size of organization

Small Large Medium

Emplyees in sw. Developmet / system testing

9 / 4-6 24 / 2 30 / 1

Country Finland Multinational Finland

Age of company 24 43 13

•Case study was conducted in three different case organizations

26.9.2013 10

Results of the case study (1/2)

Problem A B C

1. No exit criteria defined for testing (*) X X X

2. Test automation is not integrated with (*) manual test process

X X

3. Level of unit testing conducted by programmers vary a lot

X X X

4. Lack of proper test case specification X X

5. All repercussions of the code changes are not known

X X X

6. Documentation processes are vague X X

(*) less reported problem

Results of the case study (2/2)

26.9.2013 11

Improvement suggestions A B C

1. Define minimum level of unit testing X X X

2. Pay attention to test case specification. X X X

3. Provide training in software testing. X X X

4. Streamline the usage of defect tracking system. X X X

5. Define exit criteria for testing. X X X

6. Consider software testing already in the requirement specification phase.

X X

7. Integrate test automation with the manual test process.

X X

8. Improve and monitor documentation. X X

9. Provide more test resources. X X

26.9.2013 12

Conclusions

•Problems of testing are similar across different kinds of companies

•Process modeling can be used to initiate process improvement

•LAPPI technique is a simple and cost-effective tool to identify process problems

Thank you for your attention!

Lauri.Vaatainen@uef.fi

www.uef.fi

Identifying Process Improvement Targets in Test Processes: A Case Study