software testing basic concepts

42
© Copyright 2006 FPT Software 1 © FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Software Testing Software Testing Basic Concepts Basic Concepts Instructor: Dang Tu Linh Instructor: Dang Tu Linh

Upload: hung-hoang

Post on 06-May-2015

12.629 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Software testing basic concepts

© Copyright 2006 FPT Software 1© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Software Testing Software Testing Basic ConceptsBasic Concepts

Instructor: Dang Tu LinhInstructor: Dang Tu Linh

Page 2: Software testing basic concepts

© Copyright 2006 FPT Software 2© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

AgendaAgenda

Duration: 1.5 Hours Purpose: Understand basic Test concepts Audience: All Members Content:

1. Introduction2. Defect concepts3. Test process4. Test plan & test design5. Testing strategy

Page 3: Software testing basic concepts

© Copyright 2006 FPT Software 3© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

IntroductionIntroduction

Objectives:To arrive at a common understanding of the scope of testing

activityTo familiarize the audience with the terminology used in the

testing workUnderstand the required test skills of developers

Page 4: Software testing basic concepts

© Copyright 2006 FPT Software 4© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Typical Project Blues…Typical Project Blues…

Page 5: Software testing basic concepts

© Copyright 2006 FPT Software 5© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Testing definitionTesting definition

Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements - (IEEE 83a)

Process: Sequence of steps performed for a given purpose. (IEEE)

Software Process A set of activities, methods, practices, and

transformations that people use to develop and maintain software and associated products.(SEI-CMM)

Page 6: Software testing basic concepts

© Copyright 2006 FPT Software 6© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Testing ObjectivesTesting Objectives

Primary: Execute a program with the intent of finding errors to Determine whether system meets specifications Determine whether system meets user’s needs

Secondary Continuously improve the testing process

Page 7: Software testing basic concepts

© Copyright 2006 FPT Software 7© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

MisunderstandingsMisunderstandingsabout testingabout testing

Testing is debugging

Testing is not the job of a programmer

If programmers were more careful testing would be unnecessary

Testing never ends

Testing activities start only after the coding is complete

Testing is not a creative task

Page 8: Software testing basic concepts

© Copyright 2006 FPT Software 8© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Theory & RealityTheory & Reality

Theory Need sufficient time Baselined & freezed requirements Automation testing Have trained testers: test process, business flow,…

Reality Time Pressures to keep delivery date Continuously updated requirements Don’t know when to stop: Update & update

Requirement continuously (CRs) Tester updates TC and re-tests

Manual testing Lack of dedicated software test environment: share,

steal Inexperienced testers without appropriate training

Page 9: Software testing basic concepts

© Copyright 2006 FPT Software 9© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Defect conceptsDefect concepts

Objectives:Understand the definition of defect Understand the cost of a bug to software

development Understand the common types of defects and

their perceived severity

What is defect?What is defect?

A defect is any error found by testing and reviewing activities (All errors found by internal reviewer, external reviewer and customer).

Page 10: Software testing basic concepts

© Copyright 2006 FPT Software 10© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Where defects come from?Where defects come from?

Products SRS, DD, Source code, Test cases, etc.

Quality Control Review, Test, Audit, Inspection

Processes Requirements, Design, Coding, Test, etc.

Other sources: Change requests, misunderstanding, carelessness,

planning, etc.

Page 11: Software testing basic concepts

© Copyright 2006 FPT Software 11© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Cost of defectsCost of defects

Page 12: Software testing basic concepts

© Copyright 2006 FPT Software 12© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Defect still exists in the product, change status to ERROR

FSoft Defect lifecycleFSoft Defect lifecycle

Defect classification: Open defects Closed defects Leakage

Log defects Analyze defects Fix defects Re-test

Step 1 Step 2 Step 3 Step 4

What: To log defect into DMS with clear information: project, module, stage, QC activity, severity, type, due date, tester, created date ...Who: TesterDefect’s status: ERROR

What: -To analyze the causes of the defect, find-out solution and assign to person who is responsible for correcting it.- To accept the defect if it’s acceptable (by concession of authority or customer).Who: Project LeaderDefect’s status: ASSIGNED / ACCEPTED

What: To correct the defect and perform unit testing to make sure the defect has been removed.Who: DeveloperDefect’s status: CORRECTED

What: To retest the function(s) containing the defect and make sure that the defect has been corrected completelyWho: TesterDefect’s status: CLOSED

Page 13: Software testing basic concepts

© Copyright 2006 FPT Software 13© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Defect status:• Error• Assigned• Corrected• Cancelled• Accepted: by PM/PTL/Customer• Closed: by tester/PM/PTL

Leakage Defects:• Acceptance test• After Release test• After Release review

Fsoft Defect classificationFsoft Defect classification

Page 14: Software testing basic concepts

© Copyright 2006 FPT Software 14© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Fsoft Defect classificationFsoft Defect classification

Defect type• Functionality

o Requirement misunderstandingo Feature missingo Coding logico Business logic

• User Interface• Performance• Design issue• Coding standard

Severity: How bad the problem is (some time disagreement btw. Dev & Tester)

Fatal: the system is broken Serious: can not work around Medium: impact the function Cosmetic: not impact the function

Page 15: Software testing basic concepts

© Copyright 2006 FPT Software 15© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Fsoft Defect classificationFsoft Defect classification

Defect origin Requirement Design Coding Test CM Document control

Priority Immediately High priority Normal priority Low priority

Work product Software module Software Package ADD DDD Test Cases

(UT/IT/ST)

Quality control activity Code review Unit test Integration test System test Document review Inspection Audit

Page 16: Software testing basic concepts

© Copyright 2006 FPT Software 16© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test ProcessTest Process

Objectives:

Understand the steps followed for a test process

Understand test planning process FSoft Test Process

Page 17: Software testing basic concepts

© Copyright 2006 FPT Software 17© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Tester CompetencyTester Competency

Soft skills (non-technical attributes) Discipline and Perseverance

• Testing is repetitive and requests a lot of manual effort• Have the ability to withstand pressures and workload • Say ‘no’ to managers when quality is insufficient

Reading skills: study many docs of specs, design Communication & Interpreter skills

• Both Verbal and Written communication • Diplomatic skill• Be able to communicate with technical and non-

technical people, engineers, managers, customers Negative thinking: foresee things that can go wrong,

evaluate risks Attitude: ‘test to break’ attitude Time management and Effort prioritization

Page 18: Software testing basic concepts

© Copyright 2006 FPT Software 18© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Tester CompetencyTester Competency

Technical skills Familiarity with software development

architectures, processes Familiarity with testing methodologies Very clearly articulate specs, designs, business

rules, inspection reports, configurations, code changes, TP, TC, bug reports, user manuals,…

Know how and where to look for bugs Know how to report the defects effectively for

quickly reproduce

Page 19: Software testing basic concepts

© Copyright 2006 FPT Software 19© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

The Test ProcessThe Test Process

Data Analysis

- Analyze root Causes of defects- Identify actions

Test Execution

Perform testing

Bug Reporting

- Bug Fixing- TC Updating

Bug Tracking(Re-test)

Test Report

Test Preparation

SRS study

Create Test Design

Review & Approval

- Develop test cases, test scripts- Prepare test data

Test Planning

Create Test Plan

(byTest leader)

Definition Solution Construction Transition Termination

Initiation Project life cycle

Test process

Page 20: Software testing basic concepts

© Copyright 2006 FPT Software 20© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Input/Output for testingInput/Output for testing

Input:• Customer requirements and Acceptance criteria• Change requests• Software Requirement Specification (SRS)• Design documents (ADD, DDD)• Programs (Modules)

Output:• Test documents: Test plan, Test cases and

procedures, Test script, Test data• Defect list• Test report (Pass/Fail of each TC, Summary

report)

Page 21: Software testing basic concepts

© Copyright 2006 FPT Software 21© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

FSoft Test resourcesFSoft Test resources

Guideline: Test process & Test guideline Templates for test documents

Test Plan Test case specification Test report Defect analysis report

FSOFT tools Defect tracking tool: DMS Test Effort tracking tool: Timesheet Test schedule: FSoft Insight Test automation tools

– Rational Robot (Functional & Performance test)– OpenSTA (Open source)– Witir (Open source)

FATA: Test Council

Page 22: Software testing basic concepts

© Copyright 2006 FPT Software 22© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test strategyTest strategy

Objectives Understand Unit Testing concept Understand Integration Testing concept Understand System Testing concept Understand Acceptance Testing concept Overview of Test types

Page 23: Software testing basic concepts

© Copyright 2006 FPT Software 23© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

V-ModelV-Model

Page 24: Software testing basic concepts

© Copyright 2006 FPT Software 24© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Testing stagesTesting stages

There are 4 stages of Testing: Unit Test Integration Test System Test Acceptance Test

Page 25: Software testing basic concepts

© Copyright 2006 FPT Software 25© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Unit TestUnit Test

Purpose: to verify that the component/module functions properly

Check: – internal data structures – Logic– boundary conditions for input/output data

Normally: White box oriented Doer: Development team Metrics: Test coverage:

– Statement– Branch– Path– Condition

Page 26: Software testing basic concepts

© Copyright 2006 FPT Software 26© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Integration TestIntegration Test

Purpose: • To ensure that code is implemented and

designed properly • to take unit tested modules and build a

program structure that has been dictated by design IT only after UT

Combining the individual components to uncover errors associated with interfacing

Normally: Black box oriented Doer: independent Test team

Page 27: Software testing basic concepts

© Copyright 2006 FPT Software 27© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

System TestSystem Test

Purpose: To ensure that the system does what the requirement specifies

Types– Function Testing– Performance Testing– Installation Testing

Normally: Black box oriented Doer: independent Test team

Page 28: Software testing basic concepts

© Copyright 2006 FPT Software 28© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Acceptance TestAcceptance Test

Customer’s way to verify that what was wanted is what is built

Uncovers more than requirements discrepancies Allows the customers to determine what they

really want, whether specified in the document or not.

New problems may arise Customers may decide that the problem as

changed and a different solution is needed Normally: Black box oriented Doer: Customer/independent Test team

Page 29: Software testing basic concepts

© Copyright 2006 FPT Software 29© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Testing typesTesting types

Functional test Function testing User Interface testing Data and database integrity testing Business cycle testing

User Interface: Layout Performance test

Performance profiling Load testing Stress testing Volume testing

Access control & Security test Regression test

Page 30: Software testing basic concepts

© Copyright 2006 FPT Software 30© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test techniques/methodsTest techniques/methods

White Box Testing It guarantees that all independent paths within a

module have been exercised at least once Exercise all logical decisions on their true an false

sides Execute all loops their boundaries and within their

operational bounds Exercise internal data structures to assure their

validity

Black Box Testing Focuses on functional requirement of the software Derive sets of input conditions that will fully exercise

all functional requirements for a program Not an alternative to white box

Page 31: Software testing basic concepts

© Copyright 2006 FPT Software 31© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test Plan, Test Design Test Plan, Test Design & Test Cases& Test Cases

Objectives

Understand the principles of test plan & test design Understand the concepts for test case design Understand the structure of test cases

Page 32: Software testing basic concepts

© Copyright 2006 FPT Software 32© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test Planning: ObjectivesTest Planning: Objectives

Test planning is to: Define Test strategy:

– What to test: list of requirements to test– How to test (Test techniques)– Test types– Test tool, ….

Define the responsibilities Decide test metrics & manage through metrics “When to Stop Testing?”

– Not good Unit Test– Low quality of code,…– Meet requested Test coverage– All Test cases are tested and passed

Page 33: Software testing basic concepts

© Copyright 2006 FPT Software 33© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test PlanTest Plan

What to do Who does it: Test team, Development team How to do it: Techniques When to do it

Page 34: Software testing basic concepts

© Copyright 2006 FPT Software 34© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test PlanTest Plan

Test Plan to define: Scope of test: stages and types of test Risks may affect testing: quantify the loss of the risk Training needs for testers Requirements to test: what will be tested Testing strategy: how will the test be performed Completion criteria (Pass/Fail) Criteria to stop testing Tools to be used for testing Testing resource and environment (Responsibilities of

testers) Test Milestone: schedule of testing activities Test deliverables: outputs of testing

Page 35: Software testing basic concepts

© Copyright 2006 FPT Software 35© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

When to Stop Testing?When to Stop Testing?

Not good UT: Defects of UT do not meet Defect plan Low quality of code: too many defects (over the

number identified in planning phase/kick-off meeting) Does not meet requested Test coverage (by

customer/PM).It must be identified at planning phase.

Page 36: Software testing basic concepts

© Copyright 2006 FPT Software 36© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test design structureTest design structure

Specifications Basic flow Alternative flow

Test Design All test cases should be traceable to requirements Design framework and rules for test cases (large/average

items) Plan number of test cases for normal, abnormal & boundary

data Assign testers for module/function/large items

Page 37: Software testing basic concepts

© Copyright 2006 FPT Software 37© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test DesignTest Design

Structure/Framework for Test cases (TC):• Modules or functions• Large items • GUI• Functions• Performance

Number of TC: Total For each module For each type: GUI/Function For normal/abnormal/boundary cases

Page 38: Software testing basic concepts

© Copyright 2006 FPT Software 38© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Test CaseTest Case

Test environment Test condition A test case includes:

• Action (sample: click on Add button, on a link, input birth day…)

• Input data (valid/invalid/boundary data)• Expected result (message, …)

Good test case: Structure of TC is clear and reasonable. Follow requirement closely. Cover all of cases that can occur. Description and test conditions is brief and easy to

understand Procedure and expected result is a step by step

process

Page 39: Software testing basic concepts

© Copyright 2006 FPT Software 39© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Testing MeasurementTesting Measurement

Objectives Understand the need for metrics and measurement

in Testing Understand the minimal metrics required for the

measurement of the testing activity

Page 40: Software testing basic concepts

© Copyright 2006 FPT Software 40© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Examples of testing measurement?Examples of testing measurement?

Product: We’ve tested 80% of the lines of code Plan: We’ve run 80% of the test cases Results: We’ve discovered 593 defects Effort: We’ve worked 80 hours a week on this for 4

months. We’ve run 7,243 tests Quality of Testing: Beta testers have found 30 defects

that we missed. Our regression tests seems ineffective History across projects: At this milestone on previous

projects, we had fewer than 12% of the defects found still open. We should be at that percentage on this product too

Page 41: Software testing basic concepts

© Copyright 2006 FPT Software 41© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

How do we measure Testing in How do we measure Testing in FSOFTFSOFT??

Measurements: Defects in DMS Test effort in Timesheet Defect rate in Fsoft Insight: Weighted defects/ project size (in

UCP) Test coverage: number of executed test cases/ Total number

of test cases Test successful coverage: number of test cases executed

successfully/total number of test cases Defect removal efficiency

Metrics to assess tester performance: Test effectiveness

• Weighted defects/ Test execution effort Leakage

• Weighted defects found after release/ project size

Page 42: Software testing basic concepts

© Copyright 2006 FPT Software 42© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2

Q & AQ & A