testing process
DESCRIPTION
Based on V-model (Extension of waterfall model)describes testing process,strategyUnit test,Integration test,System Test,User acceptance test.TRANSCRIPT
<Insert Picture Here>
SOFTWARETESTING PROCESS
Byungwook Cho C. ([email protected])Sr Consultant 2008.08.08
ICE BREAK
AGENDA
• Test model• Test process• Static testing
Test model
Test oriented development process model
• V-Model• Enhanced traditional
waterfall model
• Verification & Validation
Verification Validation
To verify the artifacts that has been produced in each development cycle.
Valid & estimate the system
Review & Inspectionwith artifacts from each development step
Testing with system
Static testing Dynamic testing
< V-Model >
< Verification & Validation >
Test LevelLevel Description Testing Type Responsibility
Unit Test Verify software component White box test Lead by Dev Team Use Code coverage
Integration Test Verify integration between component. Verify software flow ,interface & interaction
White box test Lead by Dev Team Use Mock Object (Test Stub ,Driver)
System Test Test system over production environment to verify system itself and production environmentInclude Functional & Non Functional Test (Availibility, Stablity, Extendability, Performance etc)
Black box test Lead by QA Team(Specialized for system test)
Acceptance Test Verify customer’s requirement (End User)Verify customer’s Legal issue (Legal)Verify customer’s maintanance issue (SM)
Black box test Lead by QA Team or Customer
Alpha TestBeta Test
※ Integration strategiesTop down Bottom up Big-bang Backbone
Desc Integrate from top(use test stub)
Integrate from bottom(use test driver)
Integrate all in time Most important & high risk component first
Adv easily find architectural defect
Can test implementation logic
Short time
Ex. BPM SOA svc Small project
Test Cycle
• Each test is consists of test cycle• Test cycle
• Pre test - check up testability• Main test • Conformation test - check up the defect found in
previous “Main test”• Regression test - check up the impact from change
Pre Test Main Test
ConformationTest
RegressionTest
TEST CYCLE
Test Process
Master Test Plan
Organization
Policy
Strategy
Mgmt sub tests
Test ProcessTest plan
Define objective of Test and scope
Define Test strategy
Test estimation
Set up test organizationTest analysis and design
Evaluation test object & basis
Designing and prioritizing test case
Identify test
Identify & prepare test data
Designing test env & identify required infra & tools
Make test procedure
Test Implementation & Execution
Implement TC
Implement test script
Set up test environment
Pre Test
Run Test
Logging Test Result
Test Closure activity
Store test-ware into repository
Analyzing lesson learned for future test maturity
Evaluating exit criteria and reporting
Write a test summary report for stakeholders.
Assessing if more test are needed
Test control
Test mgmt & control
Test monitoring & reporting
Test plan
Define objective of Test and scope
Define Test strategy
Test estimation
Set up test organizationTest analysis and design
Evaluation test object & basis
Designing and prioritizing test case
Identify test
Identify & prepare test data
Designing test env & identify required infra & tools
Make test procedure
Test Implementation & Execution
Implement TC
Implement test script
Set up test environment
Pre Test
Run Test
Logging Test Result
Test Closure activity
Store test-ware into repository
Analyzing lesson learned for future test maturity
Evaluating exit criteria and reporting
Write a test summary report for stakeholders.
Assessing if more test are needed
Test control
Test mgmt & control
Test monitoring & reporting
Test plan
Define objective of Test and scope
Define Test strategy
Test estimation
Set up test organizationTest analysis and design
Evaluation test object & basis
Designing and prioritizing test case
Identify test
Identify & prepare test data
Designing test env & identify required infra & tools
Make test procedure
Test Implementation & Execution
Implement TC
Implement test script
Set up test environment
Pre Test
Run Test
Logging Test Result
Test Closure activity
Store test-ware into repository
Analyzing lesson learned for future test maturity
Evaluating exit criteria and reporting
Write a test summary report for stakeholders.
Assessing if more test are needed
Test control
Test mgmt & control
Test monitoring & reporting
Test plan
Define objective of Test and scope
Define Test strategy
Test estimation
Set up test organizationTest analysis and design
Evaluation test object & basis
Designing and prioritizing test case
Identify test
Identify & prepare test data
Designing test env & identify required infra & tools
Make test procedure
Test Implementation & Execution
Implement TC
Implement test script
Set up test environment
Pre Test
Run Test
Logging Test Result
Test Closure activity
Store test-ware into repository
Analyzing lesson learned for future test maturity
Evaluating exit criteria and reporting
Write a test summary report for stakeholders.
Assessing if more test are needed
Test control
Test mgmt & control
Test monitoring & reporting
Acceptance Test Plan
System Test Plan
Integration Test Plan
Unit Test Plan
Test Plan
• Test planning & control• Define activity of each testing level• Define milestone ,resource ,schedule.• Make a plan based on analyzed risk factor• Define test strategy based on risk• Define approach & techniques for testing
(testing techniques, coverage, test item, test ware)
• Must include time & resource for preparing testware
• Define completion condition
Define test scope
objective
Risk anal
Set up strategy
Estimateresource & time
Planning
PRE TEST
TEST
Monitoring
Report
Test Plan
• Risk analysis• Risk = Likelihood * impact• Risk identification
• Recommend < 36 identification.• Likelihood : Complexity, Implementation technical Level,
Size,Developer skill• Impact : Biz impact
Likelihood Impact
factor factor factor factor
Risk item
Risk item
Likelihood
Impact
STA
(Severe Test Area)
SSTA
(Strong Test Area)
FTA(Fundamental
Test Area)
ITA
(Intensive Test
Area)
Erik van neneendaal, Risk Based Testing, STAREAST,2006
Test Plan
• Risk based test strategy
• Other strategies• Analytical approaches• Model based approaches• Methodical approaches• Dynamic and heuristic approaches
Likelihood
Impact
STA
(Severe Test Area)
SSTA
(Strong Test Area)
FTA(Fundamental
Test Area)
ITA
(Intensive Test
Area)
Likelihood
Impact
STA
(Severe Test Area)
SSTA
(Strong Test Area)
FTA(Fundamental
Test Area)
ITA
(Intensive Test
Area)
( Low Level Test) ( High Level Test)
- Unit Test
- Integration Test
- System Test
- Acceptance Test
Test Analysis and Design
• Test analysis• Review - test basis, testability• Identify test requirement & test data• Identify test infra & tools
• Test design• Identify test condition• Test case specification• Define test procedure
Test Analysis and Design
• Design technique• Specification based
• Equivalence partitioning, Boundary value analysis• Pairwise, Decision table testing• State transition testing, Usecase testing
• Structure based• Control Flow Test• Basic Path Test – Based on Cyclomatic complexity (Edge –
Node + 2P) P = number of connected component• Elementary comparision test – Based on MC/DC Coverage* Coverage : Statement, Condition, Decision, Condition Decision
Coverage, Modified Condition/Descision Coverage(MC/DC), Multiple Condition Coverage
• Experience based• Exploratory Testing – Adhoc testing, based on test charter,
define timezone• Classification Tree method• CheckList
Test Execution
• Execution• Define serverity & priority• Develop TC,Test data,Test stub &
driver,Test script,Test suite
• Reporting• Logging expected result, real result• Current status• Time & resource usage
• Defect tracking• Use Bug(Issue) tracking system
( Mantis,Bugzilla,JIRA,Trac)• Define tracking workflow
TESTDRIVER
TESTSTUB
TARGET SYSTEM
TEST CASE
TESTINGTOOL
TEST DATA MODULE
MODULE MODULE MODULE
TEST SCRIPT
TEST SUITE1
TEST SUITE2
TEST
SIMULATOR
Test Execution
Test Report Defects
Issue Tracking System
Progress Report
Test logs
Evaluating exit criteria and reporting
• Test exit criteria• Coverage, Duration, number of unresolved defects
• Test result report• Progress report• Release advice• Final report per Level
• Matrix• Defects per KLOC, Days test effort per requirement,
Unresolved defect impact analysis, Defects / Hour ,Cumulative defects / Day ,Test coverage / Day , Risk vs number of TC
Test closure activities
• Test closure activities• Evaluating
• Evaluate test target• Evaluate test process for maturing next text
• Final reporting• Store test-ware
Static Testing
Static testing techniques
• Performed in Verification phase• Test with artifacts not a runtime software• Test by human
Manual Review
• Inspection• Objective : To find defects• Inspect artifacts• Lead by moderator
• Walkthrough• Objective : Knowledge sharing• Knowledge transfer by author
• Code review• Objective : Find defects and share knowledge• Lead by Programming Leader
• Pair programming• XP • Lead by mentor or coworker• Share just “ONE” keyboard
Formal action
Informal action
Static Analysis
• Static Analysis• Test code or diagram with “static analysis tool”• Analysis
• Can find syntax error or well known bugs• Validate naming rules• Can find dead code or missing parameter• Can analysis code complexity & dependency
• Quality?• Not so smart• It need a rule and need a maturing
• Easy to appeal your manager• Tools
• Open source – PMD, Find Bugs• Commercial – Klockworks, Jtest etc.