volodymyr prymakov and vlada benyukh detailed manual estimation approach for pre-sale phase

Detailed Manual Estimation Approach for Pre-sale phase

Vlada Benyukh, Volodymyr Prymakov

Speaker info

Vlada Benyukh, UkraineQA Consultant/QA Lead at Ciklum, TCoE

• 7 years in QA

• Manual QA Engineer, QA Lead

• Certified: SAFe, ICAgile, ISTQB

/in/vlada-benyukh

Speaker info

/in/vladimirprimakov/

Volodymyr Prymakov, UkraineSenior QA Manager at Ciklum, TCoE

Head of Performance QA Unit at TCoE

•14 years in QA

•45 projects experience

•Certified: ISTQB Advanced Test Analyst & Manager,

ICAgile CP, SAFE

Agenda

1. What is an Estimate? Why it is important?

2. Which basis and factors should be considered in estimation process?

3. What main and additional activities should be included in QA Estimation

approach?

4. How Regression testing should be calculated?

5. How QA Estimation can be adjusted by risks multipliers?

6. Demo of real-life example

Why we need to estimate?

How to estimate these tasks?

Assumptions

We assume that:

1. Project is planned to work based on Scrum

2. Development team size: up to 30 people

3. QA Engineer Competency is Middle Level

4. Non-functional requirements are estimated separately

Test Basis

Such test basis should be considered as below:

1. Functional requirements in scope (WBS) are defined

2. Test environments and their priorities are determined while pre-sale phase

3. Project duration (number of sprints) is considered before estimation

process

What factors?

1. Domain specific knowledge and technologies used

2. Integration-related problems/risks (within internal and external systems)

3. Complexity of the application (e.g. calculations verifications)

4. Compatibility testing (e.g. file compatibility, plugin computability etc)

What main activities?

1. Detailed Requirements Review*

2. Test Design

3. Test Execution

4. Bug investigation and submission. Bug Retest

5. Environments/Platforms Multipliers

How to estimate time?

Time calculation for main test activities = (test execution + bugs submission and retest) *environments/platforms multiplier+ test design+ detailed requirements review*

How to calculate env multiplier?

Environments/platforms factors to select:

1. Identify customer region and target audience where the application will be

used

2. Select top used environments/platforms based on the customer’s

region/audience

Example to calculate env multiplier?

Web-browser e.g. Test Type Coefficient

The highest priority web-browser Chrome 57Full Test (Positive, negative flows) 1

High priority web-browser IE 11, FF54 Only positive flows tests 0,5

Medium priority web-browser Edge14 Explorartory testing 0.25

Low priority web-browser Chrome 56Smoke test (at least 1 positive flow test) 0.1

EXAMPLE of multiplier calculation: 1 + 0.5 + 0.5 + 0.25 + 0.1 = 2.35

Example of estimation basis

What typical preparation activities?

1. Detailed evaluation of the product/project. Full Master Test Plan creation

2. Learning and getting acquainted with the product's documentation and/or

functionality. Preliminary requirements review and feedback

3. Checklist/Suite creation (with empty test cases)

4. Test Environment and Infrastructure Preparation & Access

Example of preparation activities?

What typical repetitive activities?

1. Sprint Grooming & Planning, Retrospective

2. Daily Communication Meetings

3. Demo Meetings

4. Global Release Readiness Analysis and Reporting

5. Regular Product Deploy and Configuration, Test Environment Configuration,

Test Data Preparation

* If estimated by PM, it should NOT be estimated here and it should be set to 0h

Example of typical repetitive activities

How to calculate Regression?

1. Number of sprints: (e.g. 12)

2. Number of regression testing cycles: (e.g. 6 or 3)

3. Pure New functionality testing time ⇒ (without time working with bugs and

test environment multiplier)= (e.g. 300h)

4. Percentage of regression-tested functionality: (e.g. 60%)

5. Test environments Multiplier: (e.g. 2)

How to calculate Regression?

Pure new functionality testing time (if 0 bugs)

How to calculate Regression?

Regression cycles: 6; regressed functionality: 60%

How to calculate Regression?

Regression testing time * Test environments Multiplier: 2

How to calculate Regression?

If number of regression cycles is 3

How to calculate Regression?

Calculation:1) Time for last regression cycle = Pure New functionality testing time (0 bugs) * % of regressed functionality * Test environments Multiplier

2) Regression increment = Time for last regression cycle * number of regression cycles

3) Time for regression testing = Regression Increment * (1+2+… number of regression cycles)

Example of Regression calculations

What basis for Smoke Testing?

1. Number of smoke testing cycles in sprint

2. Number of sprints/iterations expected

3. Approximate time for smoke

4. Test environments Multiplier

Example of Smoke calculations

What Risks should be included?

1. Changing/Not Stable Requirements, Increased scope (10%)

2. Documentation and requirements clarification problems (7%)

3. Risk of functionality integration/interdependence and regression

problems (10%)

4. Buggy application risk, because of developers competence (5%)

5. Technology risks (5%)

6. Risk of not mature processes and time spent for improving them (5%)

7. Infrastructure & environment Security & Accessibility Risks (5%)

8. Cross project communication necessity/dependency risk (5%)*

Example of Risks basis

Example of Summary calculations

Demo of real-life example

Demo of real-life example

Statistics

1. General amount of estimated projects: 15

2. General amount of the project where QA estimation should be adjusted, because of scope and deadlines changes: 5

3. Duration of the projects: 1 -12 months

4. Team size: 5-30

Questions