create sanity amidst the test methodology madness - webinar presentation

8/4/2019 Create Sanity Amidst the Test Methodology Madness - Webinar Presentation

1/16

2011 MindTree LimitedCONFIDENTIAL: For limited circulation only Slide 1

2011 MindTree LimitedCONFIDENTIAL: For limited circulation only

Creating Sanity Amidst Test

Methodology MadnessRight-sizing your test methodologies


2/16


Agenda

hen hat ow

Identify the

symptoms

Key aspects

to right-size

MindTree

Tools &

Frameworks

Processes should work for you. You should not work for processes.

Rightsizing?


3/16



Poll Question #1

Are your processes taking too much effort and time?

Yes

No

Not able to measure


4/16


When to Rightsize?

Acknowledge the need. Dont wait until you loose your sleep.

Test processes taking too much effort and time?

Metrics not helping you take decisions?

Not able to justify the investments in testing?

Test process not helping you identify the criticaldefects early?

Difficulty in quantifying the quality in your test

processes?

hen

Identify the

symptoms


5/16


What to Rightsize?

Tailor your processes for right fit.

hat

Key aspects

to right-size

Quality goals / objectives

Software life cycle methodologies

Engagement models

Governance mechanism (metrics,

reviews)

Tools and frameworks


6/16



Poll Question # 2

Do you know what metrics should you be tracking for your testingneeds?

Yes

No

I do, but not sure if they are exhaustive


7/16 2011 MindTree LimitedCONFIDENTIAL: For limited circulation only Slide 7

Test Metrics Analysis and Decisions (TMAD) Model

Metrics

BasedGovernance

Business goals

Quality goals

Governance

Metrics, reviews,baselines

Forecast

Risks, best practices

Corrective & preventive

actions


Statistical

Models

Historical

Data

Org / Industry

Baselines

Enablers

ow

MindTree

Tools &

Frameworks




Proposed Metrics Goal Associated Queries (RCA) Undesired Behaviour(Risks)

Defect Containment

Effectiveness

1. Improve testing

quality

2. Prevent defect

leakage

1. Was the criteria to classify defect detection phase

clear from a leakage perspective?

2. Are there clear entry and exit criteria between phases

3. Was the team's knowledge adequate to uncover

leaked defects?

4. Was the scope of testing for each phase clear?

5. Do we have requirement traceability matrix?

1. Blame game

2. Manipulation of defect

classification criteria

Decision Support (Scope)

Corrective ActionsRaw Data Requirements Suggestions(feedback)

1. Improve test case review

effectiveness

2. Improve process compliance

3. Improve product and test design

knowledge

1. Clear entry and exit criteria

between phases

2. Process adherence

3. Effective knowledge management

4. Update test suite

5. Get test strategy reviewed by

customers

1. Defects detected in

test phase

2. Leaked defects

Do RCA for every release

and identify preventive

actions

Metrics Analysis

Metrics Decision Table


9/16 2011 MindTree LimitedCONFIDENTIAL: For limited circulation only Slide 9 2011 MindTree LimitedCONFIDENTIAL: For limited circulation only Slide 9

Test Quality Index (TQI) Model

Bivalent questionnaire; Y or N

responses

Productivity per tester vs.quality of deployed resources

and resource deployment

Contextual metrics vs.comprehensive score; across

30 process areas

Scoring based on selfassessment

1Identify

areas of

improvement

2Customize

the scored

checklist3Improve

scores with

each

deliverable

Winner of the Testing Leadership Award at QAI Software Testing Conference



Poll Question # 3

Do you have an integrated framework which helps you rightsize thetest processes and methodologies?

Yes

No

We have point solutions, but not an integrated solution



MindTest- Integrated Test Methodology

Feedback

Analysis Audits

TestEstimates

TQI TMAD

Test Management

TestLink

Test Measurement and

Metrics - ROI

Test Dashboard

Lifecycle Processes

ProcessNet

Knowledge

Management

ProjectSpace

Defect Management

Radar

Tools Technologies Domain Type of Testing Stage of TestingCustomization

Processes Templates Guidelines Checklists Best Practices

Continuous

Processes

Metrics &

Measurement

LifecycleProcesses



Executed in-depth analysis to understand

Quality objectives and alignment between test process

and application's business goals

Used Test Metrics Analysis and Decisions (TMAD) Model

and Test Quality Index (TQI) to come up with

appropriate metrics and processes

Collected key metrics as governance enablers to measure and

drive decisions based on the health of the projects

MindTree Solution

The Business Challenge

The customer is the world's leading specialist in air transport

communications and information technology (IT) solutions.

90% coverage on test cases executed, a significantimprovement over the earlier metric

100% achievement on acceptable open defects for severity 1

and 2

Customer overhead of test case reviews reduced by 60%

Constant review of the test case execution helped us to

evaluate and report how far/near is the application from the

acceptance criteria at different phases of testing

Enabled development team to prioritize the defectsthat needs to be fixed for the immediate build

Customer is now able to ascertain the quality of the

application

Reduced risks and enabled effective decision making through

test dashboard

No testing guidelines

Inadequate process

Lack of measures for impact of testing

No single snapshot of engagement level governance

The Customer Business Impact

MindTree's expertise in metrics is paramount in managing

the engagement and seeing if process changes have made the

hoped for improvement and if the application is ready for

release. The metrics are also a great way to communicateprogress across all the groups in this engagement.

Senior Manager, Quality Integration

Case Study



In Short

You dont have to search for clues in thelabyrinth

Identify theneed

Evaluate thekey aspects

Optimize theprocess

Measure andimprove



MindTree Testing Services Overview

Testing Size and

Focus

Dedicated unit of 2200+

specialists focused on

testing

Contributes 25% of the

MindTrees global

business

Testing Depth and

Breadth

Tested over 2000

applications & products

from chip to cloud

Services ranging from

point solutions to long

term CoEs

Domain specific testing

services and solutions

Testing Process and

Delivery Excellence

Testing pillars of

excellence - MindTest,

TestLabs,

TestingAcademy

Virtual practices and

CoEs for effective

delivery

Part of ISO working

group in defining new

testing standard ISO /

IEC 29119

President of Indias

leading testing forum -

STeP-IN

Thought Leadership

1500+ 95%700+ 20+

deliveringPREDICTABLE QUALITYthrough INDEPENDENT TESTING



ARE YOU TRACKING THE RIGHT TEST METRICSFOR YOUR BUSINESS DECISIONS?

Complimentary Workshop at your Doorstep

REGISTER NOW

or email to [email protected]

www.mindtree.com/events/testingwebinar/index.html

MindTree Workshop


16/16 2011 MindTree LimitedCONFIDENTIAL: For limited circulation only Slide 16 2011 MindTree LimitedCONFIDENTIAL: For limited circulation only

Sylvia Veeraraghavan

[email protected]

www.mindtree.com

create sanity amidst the test methodology madness - webinar presentation

Documents