effort optimization through automation

© 2014 Cognizant

Effort Optimization through Automation

Diljeet Dhillon Sr QA Director (Loblaw Companies Ltd)

Jatin Pathak Associate Director (Cognizant Technology Solutions)

Mathi Natarajan Sr Consulting Manager (Cognizant Technology Solutions)

Sep-30, 2014

Diljeet (Dil) is a 14 year veteran in IT industry across various

capacities. He holds a Bachelorette degree in Electrical

engineering and is a Certified SAP, ITIL & PMP professional.

He currently holds the position of Sr Director Quality Assurance at

Loblaw.

In his current role he is in-charge of establishment of quality

standards (present and long-term strategies), development and

monitoring of polices and procedures aimed at mitigating risks to

systems quality, developing metrics in support of continuous

improvement, tracking and reporting, providing recommendations

for improvement, introducing and encouraging best practices in

the area of re-use, test automation, risk management, test

strategy, test plan / case creation and execution, defect tracking

amongst other quality related initiatives.

Diljeet Dhillon

Sr. DirectorQA at Loblaw Companies Ltd

Speaker Profile

With 19+ Years Testing Leadership experience with various leading IT

organizations, Jatin has seen the evolution of QA as a discipline over

past 2 decades. As a Testing Industry Leader, Jatin has successfully

lead Integrated QA Service Delivery for some of the large and complex

testing engagements, across Multiple Geographies such as US,

Canada, South Africa, India, APAC for several Industry Verticals such as

Retail, Banking, Insurance, E&U, eSecurity.

This has involved Establishing Trust as an Advisor to Client Senior

Leadership & building long term Client Relations, Consulting in the

spirit of Partnership aimed at improving the IT and QA Maturity of our

clients, enabling them to become Best In Class, and Partner with them

through their Transformational journey.

Jatin is an active member of various QA forums and groups, an active

writer on the subject of Leadership, Testing Models, and an avid

believer that ‘anything that is worth doing at all, is worth doing well’.

Jatin Pathak

Canada Regional QE&A Practice Leader – Cognizant

Technology Solutions

Speaker Profile

Mathi Natarajan currently work as Senior consulting manager at

Cognizant Technology Solutions, a leading provider of information

technology and consulting services. He is an SAP certified Solution

consultant with implementation, production support and product

development experience around SAP CRM, IS Retail, BPM,

Mobility and HANA technologies.

He has broad experience that spans Telecom, Media, Retail, CRM,

BPM and Business Intelligence in a wide variety of technologies

and testing techniques.

He leads the SAP Test Consulting services and is responsible for

SMAC (Social, Mobile, Analytics, Cloud) and Next Generation

Testing solutions for North American Clients.

Mathi Natarajan

Sr. Consulting MgrCognizant Tech. Solutions

Speaker Profile

© 2014 Cognizant 5

Agenda

Familiar Pain Points1

Typical Solutions & Shortcomings2

Challenges3

Automation Transformation4

Implementation Approach5

Dependencies6

QA Values from Automation7

Summary8

The Next Evolution9

© 2014 Cognizant 6

Familiar Pain Point

Environment Issues

Test Data Issues

Lack of time due to upstream schedule slippages

Lack of Speed

Complexity of multiple E2E Scenarios

Build Quality – # of Defects, Resolution & Turnaround Time

Coverage – Negative Testing as well as overall business process coverage due to time

“ What are the typical challenges faced during regression releases ?“

© 2014 Cognizant 7

are we automating it the RIGHT Way First Time

How do we solve this?

Yes, But !!!! ……………..

© 2014 Cognizant 8

Typical Automation Solution Approach & their shortcomings

Script based• Very Detailed

• No Standard Plug & Play access

Automate Component

of Solutions

Non Agile Delivery

• High Dependency on functional SME

• High Cost and Maintenance effort

• Linear automation of FIFO (First in First out)

QTP Tooling • Tool not application specific

© 2014 Cognizant 9

Challenge that we Faced

Increase the QA Coverage Coverage – Cover 680 E2E Business processes

Increase the Speed Speed – 14 days to 3 days

Reduce the Cost Cost – Reduce the cost through Increased Automation

© 2014 Cognizant 10

How did we Implement this

Business Process Automation

Scriptless Automation

Test Management Tool Integration

One-Click Reporting

Developed E2E (End to End) BusinessFlow Diagrams

Identified the Critical businessfunctionalities for Automation andBusiness process enrichment

Business functionalities decomposed tocomponents based on the applicationarea and functionality

SAP & Non SAP Business

components are identified based on

RICEFs (Custom Objects) &

Application area

Components are automated with

Worksoft Certify & QTP

All artifacts centralised in HPQC

Automation execution

integration and reporting through

HPQC

Integrated Scenario wisereporting

End to End traceability oninput and output

Built –in Data & Resultvalidations

© 2014 Cognizant

11

Example – Create a PO to Increase Stock at store

Current Scenario

Forecast Run PO Creation Pick & Pack Routing Goods Receipts Billing

IPFR SAP WMS TMS SAP SAP

© 2014 Cognizant

12

Example – Create a PO to Increase Stock at store

Forecast Run PO Creation Pick & Pack Routing Goods Receipts Billing

IPFR SAP WMS TMS SAP SAP

Transformed Scenario


QTP

Regression

Project Release Cycles

Test Case based

Traditional Scripting Techniques

Worksoft /Selenium/ QTP/ Perfecto /Spritz/ etc

SIT / Regression / UAT / PT

Project Release Cycles / Environment readiness /

Business Assurance

Business Process Scenarios based

Script less Frameworks / Technology Accelerators

/ Integration

QA

Automation Transformation

Current Scenario Transformation

What

Who

Where

How

When

Why

QA/Business/Dev


Success Depends upon


Dependencies

Shorter defect turn around

E2E test data quality

Defined test requirement scope

Success

Depends On

Business flow/Scenarios prioritization

Build Stability

Test tool availability and compatibility

Defect management process/tool

Test environment stability

© 2014 Cognizant

Communication Tool

Q1 April May JuneRegression Execution Time

Without quality issues 10 days 10 10 3 to 5 days

With existing quality issues 14 days 14 14 10

Top Issues - External to QA1) Landscape 4 Quality

Environment stability ~70% 80% 90% 100%

Environment defect resolution time (days)Sev1-2days/Sev2-4days/Sev3-5days

Sev1-2days/Sev2-4days/Sev3-5days

Sev1-4hrs/Sev2-8hrs/Sev3-12hrs


Data Refresh Accuracy ~90% 90% 95% 100%

2) Functional Defect Leakage ~20% 10% 5% 0%

3) Functional Defect resolution time (days)Sev1-2days/Sev2-4days/Sev3-5days

Sev1-2days/Sev2-4days/Sev3-5days



4) Regression Defect Resolution Time (days)Sev1-2days/Sev2-

4days/Sev3-5daysSev1-2days/Sev2-4days/Sev3-

5daysSev1-4hrs/Sev2-8hrs/Sev3-

12hrsSev1-2hrs/Sev2-4hrs/Sev3-

6hrs

5) 3 month rolling forecast/build synergies No Yes Yes Yes

6) Defect Reopen ratio 10% 7% 5% 2%

Top Issues - Internal to QAAutomation % 38% 40% 44% 48%

16


What’s Next

How has QA Role Changed in

SDLC ??

Consulting

(Risk)

Innovation

(Speed/Cost)PM

(Dependencies)


Next Evolution

Proactive Monitoring

Lights off execution

Stop tester tell you when

application is down

Extreme Automation


Next Evolution

Proactive MonitoringTest Data Management

Extreme Automation

Automated test data selections Vs

Creation from Scratch

Golden Data set mapped to

business process


Next Evolution

Proactive MonitoringSHIFT LEFT- SHIFT RIGHT

Extreme Automation

Shift Left/Right within STLC +

Start early


Next Evolution

Prod Data Maintenance

Extreme Automation

Production Data Quality

improvement by leveraging

Integrated Test Data

Solutions


Next Evolution

Dev Ops

Extreme Automation

Dev – One click simulation

Ops – Data Query/ Quality

checks to be repeated


Summary

SOLUTION

Complexity of multiple E2E Scenarios

Environment Issues

Test Data Issues

Build Quality – # of Defects, Resolution & Turnaround Time

Lack of Speed

Business Process Automation

Scriptless Automation Tool

Faster

Higher throughput of production/warranty incidents

Increased speed to Market Increased ASM Build

productivity

CHALLENGES

ISSUES

Better

Business Process availability

Higher QA coverage through increased automation

Cheaper

Regression effort reduced to 3 days ~ 25% reduction in

efforts

DEPENDENCIES

Requirement / Build Stability

Environment Stability

Data Quality

Extr

em

e A

uto

mat

ion


Questions?

Thank You

[email protected]

[email protected]

[email protected]

mailto:[email protected]



effort optimization through automation

Documents