tech mahindra digital assurance practice - schd.ws · •bfs, insurance •retail, cpg ... use the...
TRANSCRIPT
Copyright © 2017 Tech Mahindra. All rights reserved.
Tech Mahindra Digital Assurance Practice Overview
20+ Years of Industry Experience
delivering testing services
12500+Test talent pool with expertise in
domain, test tools and processes
1500+Enterprise and Telecom Business Processes
available as a Jump Start Kit for testingExperience we bring
Our Test Capabilities
Thought Leadership(Awards & Recognition)
Functional Testing (SIT, E2E Business Process Testing,
Agile, Data Migration, UAT)
Non-Functional Testing
(Performance & Security)
Specialised Testing
(Automation, Network testing, Mobile, IoT, BIG Data,
Analytics, BI, ERP, Network, BYOD, M2M, DWH, Crowd
Sourcing)
• HPE
• IBM
• CA
• TOSCA
• Microsoft
• Oracle
Key Testing Alliances
• TMMi aligned
consulting,
• Service
Virtualization,
• Automated Test
Assurance(Model
Based Testing) ,
• QA as a Service
Differentiated QA
Services
• BFS, Insurance• Retail, CPG• Travel & Logistics• HLS, Manufacturing• Oil & Gas• Energy & Utilities• Media &
Entertainment• Telecom
Domain QA Services
• ERP Testing
• TaaS Platform
• Device Testing
• Security Testing
• Agile Testing
• DevQAOps
• Managed Services
Advanced QA Services
Tech Mahindra’s Global Test Factory is the first one to get “TMMi Level 5” certification in “One Go”
Copyright © 2017 Tech Mahindra. All rights reserved.
DevQAOps Industry Trends
“By 2020, DevOps initiatives will cause 50%
of enterprises to implement continuous
testing using frameworks.”
– Gartner
According to the World Quality Report, the
reliance on manual testing is the top
technical challenge in DevQAOps alignment
Businesses see test automation as a way to
address one of their top challenges to
delivering better software faster, by
adopting Agile, DevOps and continuous
delivery practices to shorten delivery
cycles.”
– Forrester
Copyright © 2017 Tech Mahindra. All rights reserved.
First – Need To Align QA Organization for Digital Enterprises & DevQAOps
Move from traditional testing phases to more continuous testing and quality to meet shorter
deadlines
Adopt lean design and methods to reduce time to market in manual testing
Focus on Life Cycle Automation – Remove Manual Interventions wherever possible
Proactively Predict, prevent defects – Prevention over Detection is the key!
Shift to Federated Community of Practice model from tightly couple TCOE
Invest in Application and Lifecycle Management Platforms
Assign Higher Priority to Customer Experience Testing, Graduating to Brand Assurance
Combine behavior driven and exploratory user scenario based testing to understand end user
interactions
A shift in the skills of manual testers, the development of script-less or testing object based
frameworks, and the deployment of automation software tools around DevQAOps.
Copyright © 2017 Tech Mahindra. All rights reserved.
Change the way we Test – Its all about CI/CD/CT!!
Copyright © 2017 Tech Mahindra. All rights reserved.
Key QA Challenges & constraints in Agile / DevOps
Continuous Integration• Are the Requirements Testable / Valid?• Is Unit test coverage sufficient?• Is Unit Testing automated?• Is hand-off to Test automated/smooth?• Are Non-Functional Requirements defined?
Continuous Testing • Is In-Sprint/Continuous Automation practiced?• Is automation coverage sufficient to support frequent builds?• Is QA lifecycle (Test design, execution & reporting) automated?• Are standardized Automation Frameworks available?• Is Test Data managed, up to date and accessible?• Are test environments available and up to date?• Are assets reusable and utilized?• Is test reporting and defect management process automated?
Continuous Deployment• Are production defects retested after fix?
• Is RCA carried out to prevent recurrence?
Levers
Processes
Tools
People /
Organization
Copyright © 2017 Tech Mahindra. All rights reserved.
Agile- Quality Gates
Requirements
preparation
Contribute to
DoD
Test
Assessment
Release
Testing
and
Release
Planning
Operations
Sign-offs
& Steady
State
Sprint
Zero
(Exception)
Product Demo & Sign-off
Sprint
Planning
Sprint Retrospective
Sprint
Execution
Dev. + Test (from
TCoE)
Release
Definition of
Done
Test Plan (living
Document)
Prioritized
Sprint Backlog
Capacity
Planning
inputs to
TCoE
Identify QA
Improvement
Areas
Solution Definition & Planning Sprint Development Sprint E2E System Integration & Lifecycle
Completed Release
component integration
and unit test
Completed
Functional and
E2E Regression
Testing
LegendTCoE activity
(Test Architect or testers
Embedded in Agile team)
Joint Agile Team
Activity - including TCoE
Release Go /
No Go
Copyright © 2017 Tech Mahindra. All rights reserved.
Automation in Agile / DevOps – Key Considerations
Continuous Testing Is Key
• Smoke / Sanity
• Regression
• Full Functional
Services Testing
• APIs
• Microservices
• Service Virtualization
Testing at Scale
• Multi channel/device/browser/OS
Cloud
• Environments
• Cloud Migration
• Infra as a Service
Copyright © 2017 Tech Mahindra. All rights reserved.
Progressive In-sprint Automation
Backlog
Sprint
Executio
n
Sprint
Sign off
Sprint - 1
• Functional/Exploratory testing for stories
• No Continuous testing
• Regression not requiredSprint - 2
• Functional/Exploratory testing for stories
• Automated Continuous testing
• Automated Regression testingSprint - 3
• Manual/Exploratory testing for stories
• Automated Continuous testing
• Automated Regression testing
Backlog
Sprint
Executio
n
Sprint
Sign off
Backlog
Sprint
Executio
n
Sprint
Sign off
Automate
in Sprint
Automate
in Sprint
Automate
in Sprint
Functional test cases
Automated test cases
• Early Automation testing of user stories during Sprint execution
• Nightly automated testing during the sprint execution using automated suite consisting of
test cases up to previous sprint (n-1)
• Automated regression during the sprint sign-off using automation suite including of test
cases of current sprint
Copyright © 2017 Tech Mahindra. All rights reserved.
Agile Test Automation: Learnings and best practices
Focus on in-sprint automation
Use the initial dev time of the next sprint to increase automation coverage of previous sprints
Test Driven / Behavior driven development techniques to shift-left the test cycle
Investment in techniques like Model Based Testing, script-less automation can pay big dividends by quickly
updating the test suite based on changes reflected in the model
Focus on Test Lifecycle Automation from requirements to reporting to enable Continuous Delivery Pipeline
Involvement of testers in the discussions with business users from the ideation stage helps maintain the right
alignment
For major releases, having a regression team member as a part of the dev-test scrums ensures that the right
regression pack is updated and available
Copyright © 2017 Tech Mahindra. All rights reserved.
Agile Test Automation: Learnings and best practices contd..
QA and release management functions are quasi-merged to
ensure availability of right Test data, environments and version
control
Test Automation does not yet fully replace manual and
exploratory testing
Techniques like Risk based testing can help to identify and
prioritize the right candidates for automation to attain a faster
ROI
The fast timelines in an agile/Devops model necessitates the
maximum possible automation for performance and mobility
testing
Effectiveness of the release and sprint planning is directly
proportional to schedule variance
Tool selection needs to be done keeping the balance between
enterprise/regression suite and flexibility/velocity in mind
As requirements keep changing
frequently, focus on automation
efforts on Unit/component and
API testing as against the UI
layer
Unit Tests
API Tests
UI
Tests
70-90 %
10-20 %
5-10 %
Copyright © 2017 Tech Mahindra. All rights reserved.
DevQAOps - Reference Architecture
Code Quality
Functional Test Automation
Test Data Management
Serv ice Virtualization
Env ironment Automation / Prov isioning
Release & Deployment Automation Application & Infrastructure MonitoringRequirement Management Build Automation
Log-Analysis / Log Aggregation
Artifact repository
Vulnerability Assessment
Project and Portfolio Mgmt.
Capacity Management
Serv ice Management
CMDB: Infrastructure as code
Test and Defect Management
Operational Analytics
Control center for build pipelines, deployment pipelines and deliv ery pipelines
Integrated ALM
SCMCode Search Code ReviewEpics & User Stories
Orchestration
Project Trackers Change Management
Reports and Dashboards
Issue management
Collaboration / Document Management / Wiki
Build Tracker Deployment TrackerDefect Tracker
Integration Engine
AL
M P
latf
orm
Fu
ncti
on
al P
latf
orm
Continuous Planning Continuous Integration Continuous Testing Continuous Deployment Continuous Monitoring
Reports and Dashboard AdministrationOrdering Platform User Management
Copyright © 2017 Tech Mahindra. All rights reserved.
Key Process Areas for DevQAOps
SUPPORTING
PROCESSESMonitoring & Support process for projects
Training & Enablement
KPIs & Metrics Project On-Boarding Process
Roles and Responsibilities
Communication & Change
Management. Governance
CONTINUOUS
INTEGRATIONConfiguration Management
Code review and Analysis
Build and Dependency Management
Repository Management
Work Item Management
Unit Testing & Test Automation
Traceability & Feedback
CONTINUOUS
TESTING
Test Planning & Management
Test Asset Management
Test Automation (Fun & NFR) Test Environment
Test Virtualization for Stubbing
Defect Management
Traceability & Feedback
RELEASE AND
DEPLOYMENT
Environment Configuration
(Infrastructure as code)
Environment Provisioning
Release Planning and Orchestration
Deployment Strategies and Automation
Policies and
security
Workflow
AutomationTraceability & Feedback
CONTINUOUS
PLANNINGRelease
Planning
Infrastructure & Resource Planning
Agile Delivery Planning
Task PlanningBusiness Alignment
Traceability & Feedback
Automation of each process will bring the DevQAOps Maturity
Copyright © 2017 Tech Mahindra. All rights reserved.
Co
de
C
heck in
Jenki
ns Job
-T
rig
ger
Jenki
ns Job
-T
rig
ger
Je
nki
ns
Jo
b
-T
rig
ge
r
E2E Dev-QA-Ops Workflow (illustrative)
End to End DevOps Solution: Powered by ADOPT™ framework
Je
nki
ns
Jo
b
-T
rig
ge
r
Dev elopsusing
Deployment (Release
Automation)
Binary Files
Bin
ary
F
etc
h
QA Env
Je
nki
ns
Jo
b
-T
rig
ge
r
Dev Team
QualityGate
Pass
FailJe
nki
ns
Jo
b
-T
rig
ge
r
Fetch
QualityGate
Pass
Fail
QualityGate
Pass
Fail
QualityGate
Pass
Fail
DeployProject
Start
QualityGate
Pass
Fail
Unit
Testing
Repository Manager
Automated Build
Automated Defect Flow
QualityGate
Pass
Fail
Functional Testing (iAF)
Integration
Testing Performance Testing
Security testingAcceptance
Testing
QualityGate
Pass
Fail
Jenki
ns Job
-T
rig
ger
Jenki
ns Job
-T
rig
ger
Orchestration with Jenkins
Version
Control
QualityGate
Pass
Fail
Code Quality
Code Review
Gerrit
Je
nki
ns
Jo
b
-T
rig
ge
r
Monitor & FeedbackFeedback
Copyright © 2017 Tech Mahindra. All rights reserved.
Disclaimer
Tech Mahindra Limited, herein referred to as TechM provide a w ide array of presentations and reports, w ith the contributions of various professionals. These presentations and
reports are for information purposes and pr ivate circulation only and do not constitute an offer to buy or sell any services mentioned therein. They do not purport to be a complete
description of the market condit ions or developments referred to in the mater ial. While utmost care has been taken in preparing the above, w e claim no respons ibility for their
accuracy. We shall not be liable for any direct or indirect losses arising from the use thereof and the view ers are requested to use the information contained herein at their ow n
risk. These presentations and reports should not be reproduced, re-circulated, published in any media, w ebsite or otherw ise, in any form or manner, in part or as a w hole, w ithout
the express consent in writing of TechM or its subsidiar ies. Any unauthorized use, disclosure or public dissemination of information contained herein is prohibited. Individual
situations and local practices and standards may vary, so view ers and others utilizing information contained w ithin a presentation are free to adopt differing standards and
approaches as they see fit. You may not repackage or sell the presentation. Products and names mentioned in materials or presentations are the property of their respective
ow ners and the mention of them does not constitute an endorsement by TechM. Information contained in a presentation hosted or promoted by TechM is provided “as is” w ithout
warranty of any kind, either expressed or implied, including any w arranty of merchantability or f itness for a particular purpose. TechM assumes no liability or responsibility for the
contents of a presentation or the opinions expressed by the presenters. All expressions of opinion are subject to change w ithout notice.
Thank youVisit us at www.techmahindra.com