performance testing in an agile world sai …...performance testing in an agile world presented by:...
TRANSCRIPT
![Page 1: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/1.jpg)
T5 Track
10/4/2012 9:45:00 AM
Performance Testing in an Agile World
Presented by:
Sai Subramanian
Cognizant Technology Solutions
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
![Page 2: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/2.jpg)
Sai Subramanian Cognizant Technology Solutions
As a manager at Cognizant Technology Solutions, Sai Subramanian is responsible for delivering
web performance innovations and performance engineering services to Fortune 500 clients in
North America. During the past six years, Sai has supported major performance initiatives for
well-known healthcare, insurance, and retail clients in varied roles including performance
tester, performance analyst, performance architect, and performance engineer. Sai is a
member of the Computer Measurement Group and has published in its monthly journal,
MeasureIT. Prior to Cognizant, Sai was a researcher at the Software Engineering Institute and
Robotics Institute of Carnegie Mellon University working on software engineering processes
and methodologies that can be applied to robotics application development.
![Page 3: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/3.jpg)
1
Sai Subramanian
Sr.Manager–Cognizant
PERFORMANCE TESTING
IN AN AGILE WORLD
1
What we will discuss today
1
2
3
4
We Willdiscuss today
Performance testing in a Waterfall /Agile
processWhy fitting Performance testing in
Agile is difficult?
A Case Study to explain the different levels of collaboration
Scrum of scrums
approach for programs with
multiple scrums
![Page 4: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/4.jpg)
2
2
Waterfall-Gurukula
ReleaseTestingDevelopmentArchitecture & Design
RequirementsGathering
3
Agile-School
![Page 5: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/5.jpg)
3
4
Performance Testing
TestingWaterfall
Scrum
5
We learnt it the
Hard way…
Fitting Performance in Agile
![Page 6: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/6.jpg)
4
6
The case study-The Project
Life Policies
DI Policies
Annuity Policies
Master Data
Management
(Customer Data)
Policies
Internet
Multiple Channels
Phone
fax
E -mail
Desktop
Single Servicing Environment
IVR / CSR
Room
Web
Self
Service
End -to -End
Business Process
Management Workflow
Holistic
View
1 Truth
Customer Experience
Servicing Platform
PEGA
------------------------------------------------------------
7
Multiple Scrum Teams
Desktop
team
Web
self-serviceSOA
1. 3 scrum teams running in parallel with their own codebases
during the course of the release
2. Infrastructure team
3. Governance team
![Page 7: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/7.jpg)
5
8
Sample Release Plan
TO
Code
Freeze
Implement
1/1 7/1
2/1 3/1 4/1 5/1 6/1
Code
Freeze
Sprint 1
12 Days
Implement
Sprint 2 Sprint 3 Sprint 4 Sprint 1 Sprint 2 Sprint 3 Sprint 4
Plan
Plan
Sprint 1
Plan
Release 6
3/18/2010
Release 4
6/18/2010
Release 5
Retro
Retro
Release 5Release 4
DAY
9
Roll-out plan & Issues
• Application was rolled out in phases to the CSR’s and Self-service users
• Issues
Adoption
rate was low
Complaints-
overall call
handling time
increased
Training/Performance
(Sadly both!)
![Page 8: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/8.jpg)
6
10
Release Integration-Performance Testing Model
• Role : Member of the infrastructure team
• Challenges faced
Performance team and Core team speak different languages
Bottlenecks are not real/reliable
Integrated rule-sets are available just before the release
Performance team is not identified/accepted as a part of the Core team
• StrengthsLow cost to the project
Flexible demand management for the CoE
11
Integrated Build and Performance Test Release Integration-Performance Testing Model
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-1
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-2
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-3
Integrated Build
Performance Test
![Page 9: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/9.jpg)
7
12
Student-Teacher Analogy
汉语汉语汉语汉语拼拼拼拼音音音音
எ�ன??
When the student and the teacher speak different languages-it makes it really hard to evaluate
13
TESTING
MODEL
Performance
analyst
should be a
part of the
Sprint
planning
Performance
analysts and
core team
should
understand
each other
Performance
analyst
should be a
part of the
infrastructure
team
Performance
issue, root-
causes =
Unknown
Unknowns
Release Integration-Performance Testing Model Lessons Learned
![Page 10: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/10.jpg)
8
14
Student-Teacher Analogy
எW
音音音音
Student understand the language but may not really have the answer for the questions-known Unknowns
எ�ன=What =
汉语拼音汉语拼音汉语拼音汉语拼音(hanyupinyin)
15
Sprint Integration-Performance Testing Model
• Role : Member of the infrastructure team
• Challenges faced
Performance team cannot keep up with the application delivery
Bottlenecks are identified but the root-causes are not known
Code used for performance testing is not in synch with development
Performance sign-off became a critical path bottleneck to release
• Strengths
Performance Analyst understands the application, performance critical use-
cases, environment and data
![Page 11: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/11.jpg)
9
16
Integrated Build and Performance Test
Integrated Build
Performance Test
Sprint Integration-Performance Testing Model
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-1
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-2
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-3
17
Sprint Integration-Performance Testing Model Lessons Learned
TESTING
MODEL
Performance
Analyst should
be a part of the
scrum teams
Scrum team to
conduct their
own performance
tests prior to
rule-set
integration
Sr.Perf Analyst
should drive
integrated
performance
tests holistically
Performance
issues , root-
causes =
Known
Unknowns
![Page 12: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/12.jpg)
10
18
Student-Teacher Analogy
Known language with continuous evaluation and feedback processes-known Knowns
19
Scrum of scrums approach
• Role: Active members (pigs) of all the scrum meetings that run concurrently
• Sr. Performance Analyst drives the daily performance meeting:
• The agenda will be
Challenges
• Management’s commitment to dedicate time and resources to drive the performance effort
• Organizational culture shift for developers to commit on performance user stories
Strengths
• Performance analysts understands the application, performance critical use-cases, environment and data
• Bottlenecks are identified but the root-causes are also known-Known knowns
Performance tests executed yesterday
Blockers/bottlenecks identified
Readiness for integration
![Page 13: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/13.jpg)
11
2
0
Integrated Build
Performance Test
Integrated build and Performance Test Scrum of scrums model
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-1
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-2
Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-3
Scaled-down perf
tests in Dev env
21
Sprint Integration-Performance Testing Model Lessons Learned
TESTING
MODEL
Performance
should be
owned and
evaluated
continuously
Early
performance
analysis with
creative
performance
tests
Improved
performance
with a significant
increase in the
effort and cost
Performance
issues , root-
causes =
Known
Knowns
![Page 14: Performance Testing in an Agile World Sai …...Performance Testing in an Agile World Presented by: Sai Subramanian Cognizant Technology Solutions Brought to you by: 340 Corporate](https://reader030.vdocuments.us/reader030/viewer/2022040306/5ec46effef4f3c57292732f0/html5/thumbnails/14.jpg)
12
22
Summary
• Management’s realization on the criticality of performance in
Agile projects
• Sr. Performance Analyst required to manage projects with
multiple scrums
• Performance analysts to embed within each scrum teams to
understand the project
• Performance user stories to be collectively owned by the
entire team
• Significant increase in the cost