test pyramid agile pune conference
DESCRIPTION
Test Automation - A Right Way!TRANSCRIPT
![Page 1: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/1.jpg)
![Page 2: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/2.jpg)
Inverting Test Pyramid
Experience ReportSachin Natu @natusachin
Naresh Jain @nashjain
![Page 3: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/3.jpg)
25YEARS 6000+CLIENTS Nr 1M ROOMS 94COUNTRIES 95%+
ago we started
helping clients
manage their
revenue
later we’re still finding
the revenue opportunity
in every situation
around the world are
priced every single day
using IDeaS
on 6 continents have
hotels that now run on
IDeaS Revenue
Management
Systems
More than 95
percent of all
our clients are
still working
with us today.
About IDeaS
![Page 4: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/4.jpg)
Our Worldwide Clients :
Hospitality + Travel + Distribution
![Page 5: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/5.jpg)
Agenda
● Where were
we?
● Problem
● How we tried to
solve?
● How we
optimized?
![Page 6: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/6.jpg)
Status of regression time
Page
![Page 7: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/7.jpg)
No Automation !
![Page 8: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/8.jpg)
Problems
● Thousands of
regression tests
● Cross cutting defects
found were more.
● Repeat Exeuction
![Page 9: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/9.jpg)
Moving to shorter release cycle - a dream?
![Page 10: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/10.jpg)
Regression period is incompressible
![Page 11: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/11.jpg)
Manual and repetitive tasks are error prone.
![Page 12: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/12.jpg)
Issue leakages
Page
Releases
![Page 13: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/13.jpg)
Let’s automate regression
Page
![Page 14: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/14.jpg)
UI Test
Page
![Page 15: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/15.jpg)
After Automation Introduced
![Page 16: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/16.jpg)
Production feedback
Page
![Page 17: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/17.jpg)
Duration…
2
Yrs
Team of 2 full time Automation Testers
![Page 18: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/18.jpg)
State of the Product
Manual
Checking 45%
End- to End UI Tests 40%
Integration Tests10%
Unit Tests5%
![Page 19: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/19.jpg)
Are we happy now?
Page
![Page 20: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/20.jpg)
Always Catching Up
![Page 21: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/21.jpg)
No pinpointed feedback
![Page 22: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/22.jpg)
Silos
![Page 23: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/23.jpg)
Tests are slow
![Page 24: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/24.jpg)
Costly to maintain
![Page 25: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/25.jpg)
Is there way out?
Page
![Page 26: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/26.jpg)
Analysis of Bugs Report
![Page 27: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/27.jpg)
Why test from far
![Page 28: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/28.jpg)
Manual
Checking
End- to End Selenium Tests 70%
Integration Tests20%
Unit Tests10% Unit Tests 70%
Domain Logic Acceptance
Tests 10%
Integration Tests 9%
Workflow API
Tests 6%
End to
End Flow
Tests 4%
UI
1
%
Typical test strategies lead to an inverted testing pyramid
This is the need of the hour….Courtesy: Naresh Jain @AgileFAQs.com
The Test Pyramid
![Page 29: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/29.jpg)
Our transition story…
• We decided to "Test things right , at right place on right time" which would result in "right test pyramid“
Unit Tests 70%
Domain Logic Acceptance
Tests 10%
Integration Tests 9%
Workflow API
Tests 6%
End to
End Flow
Tests 4%
UI
1
%
![Page 30: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/30.jpg)
Business logic test
![Page 31: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/31.jpg)
Current Status
![Page 32: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/32.jpg)
Benefits
![Page 33: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/33.jpg)
Automation is part of development
• TDD
• BDD
• UI Tests only worry for UI
![Page 34: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/34.jpg)
Improved collaboration
Developer QA
![Page 35: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/35.jpg)
Grouping Tests - ROI
80 Scenarios - 300 + Min!
![Page 36: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/36.jpg)
Grouping Tests - ROI - Pyramid
80 Scenarios < Min!
End to End - 4 Min
UI Tests - 4 Min
![Page 37: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/37.jpg)
Regression timelines
Page
Months
![Page 38: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/38.jpg)
Feels like Agile now...
Page
![Page 39: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/39.jpg)
Challenges
• Legacy code - “Unit test unfriendly”
• Mapping acceptance tests with various slices of pyramid
• Building team competencies
• Pairing/ collaboration is backbone
![Page 40: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/40.jpg)
Key Learnings
• Do not overdo UI regression automation
• Keep it simple/pinpointed
• Design for Testability
Unit Tests 70%
Domain Logic Acceptance
Tests 10%
Integration Tests 9%
Workflow API
Tests 6%
End to
End Flow
Tests 4%
UI
1
%
![Page 41: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/41.jpg)
Naresh Jain @nashjainSachin Natu @natusachin
Page
![Page 42: Test pyramid agile pune conference](https://reader035.vdocuments.us/reader035/viewer/2022062308/5599c9f51a28ab00368b458a/html5/thumbnails/42.jpg)
Some Statistics...