Testing Tools in the Age
of DevOps and Agile
Who am I?
Eduardo Riol, QA & Testing Technical Leader at
atSistemas.
I started a decade ago as developer but have spent last
seven years working on Software Quality Assurance,
both Processes and Products oriented.
Nowadays I am mostly interested in the control of
technical debt, BDD and QA integration in Agile and
DevOps environments.
Let’s connect on: twitter.com/eduriol
github.com/eduriol
linkedin.com/in/eduriol
What are we going to discuss today?
• Testing in the age of Agile and DevOps
• What do we need from a testing tool
today?
• Modern Test Management tools: let’s
review three of them
• Transitioning our testing suites from the
Waterfall Era
Why we shouldn’t be testing the same way we used to
Testing in Waterfall. How did it use to be?
QA is a stopper for deploying to production: Testers are the bad
cops
QA people? I think those are the guys in the 4th floor or
somethingWe haveQuality
problemsAin’t my problem, Software
Quality is what testers should care about, isn’t it?
Why we shouldn’t be testing the same way we used to
Testing in Waterfall. How did it use to be?
Testing as a separate phase. Estimated and
planned from the beginning of the project
An attempt to make this meaningful: V model
But have you ever really designed and executed
all those tests suites to check user requirements,
system requirements, analysis, design and
code?
Testing in Agile. How it should be.
Collaboration between Development Team and Business
Why we shouldn’t be testing the same way we used to
BDD
Why we shouldn’t be testing the same way we used to
Testing in Agile. How it should be.
The Testing manifesto
Testing throughout OVER testing at the end
Preventing bugs OVER finding bugs
Testing understanding OVER checking functionality
Building the best system OVER breaking the system
Team responsibility for quality OVER tester responsibility
Legacy testing software was not built for an Agile World
What do we need from a testing tool today?
Hard to use Doesn’t fit agile
workflows
Lack of
integrations with
agile tools
Not cross-
platform or
cross-browser
So… what does my tool have to accomplish in these
times?
Integration capabilities with third-party tools
Collaboration
Establishing conversations around work items
Usability: friendly experience
What do we need from a testing tool today?
So… what does my tool have to accomplish in these
times?
Agile workflows: test case management following Scrum,
Kanban…
Provides support for TDD & BDD
What do we need from a testing tool today?
BDD
TDD
Write a failing test
N cycles
So… what does my tool have to accomplish in these
times?
Fosters Exploratory Testing
Assistance on test automation
Scripts management and… Yes, generation!
What do we need from a testing tool today?
Exploratory Testing is not thatunstructured messy testing we usedto do when deadline is about to hit
us
Modern Test Management tools: let’s review three of them
Free open source that
Improves:
Usability
Managing and reporting
options
when compared to traditional
testing tools
Automatic test planning
when requirement changes
Tagging system
accelerates test
management
Dynamic test execution
Provides integration with
JIRA and Bugzilla. No
built-in defect tracker.
Nice report generation
but hardly customizable
Messy installation and
setup
Basic first aproximation when
trying to overcome legacy
tools and reducing costs
Modern Test Management tools: let’s review three of them
Modern Test Management tools: let’s review three of them
Both stand-alone and JIRA
add-on version
Test management as
JIRA object, which allows
leveraging Agile boards
All the User Experience
and querying capabilities
that JIRA provides
Nice reports and real-time
information
Integration with automation
tools like Selenium and CI
servers like Jenkins and
Bamboo (ZAPI, sold
separately)
Nice tool that integrates
smoothly with continuous
testing workflows
Still lacks support to Agile
practices like TDD, BDD and
Exploratory Testing.
Modern Test Management tools: let’s review three of them
Modern Test Management tools: let’s review three of them
Modern Test Management tools: let’s review three of them
qTest is the suite of testing
tools developed by
QASymphony:
Agile First orientation
Support the transition
from classic Waterfall
development to modern
paradigms of application
development and testing
Tools included in qTest range
from test case management
and the generation of
customizable reports to
support for exploratory testing
and other modern development
paradigms such as BDD
These tools are perfectly
integrated, and favors:
Working with third-party
suites
Incorporating native
integration with JIRA
Facilitating automation
and collaboration
among stakeholders
Modern Test Management tools: let’s review three of them
Robust and easy-to-use: Track,
organize, and report on all testing
activities through one lighting fast,
easy-to-use interface.
Integrate with JIRA, Automation
& More. Manager comes with out-
of-the-box CI tool integrations and
Rest APIs for custom set up.
Share Data Across qTest
Projects. Test data can be shared
across common project types so
testers know which data they need
to use
Modern Test Management tools: let’s review three of them
Your data, your way. Arrange and
display charts and graphs exactly
the way you want. Tabs let you
create multiple dashboards for
different projects
Customized reporting. Zoom in
and dynamically adjust the charts.
Click on a chart to view and
download the data
Visualize your data with
interactive heat maps that help
quickly identify where the issues
are in the application. Then drill
down to get more insights
Modern Test Management tools: let’s review three of them
Intelligent capture technology
tracks all interactions from the
testing session and automatically
creates defect documentation.
Instantly submit detailed defect
documentation to JIRA so you can
share with team members.
Modern Test Management tools: let’s review three of them
Automate documentation and script
generation. Expand test automation
capabilities to less technical resources.
Edit Automated Scripts. Rapid new UI
enhancements to create, manage, and
organize test sessions with ability to
edit element target values.
Accelerate Test Automation. Instantly
create test automation scripts and plug
them into your Selenium & Protractor
automation frameworks.
Modern Test Management tools: let’s review three of them
Kanban / Scrum Board Planning
Supports test planning and tracking
when testing involves a
combination of manual, exploratory
and automated testing in DevOps
cycles.
CI / CD Focused Testing
Tight integration with ALMs (JIRA)
and SVNs (GiHub)
Pipeline tools trigger test item state
changes and vice-versa
Test activity and results visible in
pipeline tools
Test Markdown for manual test
cases stored in GitHub
Modern Test Management tools: let’s review three of them
The add-on for JIRA that allows to focus the tests towards the end user
Implements BDD as part of the development model, importing and creating test scenarios.
What are key factors when thinking about migration?
Ideally, automated migration processes
Data preserved and properly mapped
Minimizing down time
Softening learning curve
Providing ad-hoc training
Coach sessions
Remote support
Transitioning our testing suites from the Waterfall Era
Transitioning our testing suites from the Waterfall Era
How can I migrate from my old ways of working?
Only CSV accepted
You will have to export data from your tool and adapt it to this CSV
format
Costly process: mostly needed ad-hoc configuration
Transitioning our testing suites from the Waterfall Era
How can I migrate from my old ways of working?
Zephyr for JIRA Importer Utility
Open source, published in Bitbucket.org
Previous export to Excel is needed, a lot of legacy tools support
this
Problem is you will have to properly customize this Excel file before
importing into Zephyr
Costly process: mostly needed ad-hoc configuration
How can I migrate from my old ways of working?
Excel Import / Export: ability to move off of spreadsheets, plugins,
or other products onto qTest through importing capability
HP Migration Tool: a utility designed by QASymphony to support
enterprises moving off of Quality Center onto qTest. Data and
artifacts from 40+ companies have been migrated
Transitioning our testing suites from the Waterfall Era
LinkagesRequirements Test Cases /
Run history
Defects Users Fields / Values
About
We are a Spanish IT company with more than 900 employees.
We are organized in 18 Centres of Excellence, QA&Testing is one of them.
We offer the following services:
Consultancy services in Agile Testing and DevOps
Functional, performance and automation testing
Quality software audits.
Installation of qTest suite. Migration from HP Quality Center
Migration from ChecKing QA to Kiuwan
Training
www.atsistemas.com
MadridC/Valle de Alcudia.3 Edificio 2,
planta 1. 28232. Las Rozas, Madrid
BarcelonaPlaça de Catalunya, 21 - 2ª
08002, Barcelona
CádizEdificio Jerez Parque Empresarial, Calle del Desarrollo 2; oficina 12,
planta 1, 11047, Jerez de la Frontera, Cádiz
A CoruñaEdificio Mans, Polígono de Pocomaco,
parcela D22, 15190 A Coruña
0034 902 888 902
Palma de MallorcaRegus Palma, Gremi de Sabaters, 21,
Polígono de Son Castello 07009 Palma
ZaragozaCentro Tecnológico TIC XXI C/Bari, 57
Plataforma Logística (PLA-ZA),50197, Zaragoza
THANK YOU