Agile Testing

Agile Ottawa April 2015

What is Testing?

●Testing seeks to discover threats to the value of

software

●Questioning a product in order to evaluate it

o James Bach

●Testing is done to find information

o With that information critical project decisions

can be made

What is Agile Testing?

Agile testing applies the agile mindset

● It suggests attitudes and activities that help to

deliver valuable software

Applies to both

●Testing on an agile team

●Using agile testing on a traditional team

The Agile Testing Mindset

● The whole team is responsible for quality

o ‘QA will find it’ isn’t accepted

● Testing is not a phase, it’s an ongoing activity

The Agile Testing Mindset

● Short feedback loops

● Continuous improvement

● Collaboration

o How can I help?

Why Bother?

● To fix defects as close to when they were

introduced as possible

● To keep the project moving forward

o By continually providing test feedback as the

product is developed

The Whole Team Approach

Separate Teams

Cross Functional Team

Agile Testers

Agile Testers

Principles for agile testers

● Provide continuous feedback

● Focus on delivering value

● Prioritize high bandwidth communication

Agile Testers

Need to be critical thinkers

● Challenge assumptions

Need to be problem solvers

● Fix problems, not symptoms

Need to be good communicators

● They are required to give feedback

Agile Testers

Need to learn the business domain

● Take the customer’s point of view

Need to think differently

● Ask the what if questions

Agile Testers and Feedback

Give feedback

● With empathy for the customer

● With empathy for the developers

Giving Feedback

● Have a positive intent

o You are trying to help people

● Take responsibility for the feedback

o Provide and gather context

● You must be clear

o Otherwise your feedback is ineffective

Receiving Feedback

● Assume a positive intent

o People want to do their best work

● Give people the benefit of the doubt

o People have bad days

● Thank people for constructive feedback

o You want to improve to do your best work

Feedback Skills for Agile Testers

● Active listening

o Listen to learn

o Don’t jump to conclusions

Feedback Steps from Jurgen Appelo

● Provide context

● List observations

● Express emotions

● Sort by value

● Offer suggestions

Agile Testers

Trust is like money

● It can take years to earn it

and it only takes minutes

to lose it

Agile Testers and Programming

It is important to be technically aware

● Take a holistic view of your technical environment

Everyone cannot be good at everything

● A technical context enhances collaboration

Agile Testers and Programming

So does this mean testers need to program?

● Definitely maybe

Test automation enables agile testing

● Agile testing is not sustainable without automation

Instead of the individual look at the team

● The team needs to be able to program

T Shaped Testers

The T represents depth of knowledge and breadth of

knowledge

TDe

pth

Breadth

T Shaped Testers

Depth is the tester’s specialized knowledge

● Integration testing

● Exploratory testing

● Etc

De

pth

Breadth

T

T Shaped Testers

Breadth is the tester’s generalized knowledge

● Programming

● Configuration management

● Databases

● Etc

De

pth

Breadth

T

Square Shaped Teams

Test Automation

Test Automation

You can apply agile skills to any test project

● But automation enables agile testing to succeed

Test Automation

Why?

● Manual testing takes a long time

● Manual testing is expensive

● You want to free your testers to do more valuable

work

Test Automation

More reasons

● It provides earlier feedback more often

● It is a form of documentation

● It enables Continuous Integration and Continuous

Delivery

Testing vs. Checking

Checking

● Things that machine could do

o Expected results are well defined

Testing

● Things that a human must do

o Observe

o Explore

o Infer

Testing vs. Checking

More

● Cost

● Time

● Fragile

More tests

Agile Test Quadrants

Exploratory Testing

Exploratory Testing

Exploratory testing

●Relies on a tester’s freedom

●Relies on a tester’s responsibility

What is it?

●Testing without expected results

Exploratory Testing

Exploratory testing is simultaneous

● Learning

● Test design

● And test execution

James Bach

Exploratory Testing

How do you do it if you don’t have an expected

result?

● Use test heuristics

● Use experience and domain knowledge

What is it not?

●Undisciplined

●Undocumented

Test heuristics?

● A ‘rule of thumb’ for testing software

o Using experience to find defects

Heuristics

Variable Analysis

● Identify anything whose value can change.

● Goldilocks

o Too Big, Too Small, Just Right

● Starvation

o CPU, Memory, Network, or Disk at maximum

capacity

Elisabeth Hendrickson - Test Heuristics Cheat Sheet

Heuristics Examples

I'm not schooled in the science of human factors,

but I suspect surprise is not an element of a robust

user interface

Chip Rosenthal

Exploratory Testing

Design

● Look for interesting test variations

Executing

● When you think of the test

Learning

● Discover behaviour, look for clues about defects

Steering

● Take those clues and exercise the code more

Exploratory Testing

Ideas

● Persona testing

● Test tours

Tools

● Notes

● Mind maps

● Atlassian Bonfire

● QTest eXplorer

SBE, and BDD

Shift Left in the SDLC

We want to test early and often

● How do we start to prevent defects even before

code is written?

Shift Left in the SDLC

Involve test throughout the Software Development

Lifecycle

Prevent defects

● By collaborating early and often

o Business, dev, and test

● By challenging requirements early

Build the Right Thing, Build it Right

Testers can help:

● Build the right thing

o That meets the customers needs

o Might not be what they asked for

● Build it right

o Well designed, coded, tested

A Build the Right Thing Pyramid

SBE and BDD

Specification by Example (SBE)

Behaviour Driven Development (BDD)

Both methods involve testers earlier in the SDLC

● To discover the right thing to build

Both methods use business language

● This allows Business, Dev, and Test to collaborate

SBE and BDD

Specification by Example (SBE) uses process

patterns to collaborate

●Defines scope from business goals

●Gathers key examples

●Refines examples

●Automates testing of the examples

SBE and BDD

Behaviour Driven Development (BDD)

● Defines tests first

● Captures examples of desired system behaviour

● Automates testing of those examples

SBE and BDD

People get distracted by automation tools

It isn’t about the tools

● It’s about the conversations

● It’s about gaining a shared understanding

Paraphrasing Liz Keogh

The Outside-In Approach

Friends build products

Enemies only build

documentation

Pragmatic Marketing

Stories and SBE, BDD

Stories are a lightweight method to frame desired

system behaviour

As a student

I want to register for a course

So that I can graduate

Who

What

Why

Stories

Note:

The story

● Can’t be coded yet

o It needs more detail

● It isn’t testable yet

Specification Workshops

You need to get the details, SBE and BDD suggest a

conversation

Discuss the story in a specification workshop to

● Understand how it should behave

● Discover enough detail to develop and verify

● Create shared understanding

o Business

o Dev

o Test

Who is in that workshop? Why?

Business

QADev

The solution we want

Examples

Refine the examples from the Specification

Workshop using the Given When Then format to

● Define a story’s scope

● Make the story testable

● Give you the story’s key details

Don’t create an example for everything

● You’ll get lost in the details

Example

GIVEN Java 101 has >= 1

places available

WHEN Sarah registers for Java

101

THEN Sarah is registered in

Java 101

Postcondition

Event

Precondition

Details documented

collaboratively

A Vacation Photo

Shared Understanding

The document is nothing,

but documenting is

everything

●Gerald Weinberg paraphrasing D. Eisenhower

Key Success Factors

●Use the whole team approach

●Adopt the agile testing mindset

●Automate regression tests

●Provide and get feedback

●Build the team’s core agile practices

●Collaborate with the business, and development

●Look at the big picture

The Outside-In Approach

Quality is about being

prepared for the usual so

you have time to tackle the

unusual.

● Adam Geras

Dag Rowe

● @dagrowe

● ca.linkedin.com/in/dagrowe

Thanks to Pythian for sponsoring the pizza, and

venue!

www.pythian.com

Sources

●Agile Testing: A Practical Guide for Testers and

Agile Teams

●More Agile Testing: Learning Journeys for the

Whole Team

●User Story Mapping: Discover the Whole Story,

Build the Right Product

●Specification by Example: How Successful Teams

Deliver the Right Software

●http://dannorth.net/introducing-bdd/

Sources

●http://watirmelon.com/tag/software-testing-

pyramid/

●http://www.duncannisbet.co.uk/test-automation-

basics-levels-pyramids-quadrants

●http://www.satisfice.com/blog/archives/category/te

sting-vs-checking

●https://mysoftwarequality.wordpress.com/tag/softw

are-quality/

●http://www.developsense.com/blog/2012/02/braidi

ng-the-stories/

Sources

●https://mysoftwarequality.wordpress.com/tag/softw

are-quality/

●http://www.skillsyouneed.com/ips/active-

listening.html

●http://lizkeogh.com/2011/09/22/conversational-

patterns-in-bdd/

●http://lisacrispin.com/downloads/AgileVancouverA

gileTestersDifferent.pdf

●http://www.slideshare.net/jurgenappelo/feedback-

wrap

Sources

●https://soundcloud.com/techwell/keep-agile-

testing-agile-an

●http://joecolantonio.com/testtalks/42-paul-gerrard-

shift-left-a-new-model-for-

testing/?utm_source=twitter&utm_medium=podcas

t&utm_campaign=shift_left_test_talks

●http://joecolantonio.com/testtalks/34-janet-gregory-

agile-testing/

●http://www.slideshare.net/ColomboCampsCommu

nity/what-is-an-agile-tester-henrik-kniberg