20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting

Beyond the urban legend

–

Testing in an agile contextJochim Van Dorpe

@ Smals_ICT#DV14 # AgileContextTesting

In-house ICT shared services for e-government

• Focus on social security and e-health services

• For federal, regional, local & European institutions

• Based in Brussels

• Software development, ICT-operations & staffing

• >1700 people

Introduction: What is Smals?

@ Smals_ICT#DV14 # AgileContextTesting

Introduction: Who Am I?

(QA) ( (lead) (technical) test(er)

analyst/coördinator/manager) (automator/engineer)

I am a tester

@ Smals_ICT#DV14 # AgileContextTesting

Introduction: What do I have to tell?

@ Smals_ICT#DV14 # AgileContextTesting

What is testing?

It's not about proving that the software works, it's about finding bugs

@ Smals_ICT#DV14 # AgileContextTesting

Why is testing necessary?

To prevent:

• Death

• (Serious) injury

• Loss of business

• Loss of reputation

@ Smals_ICT#DV14 # AgileContextTesting

Why do we test?

@ Smals_ICT#DV14 # AgileContextTesting

The test project in waterfall

@ Smals_ICT#DV14 # AgileContextTesting

Agility

Agility is the continuous

delivery of prosperity (value) to

stakeholders of a system in a

sustainable and balanced

manner -Schalk Cronjé (@ysb33r)-

@ Smals_ICT#DV14 # AgileContextTesting

Agile fundamentalists are like …

Did you even read

our manifesto?

@ Smals_ICT#DV14 # AgileContextTesting

Agile manifestoOf course …

• Individuals & interactions over

processes and tools

• Working software over

comprehensive documentation

• Customer collaboration over

contract negotiation

• Responding to change over

following a plan

@ Smals_ICT#DV14 # AgileContextTesting

Urban legends

• Agile doesn’t need testers

• Agile = de facto good code, built-in quality

• Testing is dead

• Agile = TDD, ATDD, BDD

• Testers should be able to code

• Agile hasn’t time for testing

• 100% automation

• Faster, shorter, better!

• Developers and Testers are like oil and water

• You only need to unit test

• User Acceptance Testing is no longer necessary

@ Smals_ICT#DV14 # AgileContextTesting

So…

• The Agile manifesto shouldn’t stand between you &

good quality

• No documentation, few documentation or poor

documentation is no excuse for “not testing”

• Don’t put an Agile stamp on your team, just to avoid

the things you dislike, but find a way of doing them

so you like them …

@ Smals_ICT#DV14 # AgileContextTesting

No difference at all?

@ Smals_ICT#DV14 # AgileContextTesting

Many ways to be agile …

@ Smals_ICT#DV14 # AgileContextTesting

… but I’m no salesman

@ Smals_ICT#DV14 # AgileContextTesting

Principles of testing in an agile

context

-Elisabeth Hendrickson (@testobsessed)-

@ Smals_ICT#DV14 # AgileContextTesting

1. Testing moves the project forward

Shifting focus:

No longer the last line of defense

@ Smals_ICT#DV14 # AgileContextTesting

2. Testing is not a phase

Integrated & continuous:

develop & test together

@ Smals_ICT#DV14 # AgileContextTesting

3. QA = responsability of the team

@ Smals_ICT#DV14 # AgileContextTesting

4. Reduce feedback latency

No long gaps between development

& testing feedback

@ Smals_ICT#DV14 # AgileContextTesting

5. Don’t let bugs crawl around

Deal with bugs as soon as possible

@ Smals_ICT#DV14 # AgileContextTesting

6. Test should represent requirements

Don’t ASSUME, it makes an ASS of

U & ME

SCHOOLBUS

@ Smals_ICT#DV14 # AgileContextTesting

7. Reduce overhead

Think high level: don’t waste time on

too much detail

@ Smals_ICT#DV14 # AgileContextTesting

8. Don’t believe in fairy tales

If something

seems to good to be true, it probably is

@ Smals_ICT#DV14 # AgileContextTesting

9. Done means done

Done means implemented and tested

@ Smals_ICT#DV14 # AgileContextTesting

Concrete practices

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate More

• The testing triangle: This will drown your testers:

Manual tests

Acceptance tests

System &

integration tests

Comp.

tests

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate More

• The testing triangle: Make sure your testers can

cope with the pace

Component tests

System &

Integration tests

Acc.

Tests

Manual

testsManual tests

Acceptance tests

System &

integration tests

Comp.

tests

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate Testing, not only tests

Test

execution

ch

ec

kin

g

Load

Data-

sets

Reset

DB

Continuous integration

Logging

resultsGenerate

reports

Live

dashboards

Calculating

‘metrics’

Use the tools that fit your purpose

AND NOT THE OTHER WAY AROUND!

@ Smals_ICT#DV14 # AgileContextTesting

B. Explore

Leave some room for

exploratory testing

@ Smals_ICT#DV14 # AgileContextTesting

C. Diversify

• …in the test levels

• Unit-tests only won’t find

everything

• System test only, and you’ll

drown

• … in the way of testing

• Automation is good, automation

is fun, but in the end, an

automated test isn’t better than a

human based test (manual test)

@ Smals_ICT#DV14 # AgileContextTesting

D. Test the tests

• GREEN == GOOD ?

• DO: Review tests

• DONT: No test coverage fairy tales

• DONT: Mock lasagna

• DONT: I should adapt the test cases so it’s green again

@ Smals_ICT#DV14 # AgileContextTesting

E. Crowdsource …

• Crowdsource in your team to find

the good & passionate tester

• Accept that there are learning curves

• Testing is a craftmanship

• A passionate tester will tend to be more

effective than the chinese-volunteer-

tester

• Encourage collaborative testing

@ Smals_ICT#DV14 # AgileContextTesting

… or ask a testjumper

• Ask a ‘test jumper’ to join the

team

• May spend some hours as a consultant

• Or months as an ordinary tester

• Coaches or contributes himself

• Encourages testability

• Help the developers think

productively about testing

@ Smals_ICT#DV14 # AgileContextTesting

The test project in agile

@ Smals_ICT#DV14 # AgileContextTesting

What should the PL not expect?

• Metrics such as:

• # of bugs found

• Bugs per functionality

• Bugs per 1000 lines of code

• DDP (defect detection percentage)

• We can however show our QA-efforts

• QA-survey

• Test case progress

• Fixed test cases

@ Smals_ICT#DV14 # AgileContextTesting

What should the PL not expect?

• Qualification (good / bad) of the devs

• 100% green charts

• Upfront fixed number of test cases

• Automated self-generating auto-code-correcting

tests

@ Smals_ICT#DV14 # AgileContextTesting

conclusion

There are no agile testers,

but we need testers who

adapted their mindset to an

agile context

@ Smals_ICT#DV14 # AgileContextTesting

Read & experience more 1/2• Books:

• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and

Agile Teams

• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the

Whole Team

• Blogs:

• Huibschoots.nl

• LisaCrispin.com

• http://testobsessed.com : Agile testing overview

• Http://www.satisfice.com : Test jumpers: one vision of Agile testing

• pascaldufour.wordpress.com

• agile-and-testing.chriss-baumann.de

@ Smals_ICT#DV14 # AgileContextTesting

Read & experience more 2/2• Presentations:

• 40 agile methods in 40 minutes by Craig Smith

• Agile testing quadrants explained by @RubyTester

• Certification:

• CAT-training: Certified Agile Tester

• ISTQB foundation level add-on: Agile tester

• Conferences:

• Agile testing day(s)

• More references:

• Word clouds generated on wordle.net

• Images found with google image search

@ Smals_ICT#DV14 # AgileContextTesting

Questions