How to detect smelly acceptance tests and why they are wrong!

Kishen Simbhoedatpanday

ksimbhoedatpanday@xebia.com @KishenPanday

What is ATDD?

Do you think the Business cares about Unit Testing? I don’t think so

Unit Tests Facilitating the team with Design & Fast Feedback

Getting lost in details. Not meeting customers’ expectations

We need something that gives us the big picture. Acceptance tests to the rescue.

Acceptance Tests should tell us which features are fault

What do you think?

Acceptance Tests They can do so much more!

We stop treating them as Tests

We treat them as Specifications

Specifications By Example

!   Describe the solution in non-technical terms

!   Define Acceptance Criteria before development

!   Quality increases when everyone understands

My thoughts at work

Developers and Acceptance Testing

Testers and Acceptance Testing

Business and Acceptance Testing

What do you think?

It stinks!

FUN! Love and listen to your specification

Specifications that matter Understandable for everyone

Why?

!   Check common understanding

!   Get rid of assumptions

!   You’ll know when you’re done

Stop talking abstract terms Get into detail as soon as you can!

Good acceptance tests have these properties

!   Explicitly defined

!   Observable

!   Realistic scenario

!   Related to user story

!   Observed instantly

Example

As an Account Holder I want to withdraw cash from an ATM So that I can get money when the bank is closed

Explicitly defined?

Given a customer with ID "12" in front of ATM And the machine contains "100” And the user places card into ATM And the user gets "Please enter PIN” And the user enters PIN "1234” And the system verifies correct PIN And the user selects "cash withdrawal” And the user selects "Enter amount” And the user types "20” And the user selects "OK”

Explicitly defined?

When the user gets "Please wait” And the user waits until no "Please wait” And the user gets "Please remove card” And the user removes card from ATM And the user gets "20” And the user gets "Please remove cash” And the user removes cash from ATM Then the machine contains "80” And the account balance should be "80"

Observable?

Given a customer with ID "12" in front of ATM And the machine contains "100” When the user places card into ATM And the user enters PIN "1234” And the user selects "cash withdrawal” And the user selects "Enter amount” Then the user types "20” And the user selects "OK”

Realistic scenario?

Given a customer with ID "12" in front of ATM When the user enters amount “-123” Then the user gets ”Not a valid amount”

Related to user story?

Given a customer with ID "12" in front of ATM And transfers “100” from savings to withdrawal account When the user enters amount “100” Then the user gets ”100”

Observed instantly?

Given a customer with ID "12" in front of ATM And the machine contains "100” And the user places card into ATM And the user gets "Please enter PIN” And the user enters PIN "1234” And the system verifies correct PIN And the user selects "cash withdrawal” And the user selects "Enter amount” And the user types "20” And the user selects "OK” Then screen show correctly

When writing tests or specifications Think about Heidi Klum who needs to fix it tomorrow

Transforming tests into specifications

Business

Workflow

Technical

Cla

rity

Stab

ility

Less smelly

The breakdown

Withdraw Money

ATM dispensing

Account Balance

The split

Layers of Abstraction

Business

Workflow

Technical

JVM

Specifications that matter

Questions?

!   Know upfront what to “accept” as done

!   Check common understanding

!   SMART

!   Layers of Abstractions

Smelly Acceptance Tests More examples

Lost in UI details Watch out!

End – to – end testing Is not bad in itself, it’s just hard

Slow tests

Test Suites > 15 minutes are not allowed!

Huge Test Data

Huge Test Data

Huge Test Data

UI, API Huge Test Data goes here!

Create User Create Transfer Account Create products > 200 euro Order products

Given a Customer has a transfer account When a user orders funds of total price > 200 EUR Then transfer is accepted

Huge Test Data

Business

Workflow

Technical

Erratic Tests

Object

Property X: 10

Test 1

Test 2

Test 3

Read X: 10

Write X : 12

Read X : 10

Regression Bug Suite Why do you have them in the first place?

How to detect smelly Acceptance Tests

Questions?

!   Not SMART

!   Slow - End-to-End

!   Irrelevant Details

!   Edge Cases

!   Erratic Tests

!   Shared mutable objects

!   Regression checks

!   UI details

Why bother? Why care about smelly acceptance tests?

Keep the team together

Unstable tests - There is nothing worse No one likes to work with them

Tools, Tools and more Tools

Success is not in tools It’s in collaboration Select a tool that facilitates maximum collaboration

De afbeelding kan niet worden weergegeven. Mogelijk is er onvoldoende geheugen beschikbaar om de afbeelding te openen of is de afbeelding beschadigd. Start de computer opnieuw op en open het bestand opnieuw. Als de afbeelding nog steeds wordt voorgesteld door een rode X, kunt u de afbeelding verwijderen en opnieuw invoegen.

Wat Rong Khun, Chiang Rai, Thailand

Thank you! Please evaluate my presentation

See evaluation booklets in conference bag

Kishen Simbhoedatpanday

ksimbhoedatpanday@xebia.com @KishenPanday

Xebium = Fitnesse + Selenium The Love & Hate

Xebium & Layers of Abstraction

Fitnesse

Other Fixtures

API

DB

Xebium Selenium

Business level Workflow level

Technical level

Too technical level