living with acceptance tests: beyond write-once (xp nyc)
TRANSCRIPT
Living with
Acceptance Tests:
Beyond Write-Once
Daniel Wellman
Twitter: @wellman
E-mail: [email protected]
XP NYC #xpnycFebruary 10, 2015
Activity
Goal
Build the least helpful,
most frustrating
acceptance test suite
Write-Once
Tests
Revise Your Tests
With the Same Care as
Your Production Code
Check for Duplication
Refactor with
New Insights
Monitor for Problems
Problem:
Slow and Flaky Tests
Server
Database QueueTest
End-to-End Tests
New Project
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance Test
End-to-End
Acceptance TestEnd-to-End
Acceptance Test
End-to-End
Acceptance TestEnd-to-End
Acceptance Test
End-to-End
Acceptance Test
How did it get this
way?
NoAlternatives
Institutionalized
Response to
Failure
Tests
Explain
User Interface
Run Acceptance
Tests in Parallel
Faster Build
Same Risk
Failures are
Hard to Diagnose
Problem:
Flaky Tests
Don’t make every
acceptance test run
end-to-end
Ports and Adapters
a.k.a. Hexagonal Architecture
Adapters
Ports
Domain
GUI
DB
Memory
End-to-End Test
Test Using
Domain
Example: Fraud Protection
for an Amazon-like store
Example from Gojko Adzic, “Specification by Example”
Given a user with no previous transaction history,
And the user’s account registration country is the UK,
When the user places an order with delivery country U.S.,
Then the transaction is marked as suspicious,
But the user sees order status as “Pending.”
Before: End-to-EndWhen(/^the user places an order with
delivery country (.*),$/) do |country_name|
end
# ...
fill_in 'Address', with: ...
select country_name, from: 'Country Selector’
click_button 'Ship To This Address'
expect(page).to have_content
'Your order has been placed’
# ...
After: Using Domain Model
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
end
# ...
country = Country.new(country_name)
order = Order.new( ...,
ship_to : Address.new(
street: ...,
country: country))
sales_clerk.accept(order)
# ...
Service
UI
Unit
The Testing Pyramid
Create a Separate
System Test Suite
Alternate Step
Implementations
Alternate Steps:
Use Conditionals
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
if use_browser
# drive a browser using Capybara; slower
else
# use domain objects; faster
end
end
use_browser = ENV['USE_BROWSER_FOR_TESTS']
Write Tests
Without Specifying
Implementation Details
Imperative
vs.
Declarative
Style
Imperative Style
Given I am on the registration page,
And I enter "UK" as my billing country,
And I press the "Register" button,
And I add an item to my cart,
When I checkout
And I enter "U.S." as my shipping country,
And ...
Declarative Style
Given a user with no previous transaction history,
And the user’s account registration country is the UK,
When the user places an order with delivery country U.S.,
Then the transaction is marked as suspicious,
But the user sees order status as “Pending.”
Example by Gojko Adzic, “Specification by Example”
You can’t tell how these are implemented!
Use Declarative Style
to Help Keep
Implementation
Details Out
of Tests
Problem: I’m not sure
what is actually
tested…
How did it get this
way?
Periodically
Evaluate
Your
Test Suite
Have you introduced a
new domain concept?
Are you in a new stage
of your product’s life?
Suggestions
• Maintain your acceptance tests just like production
code
• Write tests which describe what, not how, so their
implementations can easily evolve later
• Decide when you can start moving acceptance
tests from end-to-end to using the domain directly
• Re-evaluate your product’s tests and testing
strategy as your system grows
Resources
Photo Credits“HighLine_061409_2779” by Jessica Sheridan is licensed under CC BY 2.0
“Highline New York” by Rebecca Krebs is licensed under CC BY 2.0
“NYC: The Highline" by Wally Gobetz is licensed under CC BY-NC-ND 2.0
“Rot”, “Yellow Cart”, and “Many Hands Make Light Work” by Oliver Rich are licensed
under CC BY-NC-ND 2.0
“highline-1010384.jpg” by keroism is licensed under CC BY-SA 2.0
“highline” by Elijah Porter is licensed under CC BY-NC-SA 2.0
“Highline_030” by Timothy Vogel is licensed under CC BY-NC 2.0
“201106073451_DSC_0014_1.JPG” by Leonel Ponce is licensed under CC BY-NC 2.0
“193” by verndogs is licensed under CC BY-NC 2.0
Living with
Acceptance Tests:
Beyond Write-Once
Daniel Wellman
Twitter: @wellman
E-mail: [email protected]
XP NYC #xpnycFebruary 10, 2015