testing and software writer a year later
TRANSCRIPT
![Page 1: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/1.jpg)
RAILSCLUB 2015
Testingand Software Writer a year later
Simon BagreevHead of Ruby development
Rambler&Co
![Page 2: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/2.jpg)
RAILSCLUB 2015
whoami
status_200
semmin
Simon Bagreev
![Page 3: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/3.jpg)
RAILSCLUB 2015
US
![Page 4: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/4.jpg)
RAILSCLUB 2015
RailsConf 2014
![Page 5: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/5.jpg)
RAILSCLUB 2015
TDD
● doesn’t test anything
● complicates design
● holy trinity (speed, coverage, ratio)
● adds cost, not value
![Page 6: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/6.jpg)
RAILSCLUB 2015
Software Writer
● does the whole thing work?
● testing at the higher level (system tests)
● adequate testing
![Page 7: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/7.jpg)
RAILSCLUB 2015
More problems
● no testing
● false confidence
● holy wars
● ignored failing tests
● endless test suite (due to units)
● “virtual” QA department
![Page 8: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/8.jpg)
RAILSCLUB 2015
No testing
● no proof that features
work
● no way to catch
regressions
● impossible to maintain
![Page 9: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/9.jpg)
RAILSCLUB 2015
No testing - TODO
● start with user stories
● write acceptance specs
● move to system tests
● create “visual” tests
![Page 10: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/10.jpg)
RAILSCLUB 2015
False confidence
● many (unit) tests
● runs fast
● 100% coverage
![Page 11: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/11.jpg)
RAILSCLUB 2015
About testing
“Testing shows the presence, not the absence of bugs.”
Edsger W. Dijkstra
NATO Software Engineering Conference, 1969
![Page 12: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/12.jpg)
RAILSCLUB 2015
Value of unit testing
“Most software failures come from the interactions between objects rather than being a property of an object or method in isolation.”
“ Functional testing typically finds twice as many bugs as unit testing ”
James Coplien
Why Most Unit Testing is Waste
![Page 13: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/13.jpg)
RAILSCLUB 2015
Coverageclass Event
def initialize(started_at)
@started_at = started_at
end
def today?
(Time.now.beginning_of_day..Time.now.end_of_day).cover? @started_at
end
end
![Page 14: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/14.jpg)
RAILSCLUB 2015
100% coverageRSpec.describe Event do
subject { Event.new(start_time) }
context 'today' do
let(:start_time) { Time.now }
it 'todays event' do
expect(subject).to be_today
end
end
end
![Page 15: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/15.jpg)
RAILSCLUB 2015
100% coverage
no expectation
RSpec.describe Event do
subject { Event.new(start_time) }
context 'today' do
let(:start_time) { Time.now }
it 'todays event' do
subject.today?
end
end
end
![Page 16: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/16.jpg)
RAILSCLUB 2015
● mutation testing● higher level testing
● “Create system tests with good feature coverage (not code coverage)”
James Coplien
Why Most Unit Testing is Waste
False confidence TODO
![Page 17: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/17.jpg)
RAILSCLUB 2015
Holy wars
● RSpec vs MiniTest
● Capybara vs Cucumber
● AR fixtures vs FactoryGirl
● stubs vs mocks
● PhantomJS vs Selenium
● etc...
![Page 18: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/18.jpg)
RAILSCLUB 2015
Ignored failures
“Oh, you have to invoke Maven with this flag that turns off those tests — they are tests that no longer work!”
James Coplien
Why Most Unit Testing is Waste
![Page 19: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/19.jpg)
RAILSCLUB 2015
“Rotten” tests
“If you want to reduce your test mass, the number one thing you should do is look at the tests that have never failed in a year and consider throwing them away.”
James Coplien
Why Most Unit Testing is Waste
![Page 20: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/20.jpg)
RAILSCLUB 2015
rspec-rotten, FTW
“rotten” – haven’t changed status (failed/passed /pending) in extended period of time
https://github.com/rambler-digital-solutions/rspec-rotten
![Page 21: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/21.jpg)
RAILSCLUB 2015
rspec-rotten
● uses custom rspec formatter● subscribes to rspec notifications● after every test run saves stats into json file:
{
"id": "./spec/controllers/calendar_controller_spec.rb[1:1:1]",
"state": "failed",
"date": "2015-09-21T22:04:39.063+03:00",
"location": "./spec/controllers/calendar_controller_spec.rb:13",
"description": "render the :show template"
}
![Page 22: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/22.jpg)
RAILSCLUB 2015
rspec-rotten
Notifies you about “rotten” tests:
![Page 23: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/23.jpg)
RAILSCLUB 2015
Endless unit testsWeinberg’s Law of Decomposition
![Page 24: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/24.jpg)
RAILSCLUB 2015
Endless unit testsAcceptance test
![Page 25: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/25.jpg)
RAILSCLUB 2015
Endless unit testsUnit testing
![Page 26: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/26.jpg)
RAILSCLUB 2015
Endless unit tests
(Testing) is identifying and resolving problems that arise in the system as a whole rather than in individual components. Testing is done by groups, as an individual programmer lacks the total knowledge needed to test large sections of the system. The emphasis in testing should never be on individual modules.
Edsger W. Dijkstra
NATO Software Conference, 1969
![Page 27: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/27.jpg)
RAILSCLUB 2015
Additionally
● hard to maintain (tests are code too, may contain bugs)
● hard to change (tends to become “binary”)● may never occur in prod
![Page 28: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/28.jpg)
RAILSCLUB 2015
Many unit tests TODO
● mutation testing
● rspec-rotten
● “Software Writer’s” approach: “If this test fails,
what business requirement is compromised?”
![Page 29: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/29.jpg)
RAILSCLUB 2015
“Virtual” QA dept
“My unit tests pass, QA / PM / Product Owner will take care of the rest” ... Nope
another team
![Page 30: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/30.jpg)
RAILSCLUB 2015
“Virtual” QA TODO
● focus on acceptance specs (outside-in)
● “visual” tests for end-user regressions
![Page 31: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/31.jpg)
RAILSCLUB 2015
rspec-visual, FTW
"visual" testing with rspec via screenshot comparison
https://github.com/rambler-digital-solutions/rspec-visual
![Page 32: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/32.jpg)
RAILSCLUB 2015
rspec-visual step 1
# spec/features/visual/home_page_spec.rb
describe 'home page', type: :feature, visual: true do
it 'home_page' do |example|
visit '/'
take_screenshot(page, example)
should look_like example.description
end
end
Create “visual” test
![Page 33: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/33.jpg)
RAILSCLUB 2015
rspec-visual step 2
# spec/features/visual/home_page_spec.rb
describe 'home page', type: :feature, visual: true do
it 'home_page' do |example|
visit '/'
take_screenshot(page, example)
should look_like example.description
end
end
Take a screenshot
![Page 34: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/34.jpg)
RAILSCLUB 2015
rspec-visual step 3
# spec/features/visual/home_page_spec.rb
describe 'home page', type: :feature, visual: true do
it 'home_page' do |example|
visit '/'
take_screenshot(page, example)
should look_like example.description
end
end
Use look_like matcher to compare with “stable”
![Page 35: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/35.jpg)
RAILSCLUB 2015
rspec-visual step 3
# rspec/visual/.../look_like_matcher.rb
RSpec::Matchers.define :look_like do |expected|
# ... set file paths
match do |actual|
# ... copy to stable folder if not found
system "compare -metric PAE -subimage-search -dissimilarity-threshold 1 \
#{expected_image_file_path} #{actual_image_file_path} #{diff_file_path}"
end
# ... customize failure message
end
look_like matcher
![Page 36: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/36.jpg)
RAILSCLUB 2015
rspec-visual result
!=
![Page 37: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/37.jpg)
RAILSCLUB 2015
rspec-visual diff
![Page 38: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/38.jpg)
RAILSCLUB 2015
rspec-visual result
!=
![Page 39: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/39.jpg)
RAILSCLUB 2015
rspec-visual diff
![Page 40: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/40.jpg)
RAILSCLUB 2015
rspec-visual gotchas
● dependencies (Capybara, Poltergeist + PhantomJS, imagemagick)
● only supports Poltergeist● dynamic data● speed
![Page 41: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/41.jpg)
RAILSCLUB 2015
To conclude
As a Software Writer:
● focus on what really matters● throw away tests that have no business
value● do “visual” testing● use Thought-Driven Development
![Page 42: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/42.jpg)
RAILSCLUB 2015
Contacts
В группе компаний Rambler&Co всегда есть открытые вакансии для тех, кто
хочет профессионально расти и развиваться, занимаясь тем, что по-
настоящему нравится
www.rambler-co.ru/jobs
![Page 43: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/43.jpg)
RAILSCLUB 2015
Links
RailsConf 2014 Keynote: Writing Software by DHH: http://bit.ly/1LwrENX
Why Most Unit Testing is Waste, James Coplien: http://bit.ly/1kBMor1
Why Most Unit Testing is Waste part 2, James Coplien: http://bit.ly/1OuzrOc
NATO Software Engineering Conference 1969: http://bit.ly/1iwANKT
rspec-visual: https://github.com/rambler-digital-solutions/rspec-visual
rspec-rotten: https://github.com/rambler-digital-solutions/rspec-rotten
![Page 44: Testing and Software Writer a year later](https://reader036.vdocuments.us/reader036/viewer/2022062503/58ee36351a28abd6518b4643/html5/thumbnails/44.jpg)
RAILSCLUB 2015
Thanks
Questions?