continuous testing meets the classroom at code.org
TRANSCRIPT
Continuous Testing Meets the Classroom
bcjordan.com
[email protected]@bcjordan
Brian Jordan, software engineer at Code.org
Code.org
non-profit
expanding participation in CS
Hour of Code
200 million tried Hour of Code
now what?
full curricula, district partnerships,
professional development, policy change
K-12 CS Curriculum
20k K-12 teachers trained
intro courses: 260k teachers 8mil students
how do you even start automated testing?
developing in 2013-2014
why test? what is Code.org's testing context?
the context
the context
the context
the context
the context
so how do we test all that?
how we started Selenium testing
how our tests are organized and run
selenium-webdriver cucumber
takeaways
Build a library of steps
Build a library of test levels
Build a library of annotations
@no_mobile
@no_ie9@skip
@db_access@as_student
who writes tests?
who/what runs tests?
what challenges did we end up facing?
challenge:multiple browsers
solution:selenium-webdriver
Sauce Labs
challenge:testing local changes
solution:chromedriver sauce-connect
challenge:
interaction, drag-and-drop
solution:jQuery Simulate
$(selector).drag…
challenge:handling
(intermittent) failures
solution:
always fixing!
solution:
always fixing!
(try rerunning!)
what causes intermittent failures?
Sauce Labs API test_flakiness.rb
what do they look like when running?
Speed Run!
(not) wasting time
speeding up test runs
2014: 1-2h of tests
parallelizationat runner level
2016: 20-30min of tests
justifying tests
the bug collection
Firefox getBBox()
my favorite bug ever
who investigates failures?
dev of the day
what's next?
per-PR tests
takeaways
start small
everybody tests
periodically invest in speed-ups
Q&A
bcjordan.com
[email protected]@bcjordan
Brian Jordan, software engineer at Code.org
code.org/help
github.com/code-dot-org