Testing in a Scrum Team In this article I will discuss what types of testing I think should be done by a Scrum Team, and what types could be done by someone outside of the team. To be able to have this discussion, I will first need to define the different types of testing I intend to use. This model is an overview of the nomenclature I will use, and how I think the different types of testing should be divided between the Scrum Team and the rest of the organization.

Here are some quotes to better understand what the test types the Scrum Team is responsible for mean:

“An integrated test is any (low level) test where when it fails you cannot pinpoint exactly what went wrong.“ [1] [4] “Integration testing is the phase in software testing in which individual software modules are combined and tested as a group.” [4] “Contract Tests explain how a class should extend a superclass or implement an interface…” [1] [2]

Scrum Team Scrum Team

Scrum Team

Scrum Team

Outside of Scrum Team

Isolated Tests

Contract/Collaboration Tests

Integrated Tests

System Tests

Equipment & Competence Specific Tests

“Collaboration Tests checks if a component talks to the next layer correctly.” [1] [3] “Collaboration tests are also called interaction tests to differentiate them from state based tests.” [1] [3] “The real benefit of isolated tests, of tests for one function at the time, is that those tests put tremendous preassure on our designs.” [1][6] “…unit testing (isolated tests) is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use.” [6]

This way the responsibility of the Scrum Team is pretty clear. How they do it; if they do it with manual tests through the UI or using automated test, is up to the team. Though the nomenclature obviously encourages writing automated tests. The Scrum Team is responsible for the quality (shippable state [7]) of every component they develop, how they communicate with each other, and that those components work together as a group. But there are some exceptions, and also some additional testing that must be done before we can be confident that our test coverage is sufficiently mitigating all relevant risks. Let us start with the exceptions. If we have multiple Scrum Teams (if we don’t then the following could, dependent on context, be included in the Scrum Teams responsibility) then we may have tests that require certain tools or competence that is not readily available to all Scrum Teams. To be able to scale, it can be a reasonable approach to centralize these tests to a specific group of people that serve all the Scrum Teams. One of the most obvious and simple examples can be localization and culturalization [8] testing of features. Every Scrum Team cannot be expected to know all languages and cultures that the product will be shipped to, and thus will have problems testing these areas. There can be many tests with similar competence or tool dependencies that must be moved out of the Scrum Team, but exactly which these tests are is dependent on context. Let’s move on to what additional tests must be performed after the Scrum Team has done their part. If we have multiple Scrum Teams (if we don’t, then again this will fall on the responsibility of the Scrum Team) and they all deliver their perfectly (in theory) tested groups of components, then we might still end up in a situation where the system does not work. But if we put this work on the Scrum Team we will not only reduce their velocity, but also require them to test other teams’ groups of components when they integrate their code. This will cause ownership problems. In this case, one option is to have testers specialized in this kind of testing taking responsibility for testing the complete system. Each Scrum Team still has ownership of the quality of their code, and responsibility for fixing any problems found during this test, but the responsibility for testing the whole

system can be transferred to this group of professional testers. The complexity of this kind of system test also requires a different skillset than less complex low level testing, which makes it suitable to have testers with this skillset handle the task. I say that low level testing is less complex, but that in no way means that it is easier, or less valuable. With this nomenclature and division of different test types between the Scrum Teams and other roles, I believe that it makes it clear what responsibility the Scrum Teams have when it comes to testing, and what they can expect other roles to help them with. I am not claiming that my way is in any way better than other existing models, but it has helped me in my understanding of how to efficiently work with test in Scrum. /Johan Hoberg

References [1] Integration tests are a scam https://vimeo.com/80533536 http://www.infoq.com/presentations/integration-tests-scam [2] Contract tests http://thecodewhisperer.tumblr.com/post/1325859246/in-brief-contract-tests http://thecodewhisperer.tumblr.com/post/1172613515/stub-your-data-access-layer-it-wont-hurt http://sharkzp.github.io/blog/2014/01/20/contract-tests/ [3] Collaboration Tests (Interaction Tests) http://googletesting.blogspot.se/2013/03/testing-on-toilet-testing-state-vs.html [4] Integrated Tests / Integration Tests http://en.wikipedia.org/wiki/Integration_testing [5] System Tests http://en.wikipedia.org/wiki/System_testing [6] Unit Tests / Isolated Tests http://en.wikipedia.org/wiki/Unit_testing [7] Shippable State http://www.infoq.com/news/2008/02/done-shippable-quality [8] Localization vs. Culturalization http://www.hitcents.com/blog/post/localization-vs-culturalization