developer tests - things to know (vilnius jug)

Developer TestsThings to Know

Vaidas Pilkauskas 2014 Vilnius JUG

[email protected]

meMy hobbies● developer at Wix.com● main language - Scala● main professional interest - developer communities

If there is time left after my hobbies● mountain bicycle rider, snowboarder ● consumer of rock music, contemporary art, etc

me - how to contact meconnect with me on LinkedIn http://lt.linkedin.com/pub/vaidas-pilkauskas/8/77/863/

add me on G+ https://www.google.com/+VaidasPilkauskas

follow on Twitter @liucijus

http://lt.linkedin.com/pub/vaidas-pilkauskas/8/77/863/



https://www.google.com/+VaidasPilkauskas

“We couldn’t understand why people without technical knowledge had to tell programmers “what” to do and, furthermore, they had to supervise “how” programmers did it.”

Cristian Rennellahttp://qz.com/260846/why-our-startup-has-no-bosses-no-office-and-a-four-day-work-week/

What this talk is about

● Things we argue about during code reviews● Things that took me time to understand and

prove that they are actually good way to go● Small things we have no time to discuss

during big talks

“Legacy Code is code without Tests”

Michael FeathersWorking Effectively with Legacy Code

So what is test?

It is system’s exercise under predefined conditions and then verification of an expected outcome.

Thing #1Test structure

Test phases

SetupExercise

VerifyTeardown

Test phases in code Server server = new NotesServer(); // setup

Note note = new Note("test note"); // setup

Status status = server.add(note); // exercise

assertEquals(SUCCESS, status); // verify

server.shutdown(); // teardown

Start everything in one method@Test

public void serverShouldAddNoteSuccessfully() {

Server server = new NotesServer(); // setup

Note note = new Note("test note"); // setup




}

Refactor to lifecycle methods@Before public void before() {

server = new NotesServer(); // setup

note = new Note("test note"); // setup

}

@Test public void serverShouldAddNoteSuccessfully() {



}

@After public void after() {


}

DON’Ts - #1

Do not start from setup/teardown

DON’Ts - #2

Do not mix setup code from different tests - shared code must be relevant to

the tests that use it

DON’Ts - #3

Setup and teardown are there to solve DRY problem and to help structure your

test code

How many colors are there in the picture?

Source: Josef Albers Interaction of Color

Context matters!

Source: Josef Albers Interaction of Color

Typical setup inside @Before@Before public void before() {



}




}

@After public void after() { server.shutdown(); // teardown }

Setup pollution - #1

● Every line in setup must be relevant to all tests.


● It is tempting to add additional setup tuning just to fix/enhance one test.


● Rich setup makes tests slow!


● Setup is to bootstrap your SUT.


● Setup hides details! Do not hide test preconditions.

Setting up “job” inside test method@Before public void before() {


}





}

But then be more specific@Test

public void serverShouldAddSingleLineNoteSuccesfully() {

// * set up which is actual for the current method

// * use scope specific name

Note singleLineNote = new Note("test note"); // setup

Status status = server.add(singleLineNote); // exercise


}

Give good names to setup methods

@Before public void createNotesServer() { server = new NotesServer(); }

Summary of test code organization

● DRY principle.● Readability. BDD vs. DRY● Consistency. Maintain the same style across

your codebase. ● Complexity. It may dictate the way you go.

Refactoring

Refactoring is about improving the design of existing code. It is the process of changing a software system in such a way that it does not alter the external behavior of the code, yet improves its internal structure.

Martin FowlerRefactoring: Improving the Design of Existing Code

Thing #2What do we test?

Test behaviour not methods

● Think of a contract


● Think of a contract ● And responsibilities


● Think of a contract ● And responsibilities● Specify requirements as tests


● Think of a contract ● And responsibilities● Specify requirements as tests● Happens naturally when done in test-first

approach

Thing #3Matchers

Matchers

● Enhance readability● Asserts on the right level of abstraction● Encapsulate testing logic● Reusable● Detailed match error messages (do not

leave them out in your custom matchers!)

Matcher libraries

● Hamcrest - standard matcher lib for JUnit● AssertJ - fluent assertions (IDE friendly)

● Provides common matchers● You can write your own custom matchers

HamcrestassertThat(frodo.getName(), equalTo("Frodo"));

assertThat(frodo.getName(), is(equalTo("Frodo")));

assertThat(frodo.getName(), is("Frodo"));

AssertJassertThat(frodo.getName()).isEqualTo("Frodo");

assertThat(frodo).isNotEqualTo(sauron).isIn(fellowshipOfTheRing);

assertThat(sauron).isNotIn(fellowshipOfTheRing);

Thing #4Custom matchers

Custom matchers

Are matchers that we write specifically for our projects.

Custom matchers

● Help communicate test intention● Abstract assertion logic in case standard

matchers are not enough● Are reusable and save time in large projects● You may have a custom message to be

more specific about test failure

Custom matchers@Test

public void shouldHaveIsbnGenerated() {

Book book = new Book(1l, "5555", "A book");

assertThat(book, hasIsbn("1234"));}

Thing #5Failing a test

fail()

In some cases (e.g. testing exceptions) you may want to force test to fail if some expected situation does not happen

fail()

try { // do stuff... fail("Exception not thrown");} catch(Exception e){ assertTrue(e.hasSomeFlag());}

fail()

● Fundamentally not bad, but better use matchers for expected failure

● Matchers help to clarify test intention● Don’t forget - expected behaviour is an

opposite of a failing test

Thing #6Anti-pattern: The Ugly Mirror

Anti-pattern: The Ugly Mirror@Test

public void personToStringShouldIncludeNameAndSurname() {

Person person = new Person("Vilkas", "Pilkas");

String expected =

"Person[" + person.getName() + " " + person.getSurname() + "]"

assertEquals(expected, person.toString());

}

Anti-pattern: The Ugly Mirror@Test

public void personToStringShouldIncludeNameAndSurname() {

Person person = new Person("Vilkas", "Pilkas");

assertEquals("Person[Vilkas Pilkas]", person.toString());

}

Thing #7How to turn off the test?

Why would you want to turn off the test?

Why would you want to turn off the test?

Well, because it fails… :)

Ignoring tests

● Always use ignore/pending API from your test library (JUnit @Ignore)

Ignoring tests


● Do not comment out or false assert your test

Ignoring tests


● Do not comment out or false assert your test● If you do not need a test - delete it

Thing #8What to do with exceptions?

Exceptions

● If you can, use matchers instead of○ @Test(expected=?)○ try-catch approach

JUnit expected exception@Test(expected=IndexOutOfBoundsException.class)public void shouldThrowIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0);}

//matcher in Specs2 (Scala)

server.process(None) must throwA[NothingToProccess]

try and catch

public void shouldThrowIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList();

try { Object o = emptyList.get(0);

fail("Should throw IndexOutOfBoundsException");

} catch(IndexOutOfBoundsException e)){

//consider asserting message!

}}

Exceptions

● catch-exception lib

catch-exception libList myList = new ArrayList();

catchException(myList).get(1);

assertThat(caughtException(),

allOf(

is(IndexOutOfBoundsException.class),

hasMessage("Index: 1, Size: 0"),

hasNoCause()

)

);

Exceptions

● What about ExpectedException Rule?○ My personal opinion - not that intuitive○ breaks arrange/act/assert flow

ExpectedException rule@Rule public ExpectedException exception = ExpectedException.none();

@Testpublic void testExpectedException() { exception.expect(IllegalArgumentException.class); exception.expectMessage(containsString('Invalid age')); new Person('Vilkas', -1);}

//Person constructor

public Person(String name, int age) { if (age <= 0) throw new IllegalArgumentException('Invalid age:' + age);

// ...

}

Thing #9Testing with time

Problempublic class MyService {

...

public void process(LocalDate date) { if (date.isBefore(LocalDate.now()) { ... } }

}

Testing with Time

● Design your system where time is a collaborator

● Inject test specific time provider in your test○ constant time○ slow time○ boundary cases time

Control time with Clockpublic class MyService { private Clock clock; // dependency inject

...

public void process(LocalDate date) { if (date.isBefore(LocalDate.now(clock)) { ... } }

}

Thing #10Collections

Collections - multiple properties to assert

● Is null?● Size● Order● Content

Collections● Most of the time you want to assert on collection content

Collections● Most of the time you want to assert on collection content● Prefer exact content matching

Collections● Most of the time you want to assert on collection content● Prefer exact content matching● Avoid incomplete assertions

Collections● Most of the time you want to assert on collection content● Prefer exact content matching● Avoid incomplete assertions● Do not sort just because it is easier to assert!

Collections● Most of the time you want to assert on collection content● Prefer exact content matching● Avoid incomplete assertions● Do not sort just because it is easier to assert!● Multiple assertions are worse than single content

assertion


assertion● Unless you want to say something important in your test!


assertion● Unless you want to say something important in your test!● Use matchers!

Thing #11Random values

Random values in tests

● Most of the time you do not want it


● Most of the time you do not want it● Unless you depend on randomness a lot (eg.

password generation*)

*Thanks to Aleksandar Tomovski for a good example


● Most of the time you do not want it● Unless you depend on randomness a lot● Use property based testing (which is also

hard)


● Most of the time you do not want it● Unless you depend on randomness a lot● Use property based testing (which is also

hard)● Do not make dummy values random

What if we still need random cases?

Generate Multiple Test Cases

● Quality over quantity


● Quality over quantity● Think of boundary cases, that you may want

to detect with random test



to detect with random test● Use parameterized tests



to detect with random test● Use parameterized tests● Random is hard to repeat



to detect with random test● Use parameterized tests● Random is hard to repeat● Flickering tests

Thing #12How many assertions per test?

How many assertions per test?

● Unit test - one assertion per test. Must be clear and readable

● Proper test should fail for exactly one reason● End to end - best case one assertion per

test, but more are allowed● Consider custom matchers

How many assertions per test?

● Unit test - one assertion per test. Must be clear and readable

● Proper test should fail for exactly one reason

● End to end - best case one assertion per test, but more are allowed

● Consider custom matchers

Thing #13Decoupling in End-to-end tests

What can be better in this test [pseudocode]

@Test shouldRetrieveUserByLogin() {

String userJson = "{\"username\": \"vaidas\"}";

HttpRequest post = new Post("https://localhost:8080/users", userJson);

HttpResponse postResp = HttpClient().execute(post);

assertThat(postResp.status, is(200));

HttpRequest get = new Get("https://localhost:8080/users/vaidas");

HttpResponse getResp = HttpClient().execute(get);

User user = mapper.readValue(getResp, User.class);

assertThat(username, is("vaidas"));

}

Decoupling from low level details [pseudocode]

@Test shouldRetrieveUserByUsername() {

CreateUserResponse createResp = aCreateUserRequest().withUsername("vaidas").execute();

assertThat(createResp, hasStatus(OK));

GetUserResponse getResp = aGetUserRequest().withUsername("vaidas").execute();

assertThat(getResp, allOf(hasStatus(OK), hasUsername("vaidas")));

}

Thoughts on end-to-end testing

● What is real e2e? REST vs. UI● Testing by using system itself ● Vs. using low level DB driver to verify data

persistence

Thing #14Comments

Comments in Test code

● Fundamentally good option to explain complicated parts, but:

● better use good method naming● custom matcher● do less, so that intention is clear● comments are not so bad in isolated well

named methods

Summary

● Context matters● Tests are controversial topic● And very opinionated

Thanks!Q & A

developer tests - things to know (vilnius jug)

Engineering

setup status status

setup code

setup pollution

code server server

new notesserver setup

rich setup

additional setup

setup hides details