dial m for mutation

JavaZone 2016

www.slideshare.net/filipvanlaenen/dial-m-for-mutation

Filip van [email protected]@filipvanlaenen

Dial M for Mutation

https://www.quora.com/What-are-the-funniest-desire-path-examples-you-know/answer/Katie-Sonder5

https://www.reddit.com/r/funny/comments/35vomd/oh_the_irony/6

Hadi Hariri, “The Silver Bullet Syndrome”, JavaZone 20167

https://blog.codecentric.de/en/2016/02/sensible-mutation-testing-dont-go-killing-spree-2/

10

MUTATION TESTING

12

Mutation Testing

• Unit tests testing the source code

13

Mutation Testing


14

Mutation Testing


• Make a change to the source code

15

Mutation Testing


• Make a change to the source code• Failing unit test

16

Mutation Testing


• Make a change to the source code• Failing unit test• Many failing unit tests

17

Mutation Testing


• Make a change to the source code• Failing unit test• Many failing unit tests• Infinite loops, compilation

errors, run-time errors, etc…

18

Mutation Testing Book

• Free e-book

• Work in progress• leanpub.com/mutationtesting

19

Mutation Testing Tools

• PITest• Java• http://pitest.org/

• Mutant• Ruby• https://github.com/mbj/mutant

20

GETTERS, SETTERS,CONSTRUCTORS AND BUILDERS

21

Getters and Setters

• How difficult can it be?

• If I had a dime for every time…

22 https://www.flickr.com/photos/mussikatz/16015546324/

Detach from the Input

• Isolate inner state from input parameters

• Detach from input parameters• Arrays• Collections• Objects in general

23 https://www.flickr.com/photos/macieksz/503731332/

Detach the Output

• Don’t expose internal state• (Deep) copies• Unmodifiable collections

24 https://www.flickr.com/photos/kelloggphotography/2099315965/

Unit Testing Getters, Setters, Constructors and Builders

• Are getters, setters, constructors and builders wired together correctly?• Primitive types, but also objects

• Are setters, constructors and builders detaching correctly from input parameters?• Arrays, collections, objects

• Are getters returning (deep) copies or unmodifiable versions?

25

BOUNDARY CHECKS

26

Unit Testing a Square

• How many unit tests do you need?

27 https://www.flickr.com/photos/jasoncooper/13204428724/


• How many unit tests do you need?• 0?



• How many unit tests do you need?• 0?• 1?



• How many unit tests do you need?• 0?• 1?• 2?



• How many unit tests do you need?• 0?• 1?• 2?• 4?



• How many unit tests do you need?• 0?• 1?• 2?• 4?• 8?


[-1, 1] × [-1, 1]

34

[-1, 1] × [-1, 1]

KISS implementation:

boolean isInside(double x, double y) {

return -1 <= x && x <= 1 && -1 <= y && y <= 1;

}

35

[-1, 1] × [-1, 1]



return -1 <= x && x <= 1 && -1 <= y && y <= 1;

}

36

“Naive” Unit Tests

assertTrue(square.isInside(0, 0));

assertFalse(square.isInside(2, 2));

37




→ 9 out of 17 mutants killed

38





39





40

Adding a Corner Case




41

Adding a Corner Case




→ 11 (+2) out of 17 mutants killed

42

Adding Corner Cases




assertTrue(square.isInside(-1, -1));

43

Adding Corner Cases






44

Adding Corner Cases






45

Adding Corner Cases






46

Conditions in Isolation





assertFalse(square.isInside(-2, 0));

47








48








49









50









51









→ 16 (+1) out of 17 mutants killed52









assertFalse(square.isInside(0, -2));

53










54

“Naive” Unit Tests?









55

Useless Unit Tests









56

Useless Unit Tests









57

Useless Unit Tests









58

Six Unit Tests







59


• How many unit tests do you need?• 0?• 1?• 2?• 4?• 8?• 6



• How many unit tests do you need?• 0?• 1?• 2?• 4?• 8?• At least 6


[-1, 1] × [-1, 1]



return -1 <= x && x <= 1 && -1 <= y && y <= 1;

}

62

[-1, 1] × [-1, 1]

A mutant (currently) not generated by PIT:


return -1 <= x && Math.floor(x) <= 1 && -1 <= y && y <= 1;

}

63

[-1, 1] × [-1, 1]




}

64

[-1, 1] × [-1, 1]




}

65

[-1, 1] × [-1, 1]

And other mutants…

66

[-1, 1] × [-1, 1]

And other mutants…

…which may require domain knowledge to construct…

67

Binding the Square







68

Binding the Square



assertFalse(square.isInside(-1.01, 0));

assertFalse(square.isInside(0, 1.01));

assertFalse(square.isInside(1.01, 0));

assertFalse(square.isInside(0, -1.01));

69


• How many unit tests do you need?• 0?• 1?• 2?• 4?• 8?• At least 6


71

https://www.flickr.com/photos/jasoncooper/13204428724/

Notice the difference

CONDITIONALS

72

Testing Conditionals

• Involves three things:• Exercising the conditional• Running the conditional

clause• Verifying the functionality

of the conditional clause

73 https://www.flickr.com/photos/borkurdotnet/6797142125/

Testing Conditionals

• Involves three things:• Exercising the conditional• Running the conditional

clause• Verifying the functionality

of the conditional clause

If you can’t reach the clause, consider removing the code!74 https://www.flickr.com/photos/borkurdotnet/6797142125/

Mutating Conditionals

• Three ways to mutate a conditional:• Negate• Replace with false• Replace with true

75


• Three ways to mutate* a conditional:• Negate• Replace with false• Replace with true

* Or get it wrong

76


• Three ways to mutate* a conditional:• Negate• Replace with false• Replace with true

* Or get it wrong

77

PRESENCE AND NECESSITYOF EVERY EXPRESSION

78







79

Unused Code

• Missing unit tests?

• Code that can be deleted?

80

LOGS AND EXCEPTION MESSAGES

81

Testing Log Messages

• Why shouldn’t you?

• Ways to get log messages wrong:• Absent• Wrong log level• Wrong message content• Exceptions

82 https://www.flickr.com/photos/waltstoneburner/7946581522/

How to Test Logging

• Create an in-memory appender

• Set up the condition for the message to be logged

• Check for:• Its presence• Log level• Message content

83 https://www.flickr.com/photos/waltstoneburner/7946581522/

How to Test Exceptions

• Set up the condition for the exception to be thrown

• Check for:• Its presence• Exception type• Message content

84 https://www.flickr.com/photos/7304439@N06/1453921573/

CONSTANTS

85

“

86

Πάντα ε ῥ ῖ – Everything flows

https://en.wikipedia.org/wiki/Heraclitus

Constant Constants

• Some constants are pretty constant:• 0, 1, 2, e, π, c, …

• Others may change:• 18 (age of majority)• 16, 21, 25, …

• Or are magic numbers :• 42, population sizes, …

87 https://www.flickr.com/photos/theilr/16621361510/

Reuse or Redefine?

• Constant constants:• Reuse

• Changing constants:• Redefine

• Magic numbers:• Redefine• Double-entry

bookkeeping88 https://www.flickr.com/photos/52814185@N02/6643480765/

Unit Testing Constants

• Constant constants:• Implicitly

• Changing constants:• Implicitly

• Magic numbers:• Explicitly

89 https://www.flickr.com/photos/quasimondo/97503572

ACCESS MODIFIERS

90

Testing Public APIs

• Functionality ≠ public API

• Testing in the same package:• Common practice• JUnit, TestNG, …• Maven conventions• IDEs

• “Interface exhibitionism”91 https://www.flickr.com/photos/neesam/28674222645/

Testing Public APIs

• Test functionality in the same package• Use the default access

modifier when needed

• Test the public API from a different package• To figure out which

interfaces, classes and methods should be public

92 https://www.flickr.com/photos/neesam/28674222645/

Access Modifier Mutator

• I could wish for an access modifier mutator:• On every interface, class,

method, field, etc…• public → protected• protected → (package)• (package) → private

93 https://www.flickr.com/photos/neesam/28674222645/

CONCLUSION

94

https://www.reddit.com/r/funny/comments/35vomd/oh_the_irony/95

Useless Unit Tests






assertFalse(square.isInside(0, 1.01));

assertFalse(square.isInside(1.01, 0));

assertFalse(square.isInside(0, -1.01));

97

JavaZone 2016

www.slideshare.net/filipvanlaenen/dial-m-for-mutation

Filip van [email protected]@filipvanlaenen

Questions?Comments?

Thanks!

dial m for mutation

Software