unit testing in silverstripe

UNIT TESTING IN SILVERSTRIPEA couple of tricks you might not know about

Ingo Schommer (@chillu), October 2010

Wednesday, 17 November 2010

MORE EXPRESSIVE ASSERTIONS

Bad: Generic assertion, little debugging value

<?phpclass MyFileCreatorTest extends FunctionalTest { function testFileGeneration() { $obj = new MyFileCreator(); $filepath = $obj->createFile(); $this->assertEquals(file_exists($filepath), true); } }

1) MyFileCreatorTest::testFileGenerationFailed asserting that <boolean:true> matches expected <boolean:false>.



Good: Use assertions that communicate failures better

<?phpclass MyFileCreatorTest extends FunctionalTest { function testFileGeneration() { $obj = new MyFileCreator(); $filepath = $obj->createFile(); $this->assertFileExists($filepath); } }

1) MyFileCreatorTest::testFileGenerationFailed asserting that file "/my/path/file.txt" exists.



Useful built-in assertions:

assertType(<string>, <object>)assertArrayHasKey(<key>, <array>)

assertContains(<needle>, <haystack>)

assertEmailSent($to, $from = null, $subject = null, $content = null)assertDOSContains($matches, $dataObjectSet)

Useful custom assertions in SapphireTest:



$members = DataObject::get('Member');$this->assertContains('[email protected]',$members->column('Email'));

$member = DataObject::get_one('Member', '"Email" = \'[email protected]\'');$this->assertType('Member', $member);

Using assertType()

Using DataObjectSet->column()


PHPUNIT EXECUTABLE INSTEAD OF SAKE

New feature since 2.4.3, backported to 2.3.9 as well

More features than proxying through “sake” CLI tool

Code coverage generation in “XML Clover” format(which can be parsed by phpUnderControl and other tools)


PHPUNIT EXECUTABLE INSTEAD OF SAKE

With “sake”

phpunit .phpunit sapphirephpunit sapphire/tests/DataObjectTest.phpphpunit --coverage-html assets/

sake dev/tests/allsake dev/tests/modules/sapphiresake dev/tests/DataObjectTestsake dev/tests/coverage

With “phpunit”


PHPUNIT: RUN ALL TESTS IN A FOLDER

phpunit --verbose sapphire/tests/securityPHPUnit 3.5.3 by Sebastian Bergmann.

sapphire/tests/security BasicAuthTest ....

GroupTest ...

Time: 9 seconds, Memory: 83.50Mb

OK (72 tests, 286 assertions)

More granular control over which tests are run

Filepath autocompletion built-in on CLI


PHPUNIT: RUN ONLY ONE TEST METHOD

Less hacky than commenting out other tests

Supports regular expressions

phpunit --filter testDelete sapphire/tests/DataObjectTest.phpPHPUnit 3.5.3 by Sebastian Bergmann.

.

Time: 1 second, Memory: 64.75Mb


FASTER TESTS THROUGH SQLITE3

Up to 10x faster test execution¹ means “Test Driven Development” (TDD) is fun again! Thanks Andreas :)

phpunit mysite/tests '' db=sqlite3...

Time: 2:44 seconds, Memory: 113.75Mb

phpunit mysite/tests...

Time: 22:56, Memory: 116.75Mb

PostgreSQL Sqlite3

¹ Internal project on SilverStripe 2.4.2, with 97 tests and 410 assertions.Postgresql and SQLite3 modules both on trunk.


FASTER TESTS THROUGH SQLITE3

Needs the sqlite3 module in your projecthttp://silverstripe.org/sqlite-database/

Simple modification in mysite/_config.php

require_once('conf/ConfigureFromEnv.php');// ...if(Director::isDev() && @$_REQUEST['db'] == 'sqlite3') { $databaseConfig['type'] = 'SQLite3Database';}

Not a complete replacement to running tests on theactual database driver, but most of the time implementations are the same.


http://silverstripe.org/sqlite-database/

http://silverstripe.org/sqlite-database/

DEBUGGING A FIXTURE

YAML fixtures are more concise than SQL inserts

Problem: You have to run queries and joins in your head


DEBUGGING A FIXTURE

For more complex data models, looking at SQL can be easier

SilverStripe lets you load a YAML fixture into a temporary databasethrough http://localhost/dev/tests/startsession


SEPARATE INTEGRATION AND UNIT TESTS

A Unit Test (SapphireTest class) covers an isolated aspect of a class, e.g. one method call.

An Integration Test (FunctionalTest class) tests the sum of its parts,in SilverStripe mostly HTTP requests and their HTML responses.


SEPARATE INTEGRATION AND UNIT TESTS

Unit and Integration Tests complement each other,both are necessary and have different intentions.

Good practice: Separate tests into different folders.Example: mysite/tests/unit and mysite/tests/integration

Idea: Run coverage reports separately for unit and integration tests


WRITE QUALITY TESTS

Lack of test coverage points to problems,but high coverage doesn’t automatically mean quality code

class HomepageTest extends FunctionalTest { function testResponse() { $response = $this->get('home'); $this->assertNotNull($response->getBody()); $this->assertEquals($response->getStatusCode(200)); }}

Probably gives you 80% code coverage for a Homepage class,but failures don’t tell you anything useful.

Bad example:


WRITE QUALITY TESTS

High code coverage is the side effect of writing quality tests,not a goal in itself.

Recommendation: Strive for high unit test coverage,

and highlevel “smoke tests” through integration tests


WRITE FLEXIBLE INTEGRATION TESTS

<div class="sidebar"> <ul> <li class="news-item"> <a href="mylink">My first news</a> </li>  </ul></div>

class HomepageTest extends FunctionalTest { function testSidebarShowsLatestNews() { $response = $this->get('home'); $newsEls = $response->cssParser()->getBySelector('.sidebar .news-item'); $this->assertEquals(3, count($newsEls), 'Shows three items'); $news1LinkEls = $newsEls[0]->xpath('.//a'); // relative xpath $this->assertEquals( 'My first news', (string)$news1LinkEls[0], 'News item link contains correct title' ); }}

WRITE FLEXIBLE INTEGRATION TESTS

Learn XPath (or use a CSS to XPath converter)

Learn SimpleXML (built-in with PHP5)

Make few assumptions about template markup

Make assertions less brittle by using semantic markup


LINKS

Unit testing in SilverStripehttp://doc.silverstripe.org/testing-guide

Usage of “phpunit” executable in SilverStripehttp://goo.gl/D23KJ

Book: “Test Driven Development: By Example” (Kent Beck)


http://goo.gl/D23KJ

http://goo.gl/D23KJ

unit testing in silverstripe

Technology

generic assertion

expressive assertions

little debugging value

couple of tricks

ingo schommer