automated tests - facts and myths

Automated testsFacts and myths

/ Wojciech Sznapka @sznapka

Cześć

An introductionI work in software industry for about 9 yearsCare a lot about robust and testable architecturesLoves software craftsmanship, sophisticated architectures, Big Dataand ice hockey

Test DrivenDevelopment

What is this?

Add a test

Write an implementation

Refactor code

Repeat!

What'simportant here?

... to have tests

You won’t go to hell if you’llwrite a test after declaring an

interface or prototyping aclass ...

... but you’ll surely end up inhell, if there won’t be a test

coverage

Myths aboutautomated tests

Boss doesn't payfor automated tests

It's a myth!A 2005 study found that using TDD meant writingmore tests and, in turn, programmers who wrote

more tests tended to be more productiveby American Scientists

But this ain't that hard ...class RomanConverter{ protected $conversions = [ 1000 => 'M', 900 => 'CM', 500 => 'D', 400 => 'CD', 100 => 'C', 90 => 'XC', 50 => 'L', 40 => 'XL', 10 => 'X', 9 => 'IX', 5 => 'V', 4 => 'IV', 1 => 'I'];

public function convert($inArabic) { if (!is_numeric($inArabic)) { throw new \InvalidArgumentException('I convert numerics'); } if ($inArabic <= 0) { return ''; } list ($arabic, $roman) = $this->conversionFactorFor($inArabic);

return $roman . $this->convert($inArabic - $arabic); }

protected function conversionFactorFor($inArabic) { foreach ($this->conversions as $arabic => $roman) { if ($arabic <= $inArabic) { return [$arabic, $roman]; } } }}

... start from obvious thingsclass RomanConverterTest extends PHPUnit_Framework_TestCase{ public function testEmpty() { $this->assertEquals('', (new RomanConverter)->convert('')); }}

... provide some meat ...class RomanConverterTest extends PHPUnit_Framework_TestCase{ /** @dataProvider provideTestData */ public function testConversions($arabic, $roman) { $converter = new RomanConverter(); $this->assertEquals($roman, $converter->convert($arabic)); }

public function provideTestData() { return array[ [3497, 'MMMCDXCVII'], [1, 'I'], [2, 'II'], [6, 'VI'], [9, 'IX'], [40, 'XL'], [45, 'XLV'], [90, 'XC'], [100, 'C'], [400, 'CD'] ]; }}

... test edge casesclass RomanConverterTest extends PHPUnit_Framework_TestCase{ /** @expectedException \InvalidArgumentException */ public function testEmpty() { (new RomanConverter)->convert('wtf I am passing here')); }}

Help yourself with tests generationphpunit-skelgen --test RomanConverter

... or using Symfony'sXSolveUnitSkelgenBundle

./app/console xsolve:skelgen:test Acme/ExampleBundle/Service/.

./app/console xsolve:skelgen:test Acme/*/Controller/*

We are buildingtoo complicated

systemfor an automated tests

TDD forces you to writecleaner code

It's easier to test smaller unitsof code

so you write smaller classeswhich are less coupledand that makes your system more stableand open for an extension in the future

Your Object Oriented designbecomes S.O.L.I.D. compilant

and this is awesome!

You will of course tacklecostly dependencies

Use PHPUnit's mockframework or Mockery

Fake it till you make itclass MockedTest extends PHPUnit_Framework_TestCase{ public function testNonExistingValueObjects() { $configuration = \Mockery::mock('\ConfigurationValueObject', [ 'getUrl' => 'http://tdd.sznapka.pl', 'getFormat' => 'xml']); $this->assertEquals('xml', $configuration->getFormat()); // OK }}

Mock thingsthat can't be tested quickly or non-

reproducableclass MockedTest extends PHPUnit_Framework_TestCase{ public function testApiCalls() { $buzzMock= \Mockery::mock('\Buzz\Browser'); $buzzMock->shouldReceive('get') ->andReturn('<response><mood>Awesomity</mood></response>');

// this also is wise solution, to write xml fixtures in file // $buzzMock->shouldReceive('get')->once() // ->andReturn(file_get_contents(__DIR__ . '/fixtures.xml'));

$api = new \ApiConsumer($buzzMock); $api->assertEquals('Awesomity', $api->getCurrentMood()); // OK }}

Expect declared behaviorsclass MockedTest extends PHPUnit_Framework_TestCase{ public function testExpectationsDeclarations() { $buzzMock= \Mockery::mock('\Buzz\Browser'); $buzzMock->shouldReceive('get') ->andReturn('<response><mood>Awesomity</mood></response>');

$loggerMock = \Mockery::mock('\Monolog\Logger'); // we just want to be sure that Logger::info was called only once $loggerMock->shouldReceive('info')->once();

$api = new \ApiConsumer($buzzMock, $loggerMock); $api->assertEquals('Awesomity', $api->getCurrentMood()); // OK }}

Be prepared for failuresand check if you prepared for unexpected situations

class MockedTest extends PHPUnit_Framework_TestCase{ /** @expectedException \MyExceptionWrapper */ public function testFailedConnection() { $buzzMock= \Mockery::mock('\Buzz\Browser'); $buzzMock->shouldReceive('get') ->andThrow('\Buzz\Exception\ClientException');

$loggerMock = \Mockery::mock('\Monolog\Logger'); $loggerMock->shouldReceive('info')->never(); $loggerMock->shouldReceive('err')->once();

(new \ApiConsumer($buzzMock, $loggerMock))->getCurrentMood(); }}

We providingAPI for external

consumersit can't be tested...

Use Symfony's WebTestCaseto test your API

I call it integration tests

Call your API and check if itreturns prepared data

class ExpenditureControllerTest extends WebTestCase{ use IsolatedTestsTrait; // it resets test environment

public function testGetListInJson() { $client = static::createClient(); $client->request('GET', '/expenditures.json'); $json = json_decode($client->getResponse()->getContent());

$this->assertTrue($client->getResponse()->isSuccessful()); $this->assertCount(80, $json); $this->assertGreaterThanOrEqual( new \DateTime($json[79]->created_at), new \DateTime($json[0]->created_at)); }}

Use fixtures and resetenvironment

IsolatedTestsTrait should do the trick

Steps required to effectivelyrun in isolation

1. configure PDO SQLite in file2. create database3. drop schema4. load fixtres5. copy database as a backup6. copy database from backup for every test7. delete database backup after test suite

Requirementschanges

frequentlywe can't keep up with unit

tests

Use behavioralapproach

Behat in PHP

Describe features as scenariosIt will be readable for: business, developers and machines

Perfectly fits into Agileprocess

Simple exampleFeature: look for a job In order to find cool job As an aspiring programmer I need to be able to list job offers

Scenario: list offers for PL version Given I am on "/" Then I should see "Dołącz do teamu" And click "Dołącz do teamu" Then I should be on "/kariera" And I should see "PHP Senior Developer (Gliwice)"

Scenario: no offers for EN site Given I am on "/en" Then I should not see "Dołącz do teamu" And I should not see an "#join-us" element

Our tests areslow!

This could be true ...

Always use in-memory sqlitedatabase

Or create clean sqlite database and copyit for every test

Group your testsUse PHPUnit's @group or Behat's @tag

Create "smoke tests" groupsThose should be fast test, which ensures your system is most likely

stable

Slower tests should run duringnight

Facts

It gives you confidenceabout changes and your code

Team is able to rapidlyexperiment with code

TDD enforces betterObject Oriented design

Smaller units of code and lower coupling always leads to betterunderstanding of the codebase by future devs

End user experiences betterquality

lower 500 error ratio in the production

Happier users↓

more $$$ in future

Conclusion

Setting up an workingenvironment for automated

testsis timely costly at the begining, but it pays off in the future

System without any kind ofautomated tests

has big potential to be a big ball of mud

Good coverage can be easilyachieved with

mix of unit, functional and behavioral tests

You need to build andcultivate

TDD culture in your surrounding

Thank you so much forattending!

Feedback is much appreciated

wojciech@sznapka.pl

Twitter: @sznapkaGitHub: @wowo

Join team :-)XSolve